will a RAID0 array speed up filtering?

[Z-C]

Hi everyone.

I was wondering if a RAID0(or even a RAID1?) array of 2 SATA II Hard discs could improve the speed of noise filters? If so - then by how much?

[jagabo]

A little if you're using uncompressed video. Otherwise, no.

[lordsmurf]

No.

[Z-C]

Thank you.

And what about filterless encoding to mpeg 2/other lossy format?

[normcar]

Be very careful with Raid 0. If one disk goes bad, you lose all your data.

[Z-C]

I'm aware of that But I'm just trying to squeeze some more performence.

[mats.hogberg]

A/V filtering/encoding speed is almost 100% CPU speed relative. Other factors are so small, it's not even worth considering.

/Mats

[jagabo]

And what about filterless encoding to mpeg 2/other lossy format?

Again, if your source is uncompressed video you will get a small improvement. Most of the time is spent by the filtering and/or compression code.

Think of it like this: how long would it take to copy an hour long video file? Obviously it depends on how compressed the video is to start with but let's say it takes 1 minute. How long does it take to filter and/or compress that video? Again, this depends on what your doing but let's say it takes 60 minutes. That filter/compression includes the time to read the source file and write the output file -- essentially the copy operation. So if the time to read and write the video data was reduced to zero with an infinitely fast drive, you would still have 59 minutes to filter/encode.

[Z-C]

Thank you guys

Since my cpu never reaches 100% usage on encoding(it's usually somewhere between 40-80%), I was thinking that the HD is the limiting factor, or else the cpu would have used all of it's power to encode.

[jagabo]

Since my cpu never reaches 100% usage on encoding(it's usually somewhere between 40-80%), I was thinking that the HD is the limiting factor

What CPU do you have? 40 percent is very low. On a single core CPU you should always see nearly 100 percent. On a dual core CPU it will vary from 50 to 100 percent depending on how well the code is multithreaded. For example, on my Core 2 Duo system TMPGEnc Plus usually runs at 100 percent, VirtualDUbMod with Xvid around 70 percent, CCE around 80 percent.

Here's an AVISynth/VirtualDubMod/Lame/Xvid encode:

[chesterfield]

RAID 0 is great. I have a mahine that runs 24/7 for two years with two seagate 160 gig SATA drives runniing raid 0. that is not the machine in my profile but it has a P4 2.8 Prescott, 2 gig PC4200 memory, ASUS p800 MOBO--runs like a top.....

[edDV]

RAID 0 is great. I have a mahine that runs 24/7 for two years with two seagate 160 gig SATA drives runniing raid 0. that is not the machine in my profile but it has a P4 2.8 Prescott, 2 gig PC4200 memory, ASUS p800 MOBO--runs like a top.....

If either of those two seagate 160 gig SATA drives fail, you have lost all 320GB of data on both drives. The risk is great.

RAID 0 should not be used unless you have a need for > 40-55 MB/s sustained rates accross the full drive. The only video that runs that fast is fully uncompressed YUV or SMPTE 240M (SDI). Huffyuv compression solves the uncompressed YUV problem by losslessly lowering datarates into single drive range*. DV, HDV and MPeg2 run with comfort on a single drive even on a slow laptop drive at high definition rates.

The only other RAID 0 advantage is slightly faster file copy (limited by either the source or target drive max speed). Gamers like to put the OS and games on a RAID 0 to speed game play. That is foolish for anything except a dedicated game machine.


* single drives are faster at one end of the drive. If you put a capture partition on the fast end of the drive, huffyuv will probably be sufficient for uncompressed video. Two drive RAID 0 will average the speeds by running one drive starting at the fast end and the other at the slow end. This allows a near constant sustained rate over the full drive capacity.

[chesterfield]

RAID 0 is great. I have a mahine that runs 24/7 for two years with two seagate 160 gig SATA drives runniing raid 0. that is not the machine in my profile but it has a P4 2.8 Prescott, 2 gig PC4200 memory, ASUS p800 MOBO--runs like a top.....

If either of those two seagate 160 gig SATA drives fail, you have lost all 320GB of data on both drives. The risk is great.

RAID 0 should not be used unless you have a need for > 40-55 MB/s sustained rates accross the full drive. The only video that runs that fast is fully uncompressed YUV or SMPTE 240M (SDI). Huffyuv compression solves the uncompressed YUV problem by losslessly lowering datarates into single drive range*. DV, HDV and MPeg2 run with comfort on a single drive even on a slow laptop drive at high definition rates.

The only other RAID 0 advantage is slightly faster file copy (limited by either the source or target drive max speed). Gamers like to put the OS and games on a RAID 0 to speed game play. That is foolish for anything except a dedicated game machine.


* single drives are faster at one end of the drive. If you put a capture partition on the fast end of the drive, huffyuv will probably be sufficient for uncompressed video. Two drive RAID 0 will average the speeds by running one drive starting at the fast end and the other at the slow end. This allows a near constant sustained rate over the full drive capacity.

i have had no problems except for the people on this site who keep dissing RAID 0

BTW--a 250 gig drive backs up all important data twice a week, in the middle of the night on its own

[edDV]

RAID 0 is great. I have a mahine that runs 24/7 for two years with two seagate 160 gig SATA drives runniing raid 0. that is not the machine in my profile but it has a P4 2.8 Prescott, 2 gig PC4200 memory, ASUS p800 MOBO--runs like a top.....

If either of those two seagate 160 gig SATA drives fail, you have lost all 320GB of data on both drives. The risk is great.

RAID 0 should not be used unless you have a need for > 40-55 MB/s sustained rates accross the full drive. The only video that runs that fast is fully uncompressed YUV or SMPTE 240M (SDI). Huffyuv compression solves the uncompressed YUV problem by losslessly lowering datarates into single drive range*. DV, HDV and MPeg2 run with comfort on a single drive even on a slow laptop drive at high definition rates.

The only other RAID 0 advantage is slightly faster file copy (limited by either the source or target drive max speed). Gamers like to put the OS and games on a RAID 0 to speed game play. That is foolish for anything except a dedicated game machine.


* single drives are faster at one end of the drive. If you put a capture partition on the fast end of the drive, huffyuv will probably be sufficient for uncompressed video. Two drive RAID 0 will average the speeds by running one drive starting at the fast end and the other at the slow end. This allows a near constant sustained rate over the full drive capacity.

i have had no problems except for the people on this site who keep dissing RAID 0

BTW--a 250 gig drive backs up all important data twice a week, in the middle of the night on its own

RAID 0 requires a disciplined backup strategy. Gamers don't need a huge backup. Video users do. If you buy into that, then RAID 0 is fine. That is beside the point.

Explain the benefit you see by using RAID 0. Be specific.

[lordsmurf]

RAID-0 is also known to be too rough on the drives and is usually the reason RAID-0 arrays die.

RAID was invented for server situations, not workstations. It will not really do anything for a workstation (home desktops included), in terms of "speeding up" anything. The speed gains are minimal percent at a huge risk. And even then, most gains are imagined moreso than actual. Usually a station that could benefit from RAID-0 array is using SCSI 10000rpm drives in a redundancy RAID setup (not a lossy 0 array).

Also, hate to be the grammar nanny here, but "dissing" is not a real word and shows lack of education. I wish people would quit using it. The root of that word is "disrespect", which is a bastardized ebonics usage from thug/rap culture. The proper word is "insult" or some other synonym.

To answer the original poster, again, such an array will not speed anything up, and will greatly increase the chance of crashing one or both drives.

[Scorpion King]

i have had no problems except for the people on this site who keep dissing RAID 0

I wouldn't get too shook up over that, ok. It won't help. lol

Personal Experience:
I've ran Raid0 arrays for years and I guess as long as I can afford to run and own them I probably will continue to do so. Once you get used to the performance and the reliability it has to offer you won't go back either.

I don't capture to single drives and I don't use single drives for my video projects probably because I don't haft to. I run both IDE raid0 and SATA Raid0 arrays on this system. And both without any, yes I did say without any, problems what-so-ever.

I'm happy with my raid arrays and that is all that matters to me. If someone is not happy with raid then don't use it. Pretty simple huh.

Will it speed up filtering? If filtering involves hdd reads & writes then yes it will, but like jagabo said above - - not much when filtering. But you let VD direct stream copy then you better look out....

Good luck.

edit

edDV wants to know what advantages raid0 offers and I had a thought, ok. We all have a little program already on our computers called 'auxsetup.exe'. It is located in the VirtualDub folder and comes with the program. Run the 'benchmark' routine in auxsetup.exe and post the results. Pretty simple.

you first.

[edDV]

...
I'm happy with my raid arrays and that is all that matters to me.

I was a user of RAID 0 as well, back when it was the only way to get a SCSI drive to work with 8MB/sec (64Mb/s) MJPEG which was common for NLE in ~1997. That setup cost me about $2000 for 18GB (about 47 minutes of video). The risks of RAID 0 were accepted as the only way to get the job done.

When DV format hit in 1998 everything changed. Now equal to higher quality (endlessly debated) video could be captured at 3.5MB/s with minimal CPU loading. ATA66 sealed the deal for single drive transfer/capture of DV format @~$400. This made reasonable quality video possible on th Mac or PC at reasonable prices.

As drive speed increased to ATA 100, huffyuv compression made uncompressed capture practical without RAID.

Pro SDI editing environments need multi channel synchronized SDI feeds (240Mb/sec each). RAID 0 is still used there. Otherwise it has fallen to the extreme gamer market where fast texture loading and crash reboots are the bottleneck.

[edDV]

edDV wants to know what advantages raid0 offers and I had a thought, ok. We all have a little program already on our computers called 'auxsetup.exe'. It is located in the VirtualDub folder and comes with the program. Run the 'benchmark' routine in auxsetup.exe and post the results. Pretty simple.

you first.

Let me get this straight. You like to risk massive data loss in order to win at a 320x240 benchmark? Give me a practical example. My drives test in the 45-58MB/s range but I only need half that.

You may be lucky that you have avoided a disk crash but they do happen and most of us are not good about backups. It's one of those things you need to experience first hand to understand.

As long as single drives work fine, why add risk?

[RabidDog]

I thought RAID stood for redundant array of inexpensive disks!

That setup cost me about $2000 for 18GB (about 47 minutes of video). The risks of RAID 0 were accepted as the only way to get the job done.

I suppose its all relative. I dont really see the benefit of raid unless your using at least four drives.

[edDV]

I thought RAID stood for redundant array of inexpensive disks!

That setup cost me about $2000 for 18GB (about 47 minutes of video). The risks of RAID 0 were accepted as the only way to get the job done.

I suppose its all relative. I dont really see the benefit of raid unless your using at least four drives.

It does. Origially "independent" then "inexpensive". That is what you had to pay in 1997 to store 47 minutes of good quality video. The big SCSI drives were far more expensive.

http://en.wikipedia.org/wiki/Redundant_array_of_independent_disks

[chesterfield]

Also, hate to be the grammar nanny here, but "dissing" is not a real word and shows lack of education. I wish people would quit using it. The root of that word is "disrespect", which is a bastardized ebonics usage from thug/rap culture. The proper word is "insult" or some other synonym.

so now lordsmrf is a "grammer nanny". some times LS, you need a dose of STFU and stop being so criticle of people not in your "cliqe".....you have been a tremendous help to many in many different ways, but your beside manor leaves much to be desired for....is that a reason you are no longer a mod??????

[Z-C]

Since my cpu never reaches 100% usage on encoding(it's usually somewhere between 40-80%), I was thinking that the HD is the limiting factor

What CPU do you have? 40 percent is very low. On a single core CPU you should always see nearly 100 percent. On a dual core CPU it will vary from 50 to 100 percent depending on how well the code is multithreaded. For example, on my Core 2 Duo system TMPGEnc Plus usually runs at 100 percent, VirtualDUbMod with Xvid around 70 percent, CCE around 80 percent.

Here's an AVISynth/VirtualDubMod/Lame/Xvid encode:

I played with the decoders settings and was able to 'maximize' the cpu usage to 70% avg. it now varies from 50%-90%.





By the way - I just wanted to say that I didn't have the time to check back in yesterday, and I'm amazed to see the interesting discussion that developed here.
If it matters, I was convinced to neglect the idea of RAID0, due to this thread.

[normcar]

RAID 0 is great. I have a mahine that runs 24/7 for two years with two seagate 160 gig SATA drives runniing raid 0. that is not the machine in my profile but it has a P4 2.8 Prescott, 2 gig PC4200 memory, ASUS p800 MOBO--runs like a top.....

RAID 0 is great until one of the drives goes bad, and you lose all data on both drives. You can get benefits from RAID 0 (how much depends on application software, etc), but understand the possible problems of data loss of more than one disk. Data recovery software may also not be as effective at recovery on RAID 0, because of greater possibility of hardware failure of one of the disks. With RAID 0, always assume your data is at greater risk, and plan for it.

[chesterfield]

this is my last post on this thread....i have been using RAID 0 for years without a SINGLE problem. bottome line, if you know what you are doing then go for it....i would bet the people that keep bashing RAID 0 have never used it......just read about it......

peace out....

[edDV]

this is my last post on this thread....i have been using RAID 0 for years without a SINGLE problem. bottome line, if you know what you are doing then go for it....i would bet the people that keep bashing RAID 0 have never used it......just read about it......

peace out....

I used RAID 0 back when it was the only choice. The data risks are obvious. Make sure anything important on that 320GB is backed up.

Nobody is saying RAID 0 doesn't work. There just isn't much benefit in using it.

If you think hard drives don't fail, you will probably learn the hard way.

[Cornucopia]

I've been working with RAID0 drives since '95, and they've performed remarkably well.

BUT...

There have been 2 or 3 drives that have gone bad in that time (in all fairness, there have probably been a couple of un-Raided drives that have gone bad in that time, too). Of course, all the stuff on them was either backed-up or was easily batch redigitizable (from tape). I wouldn't chance it any other way.

As long as you know this and can plan for it, RAID can be fine. I routinely do completely uncompressed stuff, and some HD stuff--WITH MUTLIPLE LAYERS--and for that it starts to make a lot of sense. Never once had a "dropped frame" that people so often complain about here. Of course, since I've got it on the same system that does some DV-compressed stuff, it benefits from it too. and File copy is noticeably faster.

Scott

[Scorpion King]

I think I need to look at all the vital information given in the thread so I, and any one else interested, can make the decision to use raid or not to use raid.

Lets see...

normcar says - you lose all your data
normcar says - you lose all data

lordsmurf says - RAID-0 is also known to be too rough on the drives and is usually the reason RAID-0 arrays die.
lordsmurf says - It will not really do anything for a workstation
lordsmurf says - most gains are imagined moreso than actual.
lordsmurf says - such an array will not speed anything up

OK you guys win....... I'll use RAID.

Thanks
Good luck.

[edDV]

I don't think anyone is saying RAID 0 increases drive failure. Maybe it does. If a normal drive failure happens, data recovery is difficult to impossible because data is spread over two or more drives and there are no reliable recovery utilities.

If one understands the technology and frequently backs up, then the drives can be replaced and work can continue from the backups.

For the typical home user, RAID 0 has few benefits unless you are doing realtime multilayer work from uncompressed data or are an extreme gamer. Raid 0 falls in the "don't try this at home" category because typcal users don't back up their data.

Since DV format arrived in 1998 and single SCSI and then ATA drives became fast enough there was no longer a need to bother with RAID 0 unless you were into realtime multi-layer effects processing. I've had no problem using RAID 0 for dedicated video drives because all data is backed up to tape before capture.

People doing this should know the risks and take precaution.