Matching your Hard Disk for Perfect Capture: A Primer

[halldav3]

Ahhh, the pursuit of the perfect capture. One of the major contributors to dropped frames is the hard drive. Ever wonder why your new system drops more frames than your old one? Or why that new 7200 RPM drive is dropping frames left and right? It is my hope that through this thread I'll be able to demystify the use of a hard drive for video capture and help everyone in how they choose and setup their drives. Let's review what I have gleaned from the threads I have read and been involved in concerning dropless capture:

* Stay away from WinME, its a pig. Win2000 or maybe WinXP is the way to go.
* Get the fastest CPU you can afford
* Use seperate drives for the OS and the capture drive
* Defrag your drives
* Use a 7200RPM drive for capture
- This is where I really have to question the conventional wisdom. During the capture of your video stream, the drive should be writing a continuous, sequential stream of data (assuming you have defragged). In cases where the disk data rate exceeds the video data rate, the biggest issue is in keeping the drive buffer from completely emptying. When the drive buffer empties, the drive must stop writing and wait an entire revolution before restarting. During this time, the system should be able to refill the buffer. Now, if you are blowing lots of revolutions, a 7200 RPM drive is better since it takes less time to finish that missed revolution. However, a 7200 RPM typically has a higher data rate, resulting in more missed revs. The key to all of this is that if the drive buffer ever completely FILLS (this would only happen during a missed rev assuming a faster disk data rate) and the OS has no mechanism for buffering additional data, you will drop frames. Therefore the most important qualities of a drive are:

* A data rate that is higher, but CLOSE to your video data rate.
* A large drive buffer
* An optimized set of OS settings for keeping or delaying as long as possible the buffer from completely emptying OR filling

Now, I am no OS expert and so I would like to hear from any of you regarding ways to tune drive buffer settings. I am especially curious as to how using DMA affects the ability of the OS to buffer data in excess of the drive buffer capacity.

Next, I would like to suggest that certain areas of your capture disk will be more suited to capturing certain video streams, depending on the video data rate. As I said before, it is good to match your video data rate to the disk data rate. All modern drive use multi-zone variable rate recording. This is nothing more the a fancy way of saying the recording rates are not the same across the entire disk. The highest data rate occurs at the outer diameter of the disk and the lowest at the inner diameter. LBA 0 is typically at the outmost customer cylinder and the max. LBA is at the innermost customer cylinder. Most OSs start writing at LBA 0. Therefore, high rate streams should be captured at the outer diamater and lower rate ones at the inner diameter. Let's examine a few scenarios:

352 * 240 * 24 bits/pixel = 2.02 Mbits/frame * 30 frames/sec = 60.82 Mbits/sec = 7.60 Mbytes/sec
640 * 480 * 24 bits/pixel = 7.37 Mbits/frame * 30 frames/sec = 221.1 Mbits/sec = 27.6 Mbytes/sec

With variable-bit schemes, things become a little more complicated, but average bits/pixel should work fine.

You should be able to get your drive specs from your drive manufacturer to find the best match for your stream. The question of how best to get your streams into the correct recording zone is left to the reader. You could possibly fill your drive with files until you get to the correct zone and create a file which you always use for capture. The fill files can probably then be deleted. Files could be created for each target recording rate.

Regarding RAID striping:
It is my opinion that it is not the higher overall datarate that helps you here, just the effective doubling of the buffer size. In fact, a striped system will blow revs at twice the rate of a single drive system.

Question: Can data interleaving (i.e., write every 2nd or 3rd block) still be done through modern OSs or drive mode pages? I am not an ATA drive expert (I work primarily with SCSI), so I'm not sure what options are available.

I'm sure I haven't covered everything, so please ask questions and I'll be happy to expand or clarify anything I have said. Let's get the HD out of the equation!

Dave

[holistic]

thanks for the info dave.

This info would have been great 12 months ago when i was putting my video capture machine together. But it seems ........... I GOT LUCKY

Current machine is : ABIT BE6-2 , Celeron 600 (O/C 900), 384Bm RAM and a Maxtor 30GB ATA 100 7200 spin drive. Since the mobo only supports ATA66 that's what i get.
The drive has been benchmarked at an average 28 "+ ish" Mb/sec. Just on the edge for full frames.

O/S is win98SE , under the 'performance' ,'file system' tab of system properties i have modified my machine to 'network server' and selected 'read ahead' to NONE.
Seems to work - Suggestions ?!

The oddity with this setup is that windows is convinced that my IDE controller (Hi-Point tech ATA66 controller) is SCSI - imagine that ! - Any ideas here ????
This results in NO - DMA tab in 'device manager' BUT dosen't seem to effect overall performance as the benchmarks have shown
BURST - MAX 55 Mb/sec
SUSTAINED AVG 28+ Mb/sec

Am curious as to the layout of data on a partitiond drive.
I obtain the best captures on my 'D' partion but drop frames on my outer 'H' partion. Going by the information you presented do i extrapolate that the drive heads start their 0 sector at the outer circumfrence?

thanks

[VIDEOHOUND]

Very interesting. I have read somewhere before that you should change your configuration from "Desktop Computer" to Network Server. How would you go about making the Hard Drive read from the outside track? And in addition, what would be a good benchmark program to test my hard drive? Thanks

[aykh]

Does this also apply to SCSI H/D?
Is it true that SCSI=no frame drop?

[TIbrO]

Another way to ensure you will drop no frames, or considerably less than you did, is with AVI_IO. It's excellent. In the options it has a check box that says "align to disk sector", so I assume that this program is designed with the way hardrives work in mind...

It actually aligns the frames you captured to a disk sector, and it has it's own buffers and all that jazz. You can capture anything with minimal drops in that program. I personally drop 0.

[VIDEOHOUND]

This was derived from the RagePro page mentioned in this string.

Much frustration can be avoided when capturing video by preparing/planning ahead for the massive data-stream involved.

1. Have A Dedicated Hard Drive
If you don't run a SCSI or IDE RAID to capture video to, you can still get reasonable results with a large (15-75gig) 7200rpm IDE Hard Drive. The target HDD should "Master" its own IDE channel,
(Don't stick it to slave on the same channel as your CDROM
and DMA must be enabled to maximize data throughput with a minimum of intervention from your CPU.
Think of the data-stream as a very fast conversation that can't be "interrupted" by other devices, or the result is dropped frames.
Attempting to capture to the HD/IDE Chan your OS is on is asking for grief.

2. DE-FRAG

Attempting to capture to a fragmented hard drive results in dropped frames.
Using a dedicated disk exclusively for video caps saves alot of time/headache; once you are finished with the source video clips simply/quickly reformat the drive and you're ready for the next project.

3. CLEAN BOOT
Interrupts from other apps running during capture will result in dropped frames. This can be avoided by disabling all system TSR's such as Virus-scan, Networking, Hardware Monitoring software etc. The vid-cap software should have 100% of system resources.

4. COMPRESS NOW, OR LATER
Raw AVI in full screen resolution will consume over 1gig of hard drive real estate A Minute. Most any P3/Athlon system is capable of directing the data-flow, but not every IDE hard drive <very few> can handle sustained data transfer over 24mps. On-the-fly AV compression is CPU intensive <the most demanding computer app there is>. If you are "CPU-Challenged", but have plenty/tons of disk space, you can get excellent results by using a software CODEC to compress your clips later <while you sleep>

If on the other hand, you blew your wad on CPU and forgot about the hard drive space you'd need, compress the ****e out of it on the fly and keep the CDRW's handy

[halldav3]

VIDEOHOUND - I would recommend iometer for benchmarking. It can be found here:

http://developer.intel.com/design/servers/devtools/iometer/

Note that it only runs on NT, but as all of us should soon be moving to XP (those running 98 or ME that is), it should work there. Be very careful when using this program as it has no problem in wiping your system disk if you don't know what you are doing.

aykh - I don't know that I would equate scsi=no FRAME drop, but it does have several advantages for video capture, namely large data buffers and fast track and head switch times. If you are writing very small packets of sequential data though, it will burn through it in a heartbeat, leading to blown revs.

holistic - your first partition is usually at the outer diameter (OD) of the disk. However, the OS does not control this, the drive does. The OS only deals with Logical Block Addresses (LBAs) and it is up to the drive to translate this to a physical location. That is a very interesting idea though ... creating different partitions for different capture rates.

I'm still looking for any feedback on OS tweaks for tuning buffer access.

[sherman]

With all this in mind how about an opinion on how you would
alocate 2 7200 Hd's, a DVD ROM, and a CD burner among 2 IDE controllers.

I'm new at this capturing business and am in the process of assembling a new computer.

Sherman

[holistic]

That would depend on the motherboard.
Mine (Abit BE6-2)has for 4 IDE (ATA33) devices and 2 ATA66 devices.

My setup is as follows : ata33 5200spin 3GB drive on "PRIMARY IDE" - mastered - Running my operating system (wincrap98se)
CDRW on "SECONDARY IDE" - mastered, DVD-ROM on "SECONDARY IDE) - slaved
ATA100 (only working at ATA66 due to mobo) 7200 30GB capture drive on the ATA66 controller - mastered
[drive has been partitioned into 7 GB sizes] this is not essential . I have just found this easiest to work with : i.e - i capture to 3 and enocode to the others. format and away i go again.



** If you only have 2 controllers 4 IDE channels
Suggest putting operating system on a different drive and slave the DVD-ROM off it.
Master the capture drive and slave the burner.

If you are assembling a new 'puter look for a motherboard with on board ATA100 and even RAID is a nice option.
Save some $$$ for at least 30Gb , 7200 spin drives. A 'RAIDED' EIDE system will be sufficient for fullscreen captures and come in much cheaper than SCSI.
Will not get is a Pi$$ing contest on the AMD - INTEL debate but 1Ghz is plenty 256Mb PC-133 RAM enough also. CPU / RAM will assist in faster 'realtime encodes' and post production.
Just an opinon..............

[wildcatfan]

One thing I would like to clear up....while 7200 rpm drives GENERALLY outperform 5400 rpm drives that is not always the case. In fact if you compare a newer 5400 to a slightly older 7200 you might find the 5400 rpm drive to be faster. That is because you cannot overlook areal density and the factor it has on the transfer rate of the harddrive (same basic principle as the sector size of fat16 vs fat32 vs ntfs...blah,blah,blah). So unfotunately the only way to truly test a drives speed is buy it and do it yourself, or better find a web review of it where they test the drive.

Michael

[holistic]

On 2001-09-11 02:17:46, wildcatfan wrote:
One thing I would like to clear up....while 7200 rpm drives GENERALLY outperform 5400 rpm drives that is not always the case. In fact if you compare a newer 5400 to a slightly older 7200 you might find the 5400 rpm drive to be faster. That is because you cannot overlook areal density ............

Excellent point - one i missed !

[halldav3]

wildcatfan,

Good point in general. Not to split hairs, but areal density is only loosely related to transfer rate. Areal density is defined as the linear bits per inch (BPI) times the tracks per inch (TPI). The only component significant to transfer rate is BPI. As long as I am on the subject, it bears pointing out the difference between instantaneous and sustained data rates. Instantaneous data rates refer to read/write speed while on a single track. At the end of each track however, the head must be aligned to the next track, which takes a fairly significant amount of time (~1/4 rev). To compensate for this, drive makers skew the start of each track from the end of the previous. During this 'dead-time' no data is being read or written. The datarate that takes these track switches into account is the sustained data rate and is usually the more useful number. For the really geeky amoung us, the drive also has to contend with cylinder switches, which occur when switching from the last track of a cylinder at the bottom of the disk stack to the first track of the next cylinder. Due to a condition known as stack tilt, the distance covered by the actuator may be several tracks. This means an extra long skew is used to account for this.

I would also like to retract my statement that 7200 RPM drives are generally faster than 5400 RPM in sequential performance. After I made the original post, I went to a number of drive manufacturers sites and it would seem that many of them are using the same read/write channels for both their 5400 and 7200 RPM drives. This is being done for cost reasons I sure. This means that the datarate is the almost exactly the same for both drives. To accomplish this, they crank down the bpi on the 7200 RPM drives relative to the 5400 RPM drives. For the same vintage of drive, 7200 RPM are only significantly faster in random operations.

Dave