Help - TMPEnc and *very* high disk I/O

[texneus]

Question......trying to convert some home movies to VCD to give to friends of our vacation. I captured footage from MiniDV via 1394 interface using IntroDV, edited it down to about 30 minutes, added titles etc. Then I exported it to MOV format (unfortunately exporting as MPEG is not an option) using Sorenson 3 video codec at "Best" quality and straight 16bit/44.1KHz PCM audio (no compression). I had the export resize the picture to the required VCD dimensions as well (can't recall off the top of my head but dimensions are as stated by TMPEnc). The resulting file is about 2GB.

Now using TMPEnc I start the conversion from MOV to VCD MPEG1 with all the default settings. This is what happens. First few minutes go fine. ETA is about 1 hour. At about the 4 minute mark (13% complete or so) it as slowed down noticeably and it's hitting the hard disk much more than it used to. Conversion rate is still acceptable. ETA is about 1 hour, 20 minutes. At the 5 minute mark (17% or so) the system has for whatever reason become I/O bound. HD I/O has increased to the point that TMPEnc slows to a crawl, the drive light is near solid with heads thrashing wildly about. ETA is now almost 2 hours and rapidly increasing. At the rate of exponential ETA increase (and out of fear for my HD's life, it's making a heck of a racket) I stop the conversion here because I'm not sure it will ever finish.

Is this normal? I've never done this before so I don't know. If not, what is/are the likely culprits? Should I try a different Codec (which?) or convert to AVI first? The only reason I picked Sorenson was because the IntroDV help files suggested this gives the best quality of what is available.

System Details available in my profile.

Other observations:
Drive activity appears to exponentially increase with conversion time, this is not something that just happens at the 5 minute mark.

I repeated it three times with the exact same results. During the second try I disabled all extra tasks (Norton AV, InCD, WebWasher, etc). For the third try I did a reboot AND disabled extra tasks. No effect.

Even after pressing the Stop button on TMPEnc, the wild disk I/O continues until the action is confirmed, even though TMPEnc does not appear to be doing anything except awating my at that point. Once confirmed the drive activity stops within a few seconds.

System resources remain ~70-80% from the start of the conversion until I stop it.

The original MOV file plays fine in QT Pro.

The resulting MPG file (though truncated) plays just with Windows Media Player.

[lattedfw]

It doesn't seem normal. I have two PCs, one being a 5-year old Pentium-2 266MHz with 128MB Ram and a total of 26GB hard drive (2 drives). I used to compress lots of large Quicktime files to MPEG1 (I edited everything on an iMac). The QT files were all DV codec and under 2 GB in size due to limitations tranferring files from the iMac to the PC. The drives would be busy, but usually not in constant use. I recall it taking about 3 hours to compress 7 - 9 minutes of video to MPEG1 at highest quality. I ran everything under Windows 2000. I really dislike Windows 98, especially its memory management. That would be one thing I would change (upgrade). It may also be your virtual memory is fragmented on your hard drive - try defragmenting it or moving it to a different disk or partition.

Also, try using a small, sample uncompressed QT file, or export it as uncompressed AVI. It could be the PC is swapping a lot of stuff in and out of memory for the Sorensen codec.

[texneus]

Yah, drives are "busy" at the start but far from constant use. Light probably flashes 2-3 times per second. I considered the swap file issue but if RAM falls off to the point it has to swap, won't that show in the system resources application as low memory? Swap file is permanent and contiguous and located at the font of the drive.

Hard to find fault with your analysis of 98 vs. 2000. Next time my 98 install starts to, uhhm, explode I'll probably look into 2000. My only issue is I use hardware and software that is, to put it kindly, very "legacy", so compatability and drivers may be an issue.

Just curious, what 2GB limitation are you running into? When downloading DV IntroDV wouldn't let me make files longer than 2GB. I assumed it to be a limitation with IntroDV. I cannot export the file uncompressed (assuming you mean a DV codec) for presumably the same reason. IntroDV mearly reports an I/O error and not only stops export but delets the file as well.

I am currently exporting with the Indeo 4.4 codec to see what happens. It's painfully slow (been working at 30 minutes of footage for 3 hours now, only half done).

[lattedfw]

My 2GB limitation is when transferring files between my iMac and either of my Windows 2000 PCs over my small home LAN. I use OS X on the iMac and map a drive to the PCs with the built-in Samba functionality (smb://serverhare). If I try to drag a file to the PC drive on the Mac, the transfer will bomb at 2GB. I'm pretty sure the iMac/OS X can natively handle files over 2GB, and I know Windows 2000 can. It's not a big deal now since I have a firewire card in my newest PC (1.5 years old) and do all the capturing there. I occasionally use iMovie to edit video, but mostly use the iMac to capture audio and use iTunes to create MP3's or WAV files. This is diverging from the issue at hand, but I use the Mac a lot for taking 78 RPM records and converting them to digital format. iTunes and iMovie are very good for that.

I understand the issue with legacy hardware and software. I won't jump to Windows XP because even my 'new' PC has some software and hardware that XP might not like (among other reasons). The newer machine is where I have my DVD burner as well, and I don't want to throw too much into the mix, since I'm just getting used to the DVD-R/RW world.

It takes me 56 hours on my 1.7GHz machine to compress 2 hours of MPEG2 with 2-pass VBR at highest quality (the drive doesn't grind away either). The files are AVIs with the contents being in the DV codec.

[texneus]

Interesting you mension 78s, I have a dedicated TT connected to my PC. It looks like a regular production studio over here.

Anyway...

Re: the Indeo codec. Quality was poor and export took for freaking ever. Same thing with Cinepack. Help file wasn't kidding about Sorenson quality. Way better than the others and faster to boot. In any case, no change in TMPGEnc performance. I successfully converted about 8 minutes of DV footage (MOV format) in 50 minutes with NO problems whatsoever. I don't get it. Right now I'm splicing my video into chunks. Will a DVD player play all files the continuously? I don't have one to try it out...

[lattedfw]

You will see a little break in video between tracks. Some DVD players are a better at it than others, but there will be a pause. There will be a delay even with MPEG2 on a DVD-R: I burned a DVD with two video tracks and there is a slight pause between the two (looked like a layer change on a commerical DVD - frozen image while the laser repositioned/refocused).

When I was making lots of Video CDs of old converted home videos, I just split the file at scene changes. The break was not as noticable and it made it easy to create menu entry points. My 2GB/9-minute iMac->PC issue was a major driving factor.