Restoring VHS: VHS to ?

[matt-]

I have been testing video encoding for archiving from my Dazzle capture device. VirtualDub and the huffyuv codec is used for recording.

Seems kind of simple, but after testing it looks like the DVD MPEG2 codec is extremely inferior compared to H264. That said, What is the final video format? I'm leaning towards DVD+DL and using a computer to play back .MP4 H264 files created by MEGUI. After using three different types of DVD authoring software the video looks very poor, even with a duel layer disk.

Here, take a look:

DVD Quality


H264 MPEG encode (took 3 solid days to encode using automated two-pass)


Original from capture device



The example video is 6 hours and 6 min long. Each compressed format, dvd and h264, were compressed using the same bit-rate. This raw video weighs in at 172GB using huffyuv. The DVD and H264 output each weigh around 8GB (DVD+DL).

My questions:

Am I making obvious mistakes when encoding DVD or does it always look that poor?

What do you guys normally use for preserving VHS in it's best quality? Do you just stick with the standard DVD format, or is there a better and more standard way that yeilds better quality?
In my situation, it's a choice between quality vs. playability. Note that the H264 video is only playable on a computer; however, the DVD MPEG2 standard can be played almost anywhere. But also note the dramatic quality differences.

Any suggestions for video archiving or encoding? I don't necessary intend on keeping video for editing, but for archiving. Is there a preferred medium, codec, or tool?

Thanks for any assistance,
Matt.

[lordsmurf]

Dazzle makes functional but mediocre devices. I would not use it for archiving important videos. That video you have, for example, needs a lot of work. It should be much sharper, and the color more balanced. It's also full of awful color (chroma) noise, all that red/green/blue color flecks.

What you see is not due to MPEG-2 DVD, but the hardware and software that you're using. And/or the settings you've chosen.

You also cannot put 6 hours of video on one DVD and expect high quality.
To "archive", you should do 1-hour (7Mb/s avg bitrate) 720x480 MPEG-2 interlaced for DVD.

More info at digitalFAQ.com about converting/restoring VHS to DVD.

[creamyhorror]

As lordsmurf said, you would do well to do some filtering on those captures (primarily denoising, especially chroma denoising).

In terms of archiving, it all depends on how much hard drive space you're willing to allocate. If you can't do Huffyuv, the next best option would be an intraframe-only codec at high bitrate, which will preserve the video accurately enough for editing ("visually lossless"). The next step down is H.264 at high bitrate.

You can do intraframe-only with x264 in MeGUI by changing the Keyframe Interval setting to 1. Try a CRF setting of 16 if you're on a version of x264 prior to 1197, and a setting of 12 if your version is above that.

But settle the filtering first, because you want this first encode to be close to perfect imo.

[AlanHK]

The example video is 6 hours and 6 min long. Each compressed format, dvd and h264, were compressed using the same bit-rate. This raw video weighs in at 172GB using huffyuv. The DVD and H264 output each weigh around 8GB (DVD+DL).
My questions:
Am I making obvious mistakes when encoding DVD or does it always look that poor?

If you put 6 hours on one DVD, it will be pretty mediocre.

MPEG2 is less efficient than newer formats like H264. It can certainly achieve high quality, but will need more space (i.e., a higher bit rate). Three hours or less per 4.7 GB DVD should give a good result.

[jagabo]

And don't use a media player as your sole means of checking image quality. They will not show you exactly what's in the file. The image will be resized to correct the aspect ratio. There may be other processing going on, like deinterlacing, brightness/contrast adjustments, etc. And different sources may be treated differently.

Especially avoid using two media players side by side for comparisons. Usually one will use the graphics card's video overlay feature and the other will not. Video overlay has it's own proc amp settings and may include other video processing. Even the same video viewed with two players simultaneously will look different.

[lordsmurf]

Buy another hard drive if you don't have enough room for HuffYUV files. You can get a 1TB drive for under $100 now. An internal goes as low as $50-65, externals as low as $80-90.

Not having enough hard drive (when taking on a video project) is like driving a car in a race with only 3 good tires.

[matt-]

Original Quality issues:

As with the chroma noise, it seems to be a result of low light for the camera sensor. Other scenes provide much better color as long as enough light was provided. I probably won't correct this as the noise varies dramatically between scenes. I did take a look at Gilles Mouchard's chroma noise reduction filter for VirtualDub. It didn't provide results without configuration.

As with the fuzziness in the original quality video, I do believe I have an on-the-fly de-interlacing filter running which may contribute. The Dazzle DVC-100 is also far from perfect, but it's just what I could get my hands on at the time. Would you suggest an ati wonder over this? Or perhaps an Osprey 210/220? capture card?

I currently have a Prime Image TBC in the mix. This may also contribute the fuzzing of the image. An advantage of using the TBC is that it gives me control over video level, setup, color, hue, etc. Most importantly, it would have been impossible to dub what I did without a TBC.

Archiving and Compression

1Hr per DVD-5?! It seems funny that I would have to span a single EP VHS over six DVDs (~3 DVD-9s). Would I have to use a BD-R 25GB medium for the MP4 H264 files just to keep the disks from being discontinuous?

I'll have to put prices together on different storage methods: DVD-5(22 Cents Per), DVD-9(80 Cents Per), BD-R($4 per), BD-R DL ($20 per!)

Note that Blu-ray would cost a good $180 extra for burning equipment. Also note that I will not be burning Blu-ray compatible content. This is because blu-ray authoring software requires a huge resolution upscale for it to work properly. To me, an upscale is a waste of space and causes determinant degradation as it must go through a fixed scaling algorithm (rather than continuously improving on-the-fly scaling algorithms.
As a result, I would probably remain using MeGUI to encode to MP4 files, but using quality settings rather than target bit-rate 2pass.

Advantages / Disadvantages

DVD-5 or DVD-9: MPEG2: Can create DVDs compatible with everyone's trashy DVD players. Disadvantages: Must apply extremely high bit-rates resulting in many discontinuous disks.

DVD-5 or DVD-9: MPEG4/H264 files: Much better quality and much better space efficiency. Can squeeze a 6HR EP tape onto a DVD-9 with some quality loss. Better quality would result in some discontinuous disks. The video cannot be played by most people (in my family) without transcoding to DVD for them.

BD-R: MPEG4/H264 files: Awesome quality with continuous disks! Disadvantages: Almost no playability for anyone else as it requires a strong computer and expensive optical hardware.

HDD Redundant RAID: MPEG4/H264 files: Awesome quality with continuous disks! Also provides redundancy and the price may end up being equal to the Blu-ray hardware and disks. Disadvantages: Almost no playability for others without conversion.

BD-R: Native format and BD-R DL are out of the question as BD-R: native format is inefficient and BD-R DL disks are far more expensive than anything on the planet!


I'll still have to get prices together for the above ideas (total costs). I'm lost as of now.

[johns0]

Not having enough hard drive (when taking on a video project) is like driving a car in a race with only 3 good tires.

I would say not having enough hard drive (when taking on a video project) is like driving a yugo in a race.

[thecoalman]

I have been testing video encoding for archiving from my Dazzle capture device. VirtualDub and the huffyuv codec is used for recording.

Seems kind of simple, but after testing it looks like the DVD MPEG2 codec .........

According to what I'm reading that's DVD compliant MPEG hardware encoder, you should be using the file it's encoding and not converting it to anything. That file could then be transferred directly to DVD. That's the file you want to archive as it's the most original.

Use the software that came with it for capture, if you can change the hardware settings use 720X480 at the highest bitrate it allows for archiving.

You can get better cards, for that matter a good DVD recorder would accomplish the same thing.

[Abas-Avara]

I would choose the highest quality possible, it takes a lot of space but
hard drives are becoming cheap. You wont regret, after a few years capacity increases and that video size is a joke.

First I also wanted to convert all the DV AVI files to MPEG2 but at the end I decided to keep the original DV AVI files (cause of the statement above).

If you have an Camera with Firewire you could maybe use that as you capture device (firwire passtrough)

[matt-]

@thecoalman- I've read about that too, but I can't find exact hardware specifications that state so. The capture device uses the YUY2 colorspace/compressor from the video pin. I also noticed when using the software that came with the Dazzle, there is intensive CPU usage during a dub. This may imply that it is simply encoding MPEG2 on the fly rather than strait from the hardware device. There isn't much CPU usage at all when the DVC-100 is recording to Uncompressed.

So do you think storing an entire VHS collection over a HDD RAID may be the answer? I still need to find prices, but I would still like to see what others found most useful. This would obviously be converted to a high quality h264 as storing lossless would be petabytes of data.

[Abas-Avara]

I dont know how h264 is but I use DV AVI (firewire) and that is 13GB per hour. If h264 is much more then that maybe there is a another lossless (or very small) compression. In that direction I have no experience with H264. I would say go as high as you can but not abnormal high, 1 Hour 4,7GB must be minimal I think.


But in this forum there are lots of experts they could help you out with the best codec.

[matt-]

For the H264 video above, I used a bitrate of 2839. This was calculated using the videohelp bitrate calculator using the following: 6 hr 6min, 192 Kbit audio, DVD-9, DVD preset (to accommodate for possible overhead).

[jagabo]

For purposes of archiving you never want to deinterlace. By deinterlacing you are throwing away half the temporal resolution (30 pictures per second to 60 pictures per second) and as much as 50 percent of the spacial resolution depending on the method used.

Your MPEG encoding was probably saved at 352x240 (and definitely deinterlaced with only 240 lines) in order to accommodate the low bitrate you're using. The h.264 encodings were probably 720x480 or 352x480 (the latter is close to the actual resolution of VHS tape). h.264 encoding at those low bitrates and those frame sizes will look reasonably good with a good encoder. But h.264 isn't really an "archival" format. It's a final delivery (viewing) format.

I think you should save as DV AVI (720x480 29.97 interlaced frames per second). That will give you good quality interlaced video that is easy to convert to other formats like DVD. At 13 GB/hr a 1 TB drive will hold about 75 hours of DV.

And to get really good quality from your VHS tapes you need an S-VHS deck with a built in line time base corrector. But that's another discussion...

[lordsmurf]

Nobody is going to watch all 6 hours at once anyway. Break it up into digestable chunks. You should have had 2-hour VHS tapes anyway, using longest play recording mode was a mistake on your part. Don't repeat it digitally.

[matt-]

That's a good plan. I'm leaning towards the DVD-9 medium and using MeGUI or a DVD authoring tool to chunk the video into segments. I may stick with H264 files; but I'll retest using higher bit-rates.

I'll compare the following options:

DVD-5: H264 with varying bit-rates (fit 1hr, 2hr, 3hr)
DVD-5: MPEG2 DVD with varying bit-rates (fit 1hr, 2hr, 3hr)
DVD-9: H264 with varying bit-rates (fit 2hr, 3hr, 4hr) *6hr test already done
DVD-9: MPEG2 DVD with varying bit-rates (fit 2hr, 3hr, 4hr) *6hr test already done

I don't necessarily care about discontinuous disks anymore (as lordsmurf stated). All these tests may allow me to determine if H264's abilities are worth the playability sacrifice.

Lordsmurf, I wish these were all two hour tapes as it would be much easier on my part. Get this: my father constantly re-recorded onto a single camera tape. After this tape was full he would use a VCR to dub the video from the camera using EP onto an old tape used for recording his programs! He would then rewind the camera tape and repeat until the EP tape in the VCR was full. I wish I was there to tell him, but I didn't exist yet.
Also, I sent a PM to you and didn't realize you responded until now. I'm sorry! I'll respond once I get a chance. Thank you so much for responding!

@jagabo: Preserving interlace is important. I was sure to set MeGUI to do so. Hopefully it doesn't loose too much field data during encoding; but I can't be sure of it. Also, I believe I was mistaken about archiving. A viewing format is what I'm looking for. No editing is expected to be done with these videos.

[creamyhorror]

@thecoalman- I've read about that too, but I can't find exact hardware specifications that state so. The capture device uses the YUY2 colorspace/compressor from the video pin. I also noticed when using the software that came with the Dazzle, there is intensive CPU usage during a dub. This may imply that it is simply encoding MPEG2 on the fly rather than strait from the hardware device. There isn't much CPU usage at all when the DVC-100 is recording to Uncompressed.

If the Dazzle captures in MPEG-2, you shouldn't be converting any more from there. That's the best you can hope to achieve (and it probably isn't very good, assuming average bitrates). The exception is if you want to do chroma denoising (Avisynth's FFT3DGPU filter in chroma mode works great for this, but I'm sure lordsmurf has better suggestions).

The h.264 encodings were probably 720x480 or 352x480 (the latter is close to the actual resolution of VHS tape). h.264 encoding at those low bitrates and those frame sizes will look reasonably good with a good encoder. But h.264 isn't really an "archival" format. It's a final delivery (viewing) format.

What qualifies as an "archival" format?

I suggested using Keyframe Interval = 1 in MeGUI/x264 earlier, which would produce an intraframe-only, fast-decoding H.264 video (which could be worked with comfortably in an editor like Premiere). At a bitrate of 8-10Mbps (~4.5GiB/hr), everything would be visually losslessly preserved.

This would obviously be converted to a high quality h264 as storing lossless would be petabytes of data.

Try what I suggested in MeGUI. In x264 options, set Keyframe Interval to 1, check Interlaced Video (very important), and choose Constant Quality mode with a CRF of 12-16. Encode a sample video and compare frames between the source and encode. If you find the output bitrate/size too high, you can increase CRF (probably not beyond 16).

If you're sure you want even smaller files at the expense of near-perfect fidelity (edit: i.e. a playback and not archival format), you can go with standard x264 options. But note this means you shouldn't be editing and re-encoding those clips in future (because of their lower fidelity to the source).

Also, if you find the Dazzle is actually capturing to MPEG2, then don't convert it to anything else, unless you're willing to throw away a part of the quality forever.

edit: If you're going to spend hours upon hours re-encoding to H.264, you should certainly make sure you get the denoising/filtering right!

[lordsmurf]

I would avoid H.264 for this exact project, to be quite honest.

[matt-]

If the Dazzle captures in MPEG-2, you shouldn't be converting any more from there. That's the best you can hope to achieve (and it probably isn't very good, assuming average bitrates). The exception is if you want to do chroma denoising (Avisynth's FFT3DGPU filter in chroma mode works great for this, but I'm sure lordsmurf has better suggestions).

I have reason to believe that the DVC-100 doesn't encode to MPEG2 on the fly. Many of the Conexant chips do, but not knowing what was really inside the Dazzle was the issue. It may seem that we are getting off topic, but I think it's important to know as it may affect the delivery or final format for my project.

So I cracked the DVC-100 open:



Take a look at the chips



I think the EM2820 is the USB communications chip. The EMP202 looks like an audio codec (according to other Datasheets. The SAA7113H is the main video encoder. It's a 9-bit video input processor.

Looking at the datasheet, there is no mention that it uses MPEG2. In fact, it states that it uses a standard YUV output:

Standard ITU-R BT 656 YUV 4 : 2 : 2 format (8-bit) on VPO output bus

I will do a comparison between an MPEG2 frame pulled from the Pinnacle program vs an uncompressed frame pulled from VirtualDub. I'll use the "freeze" feature on my TBC to ensure that it's uniform.[/quote]

[matt-]

Ok, here are the image comparisons. There doesn't seem to be an obvious quality difference. To prevent DVD filters from de-interlacing, I used VDubMod to save as decompressed AVI before snapshots were taken. The Pinnacle DVD creator software that came with the unit was used. This means either the bit-rate is high enough to detour artifacts or the device is indeed a hardware MPEG2 encoder. Notice that the good quality version shows slight artifacts. And that tape... found it five feet away from me as I was in the basement messing with my capping machine.

Uncompressed VirtualDub



DVD Best quality ~7.5Mbit/sec



DVD Good quality ~5.7Mbit/sec

[lordsmurf]

It's software encoding, not hardware.

[jagabo]

You should create your sample images from VirtualDubMod using Video -> Snapshot Source Frame. Use PNG, not JPG, so nothing is lost.

[Jim44]

The SAA7113H is the main video encoder. It's a 9-bit video input processor.

Looking at the datasheet, there is no mention that it uses MPEG2. In fact, it states that it uses a standard YUV output:

Standard ITU-R BT 656 YUV 4 : 2 : 2 format (8-bit) on VPO output bus

The SAA7113 is not an encoder. It is a video processor, whose main function is as an analog-to-digital converter. It has some other functional blocks for doing some processing of the video signal, but it does no MPEG encoding.

[PuzZLeR]

If you capture to lossless, or DV, etc for high quality retention, then encode with a good MPEG-2 encoder like CCE or HC Enc, you will notice a dramatic improvement in your MPEG-2 encodes for higher bitrate, high quality video.

Then again, congratulations on getting anything worthwhile from Dazzle, which normally sucks. Not only did I gut my Dazzle wide open like you did, I also got to watch, and smell, those chips and plastic components as I was melting and burning them...

[PuzZLeR]

I would avoid H.264 for this exact project, to be quite honest.

I agree 100%. All those tests showing how great H.264 is, especially x264, are greatly misleading (and very boring which is why I didn't participate in that other thread).

x264 only does well with clear, progressive video source and nothing else, which is where praise for it comes from and solves a problem nobody has. It does not even come close to MPEG-2, with a good encoder, at the higher bitrates, with interlaced source, interlacing encoding, noisy or restoration projects, or any "real stuff" like a project like this would entail.

(Uh-oh, here come the Fan Boys now... sigh...)

[creamyhorror]

The trouble with you is that you post your assertions but refuse to engage in testing. Is everyone then to simply accept your claim? Have you posted results anywhere that people can look at?

The fact that you can use x264 as an intra-only codec means that at higher bitrates it can match or outperform other codecs in achieving visual losslessness, whatever the source is. That's what I suggested to the OP.

How about we do tests to see how x264 stacks up against MPEG-2 under your conditions? I'm curious to see the results.

[PuzZLeR]

Well, for example, do I need to post proof that MPEG-2, with a good encoder, at the higher bitrates, is better than x264 with interlaced source or interlaced encoding?

Nope. Don't think so.

From Canada, I wish you good night my friend. It's currently late up here.

[jagabo]

Well, for example, do I need to post proof that MPEG-2, with a good encoder, at the higher bitrates, is better than x264 with interlaced source or interlaced encoding?

Please do.

[creamyhorror]

Well, for example, do I need to post proof that MPEG-2, with a good encoder, at the higher bitrates, is better than x264 with interlaced source or interlaced encoding?

Nope. Don't think so.

Well, why not? You can post a source clip and your MPEG-2 encode of it, and I (or someone else) can post an x264 encode. Some "real stuff" that you've been working on, maybe.

If we're talking about upper-reaches-of-the-stratosphere bitrates here, the differences (if any) are going to be small, but I'm betting that x264 won't be any worse than MPEG-2 (regardless of the encoder).

[PuzZLeR]

Here's a good one. Turn "interlacing on" in x264 and show me a better progressive encode. This would be necessary to run SD (interlaced or progressive source) in blu-ray.

EDIT: At the higher bitrates please. Good night.