Analog sources on Hi-Def displays

[KenJ57]

With all the folks who have analog video tapes of their family memories, it seems good to have a way to view them in the future when analog displays disappear. And, as almost everyone here knows, analog video sources always look best when viewed on an analog display.

I very recently bought a 32 inch LCD panel and, with the help of a new "upconverter" DVD player, my rental SD movies look suprisingly very good. Also, the DVDs made from miniDV camcorder video looks pretty good when viewed with this newer gear. However, I have just spent several hours patiently viewing DVDs made from 10 years worth of Hi8 home video and the video quality is horrible!

The DVDs made from Hi8 source look "cartoonish" on the HD LCD panel. The SD satellite feed to the panel looks cartoonish also, but that I expected. These same DVDs from both sources look quite good on a good quality SDTV monitor. The scaling (apparently) to display on the HD LCD panel is revealing flaws that are very effectively masked on even a good SDTV.

The DVDs from Hi8 were digitised via a Pyro A/V Link A/D converter and both they and the miniDV sources were edited and burned with Adobe Premiere Elements version 2.


Now, I have read a great deal about the capabilities of AviSynth and VirtualDub filters and the wonderful things folks here are able to do with marginal or problematic video. Someone must have an answer for all of us who want our DVDs made from analog sources to look reasonably good on modern HD display equipment.

I have been dabbling with AviSynth and the various versions of VirtualDub for several years and have current versions of them installed. I also have the latest version of the AvsP editor which looks wonderful for AviSynth scripts.

However, even though I am a retired IT guy who still builds all his own PCs, I am math challanged. If I could speculate what is needed for the processing needed to this job properly I would say -- I think we need an "un-cartoonize" filter or a cartoonize filter with a negative coefficient. See there - I don't even know what I'm asking, do I?

Well, I hope this topic generates some useful discussion. I'm sure there are many thousands of folks who wish they had a solution for this problem.

Do I want to, as the saying goes, "make a silk purse from a sow's ear"? Absolutely!! After reading about all the wonderful things you guys can do fixing up video, I know you can come up with a good solution.

[edDV]

With all the folks who have analog video tapes of their family memories, it seems good to have a way to view them in the future when analog displays disappear. And, as almost everyone here knows, analog video sources always look best when viewed on an analog display.

I very recently bought a 32 inch LCD panel and, with the help of a new "upconverter" DVD player, my rental SD movies look suprisingly very good. Also, the DVDs made from miniDV camcorder video looks pretty good when viewed with this newer gear. However, I have just spent several hours patiently viewing DVDs made from 10 years worth of Hi8 home video and the video quality is horrible!

The DVDs made from Hi8 source look "cartoonish" on the HD LCD panel. The SD satellite feed to the panel looks cartoonish also, but that I expected. These same DVDs from both sources look quite good on a good quality SDTV monitor. The scaling (apparently) to display on the HD LCD panel is revealing flaws that are very effectively masked on even a good SDTV.

The DVDs from Hi8 were digitised via a Pyro A/V Link A/D converter and both they and the miniDV sources were edited and burned with Adobe Premiere Elements version 2.


Now, I have read a great deal about the capabilities of AviSynth and VirtualDub filters and the wonderful things folks here are able to do with marginal or problematic video. Someone must have an answer for all of us who want our DVDs made from analog sources to look reasonably good on modern HD display equipment.

I have been dabbling with AviSynth and the various versions of VirtualDub for several years and have current versions of them installed. I also have the latest version of the AvsP editor which looks wonderful for AviSynth scripts.

However, even though I am a retired IT guy who still builds all his own PCs, I am math challanged. If I could speculate what is needed for the processing needed to this job properly I would say -- I think we need an "un-catoonize" filter or a cartoonize filter with a negative coefficient. See there - I don't even know what I'm asking, do I?

Well, I hope this topic generates some useful discussion. I'm sure there are many thousands of folks who wish they had a solution for this problem.

Do I want to, as the saying goes, "make a silk purse from a sow's ear"? Absolutely!! After reading about all the wonderful things you guys can do fixing up video, I know you can come up with a good solution.

If the satellite box composit output looks "cartoonish" something else is going on with TV settings. can you post a sample frame of the DVD MPeg? VLC makes a good snapshot frame.

Here is a sample uncorrected Hi8 frame from a Digital8 playback.

[jagabo]

Turn off your TV's noise filters.

[KenJ57]

If the satellite box composit output looks "cartoonish" something else is going on with TV settings.

The satellite box is Dish Network distributed via coax on channel 3. I expected its output to be that way. I need to get the DVDs looking better.

I'll have to work on getting a screen shot from my DVD.

Turn off your TV's noise filters.

I tried the noise filters in the TV and the DVD player. They don't alter the picture much at all. So they are turned off. Thanks.

[edDV]

Use S-Video, not coax to get a better capture. RF Ch3 is the lowest quality.

[KenJ57]

Using the video Snapshot feature of VLC, I got this frame from the VOB. It looks much better than the display on my new HDTV.

[KenJ57]

I guess it takes some time to understand what one's eyes are really seeing.

I went back and played the original Hi8 footage of the wedding from which the snapshot was taken directly to the new 32 inch HDTV via S-Video and I was surprised at how reasonable it looked. But, there was still an unusual problem that I could not figure out at first. And when you take into account the known problems of low light causing lots of analog noise, bright sunshine streaming through exterior windows causing blown-out highlights, a rather poor lens causing long-shots to be rather indistinct, block-artifacting from the A/D conversion, and noise filtering during editing, it was difficult for me to even describe what I was now seeing.

Here is what I came up with after several hours of just watching and comparing. This Hi8 camcorder is a Sony CCD-TRV72 made in 1997 and amongst its various nice features such as TBC, DNR and Faroudja color seperation circuitry, it also has the electronic image stabilizer they call SteadyShot. What it does is cause a very subtle smearing of the images which is not normally detectable on SDTV displays. I watched the DVD on my older JVC DVD player connected via component cables to a 13 inch Toshiba and, although I could tell something was being messed with in the picture, it all seemed real acceptable on that nice, crisp display.

So, I appreciate the offerings of help given here and hope this information is useful to others. Thanks again.

[KenJ57]

BTW, is anyone familiar with the effect I am describing? It mainly manifests during movement of the camera. Since I almost always have the camcorder in my hands, that means it is noticeable much of the time.

Trying to describe what it looks like - its as though you were viewing through a fine "mesh", almost like viewing the scene through a window screen. And its most noticeable when doing a slow pan to the left or right - and it appears as though the screen or mesh can't keep up with the emovement. That's about the best I can do describing it.

Is there a "magic" filter of some sort that can mitigate this smearing caused by the electronic image stabilization?

You can bet that my next camcorder will have optical image stabilization!

[jagabo]

What you're describing sounds like the result of temporal noise reduction -- probably from a low light mode of the camcorder. I doubt it has anything to do with the motion stabilizer. I don't think there's anything you can do to fix it.

If you post a short segment I'm sure someone will look at it. The still frame looked ok.

[KenJ57]

What really gave a clue was when I turned the color on the HDTV panel all the way down. I continued to observe the same phenonena for just the B & W information.

This camcorder is a consumer model and does not have gain switches like pro models. It does have various "program" modes but I never use those. Probably should - but never do. The camcorder has Time Base Corrector, Dynamic Noise Correction and an Edit mode to be used when transfering the video to another device for editing. I tried turning each one of these off, in turn, and in various combinations but continued to see the same phenomena no matter what I changed.

Posting a clip may not be too useful - because the video looks quite acceptable even on a good quality analog display. Would you be suggesting clipping a portion of the VOB? Never done that before. Always worked with DV-AVIs. I suppose VirtualDub_MPEG could cut out a short clip? I figured out how to post a snapshot here, but how/where does one post a clip?

[jagabo]

The camcorder has... Dynamic Noise Correction...

That's likely to include a temporal noise filter under low light conditions. This would have been applied before recording onto the tape. But it should be visible when playing the tape directly to your TV as well as from your DVD. Maybe the MPG was just encoded with too little bitrate? That will also cause the sort of effect you're seeing.

Posting a clip may not be too useful - because the video looks quite acceptable even on a good quality analog display. Would you be suggesting clipping a portion of the VOB? Never done that before. Always worked with DV-AVIs. I suppose VirtualDub_MPEG could cut out a short clip? I figured out how to post a snapshot here, but how/where does one post a clip?

Download DGMPGDec. Run the included DGIndex. Open one of your VOB files with it. Use the [ and ] to mark the secion you want so save. Select File -> Save Project and Demux Video. That will create a M2V (MPEG 2 elementary stream) of just the marked section. It might be best if you could post both the original AVI and the VOB. Do you see the difference if you view them in VirtualDubMPEG2?

[KenJ57]

The Sony CCD-TRV-72 manual says that Edit, TBC and DNR modes only work during tape playback. Since I viewed playback of the video clip with all iterations of these settings my conclusion is that the damage is recorded on the source tape. (BTW, the recording was made with this camcorder so there is no interchange problem.)

The damage is detectable on an analog display but is, for the most part, masked (apparently) by the bandwidth of the display system and is seldom bothersome (except to folks like me!). My speculation is that it is the increased bandwidth of the HDTV display that reveals more picture information - so much so that the picture damage then becomes painfully obvious so that the viewer says "what is wrong with the picture?" All aberrations that were reasonably masked are now revealed. A frame by frame advance through the clip shows just how much picture damage is masked when played at the standard NTSC frame rate.

I will have to work on getting the tools you listed for posting a clip. It may take me a couple of days to get set up. The expectation of the viewer to reproduce this problem is to view the clip on an HDTV. Thank you again for the dialog and help.

[jagabo]

Keep in mind that HDTVs usually do a lot of processing of SD video in an attempt to make it look better. Even if you have turned off the user selectable noise reduction there may still be some noise reduction going on. And LCDs often have issues with blurring of low contrast areas in motion.

I have an LCD HDTV so I can tell you what I see once I get the samples.

[KenJ57]

Turn off your TV's noise filters.

As it turns out, the 32 inch HDTV does not have any filters to turn off. Only the DVD player has them. I tried them earlier, they don't help anything, so they remain turned off.

there may still be some noise reduction going on.

I agree. The S-Video playback on this set is much better than a set I tested in a store a year and a half ago. It still doesn't compare to even an average SDTV display though.

And LCDs often have issues with blurring of low contrast areas in motion.

How about areas of high contrast, which this appears to be?

It might be best if you could post both the original AVI and the VOB. Do you see the difference if you view them in VirtualDubMPEG2?

I don't have the original AVI. I could recapture it if necessary.

I viewed the d2v file in VD_MPEG and could not see anything relevant. But then I can never see anything concerning video quality on a PC CRT display. Too dark and not enough resolution.

I don't know how anyone can edit using an editor such as VirtualDub since you can't really see if a filter is needed or has done anything useful. That's why I edit with Adobe Premiere Elements - because it sends the Timeline window display out the Firewire port and you can use an A/D converter such as Pyro, Canopus, Datavideo to feed an external monitor TV and see EXACTLY what the edited video looks like in real-time.

I download DGMPGDec and ran DGIndex. Nice program. I selected three small clips which highlight the damage I am trying to deal with.

The first clip is of a woman with a dress having a high-contrast pattern which "sizzles" (for want of a better term) with artifacts.

http://www.jarstad.org/images/Wed-test1.demuxed.m2v

The second is a table with wedding presents - panning to some folks waiting in the narthex to go into the sanctuary. Whenever the camera moves the "sizzling" is noticeable. Pause the playback when the older man is in the frame and notice the cartoonish picture.

http://www.jarstad.org/images/Wed-test2.demuxed.m2v

The third clip is of folks coming into the sanctuary. The camera is remains mostly stable for a moment and then slowly pans to the right. Notice the man in the light colored jacket who's head is centered in the following frame.


As the camera pans to the right his jacket exhibits the problem big-time. Presumeably you will play back these clips from a DVD - R/W.

http://www.jarstad.org/images/Wed-test3.demuxed.m2v

[jagabo]

I think a lot of your issues come from oversharpening a noisy source. This has created several problems.

Sharpening has increased the noise and subsequently made it more difficult for the MPEG encoder -- hence the "mesh" and posterization artifacts. MPEG encoding relies largely on the fact that the picture usually doesn't change much from frame to frame. It can save lots of bitrate by only encoding parts of the picture that change. With lots of noise every pixel changes from one frame to the next. MPEG is forced to reencode the entire picture with every frame. It has to fall back on droping details (resulting in macroblock and posterization artifacts) and ignoring the smaller changes (mesh artifacts, low contrast areas moving differently from high contrast edges).

Since LCD displays are progressive the video has to be deinterlaced as it's displayed. This, coupled with the oversharpening is creating the buzzing artifacts on high contrast edges. Deinterlacing quality varies from HDTV to HDTV (or DVD player to DVD Player if the player is doing it).

Oversharpening also creates overshoot artifacts:



In the center of this 4x, nearest-neighbor enlargement you can see that the picture transitions from a light shade of gray to a darker shade of gray. Right at the transition you see an extra light band on the light side and an extra dark band on the dark side -- overshoot.

Standard definition televisions normally receive rather unsharp images (on the horizontal axis) and sharpen them up for display. If you send a very sharp image to the TV it will sharpen it up even more resulting in even worse overshoot. This may be less visible on a CRT because the tube is working near its resolution limit. But when you enlarge the pixels for a high resolution LCD the overshoot is much more visible.

Other possible issues:

The video is encoded bottom field first. That wouldn't normally be a problem but I wonder if it could be confusing your TV? Doubtful. You might try converting to TFF just as a test.

Make sure the DVD authoring program recognizes the sources are BFF (or TFF if you convert). Wrong field order problems would cause very fast-jerky motions: 2-steps forward, one step back, 30 times a second. I doubt this is your problem, and I think you already understand field order issues, but I thought I'd mention it.

How is the player connected to the TV? HDMI? S-video? Composite? Are you using an upscaling player? If you're using an upscaling player you might try letting the TV upscale instead of the player. The TV might do a better job. Or vice versa.

[KenJ57]

Wow. Nice analysis - and greatly appreciated.

I think a lot of your issues come from oversharpening a noisy source.

I don't recall using any sharpening on my videos while editing from Hi8 sources. However, my workflow might give some clues:

Hi8 camcorder S-Video -->Pyro A/V Link A/D converter --> Firewire to Adobe Premiere Elements capture (RGB32) - burn DVD in Elements.

Perhaps what you perceive as sharpening is the A/D conversion process. I don't know to what exactly to atrtribute the overshoot (BTW - I do understand overshoot and can see it in your inset).

Televisions normally receive rather unsharp images (on the horizontal axis) and sharpen them up for display. If you send a very sharp image to the TV it will sharpen it up even more resulting in even worse overshoot. This may be less visible on a CRT because the tube is working near its resolution limit. But when you enlarge the pixels for a high resolution LCD the overshoot is much more visible.

This is what I was speculating earlier - the artifacts are not noticed by MOST viewers on SDTV and the picture quality is judged as being quite good. I still have a Datavideo DAC-100 A/D converter which I no longer use because I always had to use a noise filter on the video it produced. It had extra "clarity" but produced a noisier picture. So, it appears that there is a trade-off in designing these converters between detail and noise/artifacts.

The video is encoded bottom field first. That wouldn't normally be a problem but I wonder if it could be confusing your TV? Doubtful. You might try converting to TFF just as a test.

Make sure the DVD authoring program recognizes the sources are BFF (or TFF if you convert). Wrong field order problems would cause very fast-jerky motions: 2-steps forward, one step back, 30 times a second. I doubt this is your problem, and I think you already understand field order issues, but I thought I'd mention it.

Since the editor's native format is DV-AVI, the DVD authoring is built-in and the DVD does not exhibit field order flaging I am sure my DVD is authored and burned correctly. DV-AVI is BFF and standard DVD is 480i TFF in my understanding and Elements will get it correct. I checked a clip in GSpot and found you are right - BFF. Perhaps the DGIndex utility changed the field order of the clips I posted?

How is the player connected to the TV? HDMI? S-video? Composite? Are you using an upscaling player? If you're using an upscaling player you might try letting the TV upscale instead of the player. The TV might do a better job.

My new Sony DVP-NS71HP DVD player is one of the new generation of upconverting units which I connect via HDMI. This unit is phenominal! Some of the commercial std-def DVDs look like hi-def to me.

I tried all three output resolutions in hope the scaler would work differently between them on my home-made DVD. The unit is able to send out 720x480p, 1280x720p and 1920x1080i. My Emerson 32 inch panel I am told is 720p native so (am quite sure) it would scale for any other input. It has no scaler selection. I could not see any difference between them except for perhaps a slight lack of "fluidness" during movement on one of the settings which I can't remember now. I leave this set to "automatic" and let the set choose. The important thing to state is that the visual damage remains visible in all three settings. Kind of rules out the scaler(s) doesn't it?

Just for fun I connected the DVD player via COMPOSITE connection and turned the sharpness on the panel all the way down. Then my DVD looked just like the signals coming from the NTSC tuner! The artifacts were then as acceptable as playing on an SDTV.

I must experiment more with my A/D converter to see how/if I can mitigate the problem.

Thank you for your indulgence and help in tracking down the problem.

Ken

[spysr]

Im having the exact same problem. I havn't been able to solve this yet. I susupect it might have something to do with bitrate. Im scanning an old VHS Tape to my PC thorugh a Sony Handcam with a composite cable, then to the pc trough Firewire. I thought maybe it was because my VHS-DVD combo Sony player is not SVHS. But when i play the original tape to my RCA Analog TV on a regular VHS it looks very clear. So at least i should be able to come close to matching this. What i see on the final burn after editing in Premiere CS3 is the exact same artifacting problem that was posted in the previous comments. I am considering buyng an S-Video Cable to go from my VCR-DVD combo to my Sony Camcorder for encoding, maybe that will help.

[KenJ57]

To paraphrase what jagabo said, a common way to get greater detail in analog systems was to crank up the "detail", "peaking" or sharpness in a source camcorder with the understanding that, in an analog system, the "overshoot" would not be noticed on the end-user display. Unfortunately, digital systems are a completely different breed of animal. They behave differently and the old rules no longer apply - you can't get away with enhanced detail in the source because the A/D converter will pick up the "ringing" or "overshoot".

Therefore, I am afraid the A/D converter in your DV camcorder passthrough may exhibit the very same problem I am getting. I plan to shoot some new video with my Hi8 with the electronic image stabilizer off, panning to see if I still get an image "afterglow". At least it will elliminate (or not) that as part of the problem.

BTW, the bitrate of my DVD is slightly under 8 Mb/s - 45 minutes of video on a DVD +R.


An after thought ------ since most Digital 8 or D8 camcorders can play back 8mm and Hi8 tapes, could those cams have some compensation built-in to "tone-down" the high frequency peaking before submitting the signal to the A/D converter???? Hmmmmmm. Maybe! Anybody know?

I think I will see if I can find someone with a D8 rig to test on.

Another thought - passthrough on a miniDV camcorder may already have muted detail response just to overcome these problems. But I really don't know for sure. I would enjoy hearing about your results.

Why, maybe I will even try capturing my Hi8 video using a composite cable instead of an S-Video cable to see if the ringing is reduced.

[jagabo]

I'm looking forward to hearing your findings...

If you want to try unsharpening the video in AVISynth try the Blur() filter. It allows different amounts of bluring on each axis:

Blur(1.5, 0.0)

That will blur on the horizontal axis but not the vertical axis. You don't want to blur along the vertical axis because your video is interlaced.

[KenJ57]

[I originally tried to post this Tuesday night , but the forum was not available]

Thanks, jagabo for staying with me.

I re-captured three short segments of video from the original Hi8 tape. Each was roughly the same problem segments I posted earlier. Each collection was less than a minute long.

And each of the problem segments was captured three times, the first with the Pyro via S-Video. The second was with the Pyro via composite video, and the third time was with the DataVideo DAC-100 via Firewire.

The three captures of the three problem segments were placed on my Adobe Premiere Elements v2 Timeline, saved and simply burned directly with Elements to a Taiyo Yuden 16X DVD - R.

I played the DVD on the Sony/Emerson system and found that the picture damage problem was not nearly as visible from any of the clips as it was in my original wedding video DVD that I made last February. Huh!

My eyes told me this: the composite capture was nearly as good as the Firewire capture. What a surprise! And the DAC-100 did trade off a little extra noise for a bit more clarity, but it did not damage the video. I might add that the ordinary viewer would be hard-pressed to see any differences. Only trained eyes can see this.

You have good eyes, and have excellent experience, jagabo! I MUST have done SOMETHING to that video for it to become so degraded.

Usually, if I have the time I will run the Hi8 captures through VirtualDub using the MSU Denoiser to get rid of the significant analog noise. However, I don't remember doing that with this wedding video.

I will now apply some effects and noise reduction, using this DVD as a standard, to see what might have gone wrong.

I have to confess my original complaint was unfounded, and I stand corrected with the help of jagabo and all the fine folks who routinely help on this forum. Thanks. all.

Ken

[jagabo]

I'm glad to hear your problem has gone away.

Regarding composite capture: the major difference you will see between s-video and composite is in the colors. With composite the resolution of the chroma (color) channels may be lower and you may see dot-crawl artifacts. The Pyro may have a good comb filter to get rid of the dot crawl artifacts. With a low res source the color resolution loss may be hard to detect.