DV vs lossless capture of VHS

[2Bdecided]

Thank you. Glad I didn't even try to re-create that myself!

Cheers,
David.

[jagabo]

You might want to add default values so you don't have to specify them every time you call the function:

# AlignExplode v1.0
#
#   Requires VideoScope plugin. YUY2 input only!
#
#   Parameters are a text Label and Left, Right, Top, Bottom padding. L/R must be even (mod2).
#
function AlignExplode(clip c, string "Label", int "L", int "R", int "T", int "B")
{
    Label = default(Label, "")
    L = default(L, 0)
    R = default(R, 0)
    T = default(T, 0)
    B = default(B, 0)

    c
    # Temp 
    TempChromaU = UtoY().VideoScope("bottom",false,HistoTypeBottom="Y")
    TempChromaV = VtoY().VideoScope("bottom",false,HistoTypeBottom="Y")
    TempYCScope = VideoScope("both",false,"Y","Y","UV").Crop(0,height,-0,-0)

    # Layer 1
    Pane1Source = ConvertToRGB().AddBorders(L,T,R,B,color_white)
    Pane2Hist   = Histogram().Crop(width,0,-0,-0).ConvertToRGB().AddBorders(0,T,0,B,color_white)
    Pane3Chroma = TempChromaU.Crop(0,0,width/2,height).ConvertToRGB().AddBorders(L/2,T,R/2,B,color_white)
    Pane4Chroma = TempChromaV.Crop(0,0,width/2,height).ConvertToRGB().AddBorders(L/2,T,R/2,B,color_white)
    Layer1      = StackHorizontal(Pane1Source,Pane2Hist,Pane3Chroma,Pane4Chroma)

    # Layer 2
    Pane5YScope = TempYCScope.Crop(0,0,width,256).ConvertToRGB().AddBorders(L,0,R,0,color_white)
    Pane6CScope = TempYCScope.Crop(width,0,-0,-0).ConvertToRGB()
    Pane7UScope = TempChromaU.Crop(0,height,-0,-0).ConvertToRGB().AddBorders(L/2,0,R/2,0,color_white)
    Pane8VScope = TempChromaV.Crop(0,height,-0,-0).ConvertToRGB().AddBorders(L/2,0,R/2,0,color_white)
    Layer2      = StackHorizontal(Pane5YScope,Pane6CScope,Pane7UScope,Pane8Vscope)

    # Output stack
    StackVertical(Layer1,Layer2)
    AddBorders(0,20,0,0,color_black).Subtitle(Label,size=19,x=2,y=0,font="Courier New").Subtitle("Chroma only",size=19,x=978+L+R,y=0,font="Courier New")

    return last
}

[Brad]

Shouldn't that be v1.1?

Got a lossless capture of EP-mode from 80s NTSC cable whose DV conversion shows DCT artifacts and, unbelievably, 4:1:1 issues when compared as still frames at 200%. The differences are barely visible, mind you.

Here's a shot at 1440x1080 (200% x 225%).

[sanlyn]

[QUOTE=jagabo;2286933]

No it doesn't. Get a VHS source and capture it to lossless AVI (and not with a $20 capture stick). Get the same VHS source and capture it to DV.

The two devices will be using different A/D chips, proc amps, AGC, sharpening filters, etc. That makes it difficult to determine what differences are caused by DV compression vs. the other uncontrolled parameters.

What "uncontrolled parameters" are you talking about?
VHS-> TBC -> Proc amp -> capture device -> Lossless
VHS-> TBC -> Proc amp -> capture device -> DV-AVI

The only difference I can see here are the capture devices and the capture output. Everything in front of those two guys stays the same. You're saying that people use sharpeners, AGC, proc amps, etc., only with certain cards? Or what? Your firewire or DV capture device doesn't have a way to disable AGC or sharpeners? But your major objection to a straight test is that that a lossless device and a DV device use different processing chips and codecs? Well, quelle surprise! and DUH!

To me the argument being foisted here is that encoding something twice is better than (or as good as) encoding it once. By that logic, encoding it 3 times is better than encoding it twice, encoding it 4 times is better than encoding it 3 times, and so forth. Saying that "Well, you do lose something, but not that much." Cool. Get a job with a corporation writing programs to keep up their accounts and payroll and tell them that your Magic Bullet Method does make a few payroll errors and screw up some corporate bonuses and loses about 20% to 40% of the profits. But it's not that much!

Good luck with that one.

[jagabo]

No it doesn't. Get a VHS source and capture it to lossless AVI (and not with a $20 capture stick). Get the same VHS source and capture it to DV.

The two devices will be using different A/D chips, proc amps, AGC, sharpening filters, etc. That makes it difficult to determine what differences are caused by DV compression vs. the other uncontrolled parameters.

What "uncontrolled parameters" are you talking about?

All the hardware I mentioned is within every capture device -- A/D chips, proc amps, gain controls, sharping filters, etc. No two capture devices will deliver exactly the same results. You realize that the first thing a DV capture device does is digitize the incoming video signal exactly the same way an raw capture device does, right? Then it adds DV compression.

To me the argument being foisted here is that encoding something twice is better than (or as good as) encoding it once.

Not at all. Everyone already knows using a DV capture device is worse (all else being equal) than using a lossless device. The question is how much worse? All comparisons I've seen are theoretical and/or anecdotal.

[sanlyn]

I have no problem with any of that. But why is this loss, whatever the extent, preferred by some? There are "good" DV compressors and "bad" (i.e., more optimal or less for specific sources). And we all know that there were/are plenty of crappy cards for lossless capture. Other than the fact that for some users there is no other choice than DV, I don't see the point in preferring it. Even if performed with very low loss, VHS->DV color looks weird to me, it makes tape noise and defects more difficult to repair, and it almost always has a 'toon look to it with rough sparkles and bristles on high-contrast edges that persist like the devil. So you get re-encoding loss as well as more loss through more filtering. VHS is crappy enough without making it more difficult.

[jagabo]

But why is this loss, whatever the extent, preferred by some?

Because they already have a DV capture device? Because DV capture is much easier than raw YUY2 capture? Because there are less A/V sync problems with DV? Because DV gives smaller files? Because they believe their $200 ADVC-110 must be superior to a $50 YUY2 capture device?

[sanlyn]

All true. Thank the saints I'm not stuck with one of those $50 cards.

[2Bdecided]

But why is this loss, whatever the extent, preferred by some?

Because they already have a DV capture device? Because DV capture is much easier than raw YUY2 capture? Because there are less A/V sync problems with DV? Because DV gives smaller files? Because they believe their $200 ADVC-110 must be superior to a $50 YUY2 capture device?

All of the above except the last one.


At the time I started, you can add "because I knew I could capture DV but didn't think my comparatively low-spec PC would keep up with lossless", and "because it's trivial to edit DV properly (you can preview edits+effects it in real time!) and "losslessly" (no generation loss with simple edits) on a low-spec PC".


Cheers,
David.

[2Bdecided]

Got a lossless capture of EP-mode from 80s NTSC cable whose DV conversion shows DCT artifacts and, unbelievably, 4:1:1 issues when compared as still frames at 200%. The differences are barely visible, mind you.

Thank you. I'd agree.

I'm looking for some difference that's visible in the final output though (i.e. restored and encoded for viewing somewhere - DVD, YouTube, BluRay, whatever). I think upscaling to HD might just sometimes reveal a small difference. Maybe. I think even for BluRay the final encoding would create a much bigger difference.



Cheers,
David.

[Brad]

dianedebuda has posted samples from her Sony TRV480 (VHS passthrough and Hi8) that show chroma macroblocking. (There also doesn't seem to be an option to turn off the chroma NR, but that's a separate issue.)

In viewing these samples I've noticed that AVISource is giving me YV12 with NTSC DV sources even though I have Cedocida installed! How about you? Every instance of "dvsd" in my registry is related to Cedocida so I don't think another codec is taking its place.

[jagabo]

In viewing these samples I've noticed that AVISource is giving me YV12 with NTSC DV sources even though I have Cedocida installed!

Disable Cedicida's YV12 output. Force it to YUY2.

[2Bdecided]

I think you can just use

AviSource("file.avi", pixel_type="YUY2")


No need to delve into the Cedocida configuration for this - though if you're always using NTSC DV with something that accepts YUY2, it would be more convenient to do so.

Cheers,
David.

[Brad]

I thought about that, but I need a good test to see whether it isn't then just doing 4:1:1 -> 4:2:0 -> 4:2:2. It should be doing 4:1:1 -> 4:2:2 regardless of having YV12 checked since I also have YUY2 checked.

[jagabo]

I need a good test to see whether it isn't then just doing 4:1:1 -> 4:2:0 -> 4:2:2.

I've tested it in the past. It did 4:1:1 to 4:2:2. I haven't tested recently though.

It should be doing 4:1:1 -> 4:2:2 regardless of having YV12 checked since I also have YUY2 checked.

I get the same behavior -- with NTSC DV it outputs YV12 instead of YUY2 unless YV12 is disabled.

[Brad]

Well that's dumb! Anyway, I just confirmed it before reading your most recent post.

Encoded this with Cedocida:

ColorBars(720,480,"YUY2")

Then opened it and compared:

A = AVISource("ColorBarsDV.avi").ConvertToYUY2(interlaced=true)
B = AVISource("ColorBarsDV.avi",true,"YUY2")

Interleave(A,B)

A has reduced vertical res and B has full. Gonna email the author.

[jagabo]

I just confirmed it before reading your most recent post.

Me too. I did this:

ColorBars(720, 480, "YUY2")
SeparateFields()
AssumeFrameBased()
SeparateFields()
Interleave(SelectEven(), SelectOdd().MergeChroma(Invert()))
Weave()
AssumeFieldBased()
Weave()
Trim(0,29)

Saved with Cidocida in VirtualDub (everything forced to YUY2). Then opened with AviSource() in AviSynth. With forced YUY2 everything was OK. Without it you get YV12 and screwed up colors.

[2Bdecided]

Well that's dumb! ... Gonna email the author.

Why? It's delivering MPEG-2 compatible interlaced YV12 by default. That's what most people need. It's not a bug.

I can see the argument that YUY2 is far more faithful for NTSC DV (and slightly more faithful for PAL DV), but if the author changes the default behaviour now, they will break lots of AVIsynth scripts which assume YV12 input because that's what they've always received. Whereas you only need to add a few extra characters to your script to get what you need.





You could more persuasively argue that VirtualDUB and AVIsynth defaulting to progressive chroma rather than interlaced chroma is dumb, given that this assumption causes major damage when wrong, whereas the converse assumption causes less major damage when wrong. Not to mention that there is far more interlaced-encoded video in the world than progressive-encoded video. But it is the way it is. You can't change defaults once something is so widely used.


Cheers,
David.

[Brad]

Why? It's delivering MPEG-2 compatible interlaced YV12 by default. That's what most people need. It's not a bug.

I'm not sure what your "most people" is based on. Possibly the idea that most people are just using the codec to open their DV files in an MPEG-2 encoder without doing any processing?

It's downgrading my video without telling me. A note in the dialogue box that it will prefer YV12 output even for NTSC would be sufficient, though I would prefer a tickbox to prefer NTSC->YUY2 while leaving PAL->YV12.

I can see the argument that YUY2 is far more faithful for NTSC DV (and slightly more faithful for PAL DV), but if the author changes the default behaviour now, they will break lots of AVIsynth scripts which assume YV12 input because that's what they've always received.

Well, change logs do exist for a reason. Not my fault if someone upgrades without reading what they are changing.

[2Bdecided]

Why? It's delivering MPEG-2 compatible interlaced YV12 by default. That's what most people need. It's not a bug.

I'm not sure what your "most people" is based on. Possibly the idea that most people are just using the codec to open their DV files in an MPEG-2 encoder without doing any processing?

Even if they are processing, unless they're upscaling, it's hard to see how they can keep the "extra" chroma resolution. All consumer video formats are 4:2:0/YV12. The bottleneck is usually earlier though - all the decent AVIsynth deinterlacers are 4:2:0 only.

It's downgrading my video without telling me. A note in the dialogue box that it will prefer YV12 output even for NTSC would be sufficient

That would be helpful.


In a way it is telling you - you just never asked. It can't possibly hide the fact...


info()


see?


(sorry, I know you know this - and if you care about this for some reason, you must be kicking yourself that you didn't check already - we all get caught assuming things though.)

though I would prefer a tickbox to prefer NTSC->YUY2 while leaving PAL->YV12.

If you really don't want any downgrading, you need YUY2 for both.

Well, change logs do exist for a reason. Not my fault if someone upgrades without reading what they are changing.

It doesn't sound reasonable though. Anyone who wants this new functionality can already get it by adding a few characters to their script. Anyone who doesn't want it is going to find their scripts broken by the change.



Programmers are, rightly, very wary of changing default functionality.


Cheers,
David.

[Brad]

Even if they are processing, unless they're upscaling, it's hard to see how they can keep the "extra" chroma resolution. All consumer video formats are 4:2:0/YV12.

NTSC DV isn't a consumer video format?!

If you really don't want any downgrading, you need YUY2 for both.

As I said earlier (somewhere, maybe not in this thread), I don't understand the way the whole siting issue works. Perhaps you're right; I would welcome a visual explanation. It's moot anyway since I will never have PAL DV footage of my own; I only see it and play with it via samples from here.

It doesn't sound reasonable though. Anyone who wants this new functionality can already get it by adding a few characters to their script.

I realize it's selfish, but my characters are precious!

"If I got a paper cut, that’s a tragedy. If you fell down an open manhole and died, that's comedy."

[2Bdecided]

NTSC DV isn't a consumer video format?!

Go on then, consumer video delivery format. Or, frankly, legacy video format.

I would welcome a visual explanation.

This one covers PAL DV...
http://www.mir.com/DMG/chroma.html
...I'm sure it made sense in the all-analogue world it was born into.


Cheers,
David.

[Brad]

If I've got this right:

PAL DV -> 4:2:2 = adding 0Cr + 1Cb to create 0CbCr and 1CbCr (no loss)
PAL DV -> MPEG-2 siting = adding 0Cr + 1Cb to create 0CbCr and 1CbCr, then performing 1/4 sample shifting to create "0.25"CbCr and "0.75"CbCr (lossy scaling)

Is that on the right track?

[Brad]

Here is the developer's response:

As far as I can remember, the output format is requested by the application.
I.e. the application asks the codec if it can decode into YV12 (or any other format) and the codec can accept or deny.
Since YV12 is a supported output format Cedocida accepts this, unless it is configured to *not* use YV12. Therefor you can resctrict the usage of the allowed output formats by means of the settings.

Ah well.

[Brad]

I'm looking for some difference that's visible in the final output though (i.e. restored and encoded for viewing somewhere - DVD, YouTube, BluRay, whatever). I think upscaling to HD might just sometimes reveal a small difference. Maybe. I think even for BluRay the final encoding would create a much bigger difference.

You may have seen it already, but a user posted a sample from his ADVC-300 that shows mosquito noise I found to be shockingly obvious.

I've done a "proof-of-concept" with a test pattern recorded on VHS. The Cedocida DV conversion of my lossless capture has dancing mosquito edges. They are still visible in an x264 encode, while an encode of the lossless file doesn't show notable mosquito noise.

I need to see if I have any real-world footage that shows the same effect.

[johnmeyer]

I wasn't reading this forum when the initial posts were made, but the underlying concept that DV is the cause of any significant mosquito noise artifact is very hard to believe. Although any DCT compression (which is what DV uses) is certainly capable of creating mosquito noise if the compression is turned way up, I don't think that is what is going on here.

I looked at the "sample.avi" video in the thread linked to above, and I think perhaps people are confusing the standard VHS "snow" noise with compression artifacts. The sample appears to be VHS captured on "bargain" videotape (note the large numbers of dropouts). If you look at the green trees in the forest on the far shore, and zoom in while playing, you'll see a huge amount of snow noise. This actually indicates that the capture was done properly, with the "edit switch" turned on, thus defeating the VCR's smoothing circuitry which is designed to suppress all this noise, but which also kills detail.

You can reduce this typical VHS noise by averaging several captures, but having pioneered this concept over a decade ago (My "ultimate" VHS tape restoration recipe) , after trying it a few times I gave up on it, not only because of the 2-3x amount of time needed to capture each tape, but also the difficulty in lining up captures, especially since it is quite common to have each capture "off" by one field, something which completely invalidates the process.

(BTW, since I posted that eleven years ago, I learned a LOT about this process, and some of the things I was doing back then, I have since found out are less than optimal).

Back to sample.avi.

I did zoom in on the tree branches and I do see the noise. However, I am reasonably certain that is not noise which was created by the compression process, but is instead simply another example of the noise which is so evident in the green trees.

For my work (I've captured close to 1,000 hours of tapes over the past decade), I treat VHS video, captured via the DV codec embedded in my VHS-DV VCR, with a series of simple AVISynth filters, with the settings turned down as low as possible ("less is more"). These filters get rid of the usual VHS/NTSC chroma problems (the issues with reds); the rainbows; the chroma noise and, of course, the snow.

I still haven't found a good solution to the residual sharpening artifacts which, even with the HQ circuitry bypassed, are still an issue. DeHalo just doesn't work, although I sure have tried.

Bottom line on this clip: no mosquito noise here, and I sure would not recommend wasting time and disk space capturing to uncompressed. DV is a great codec for this application, and it is what I would recommend that everyone use. There are simply too many other much more important issues to consider when digitizing VHS tapes, and editing using the DV codec is so fast, efficient, and simple.

[2Bdecided]

I'm interested if you've found dancing mosquito noise vaporeon800.

[Brad]

I did zoom in on the tree branches and I do see the noise. However, I am reasonably certain that is not noise which was created by the compression process, but is instead simply another example of the noise which is so evident in the green trees.

Not sure what to tell you. It clearly looks like mosquito noise to me. Blocking is also evident in the trees, in still frames.

If you'd like, add a sharpening filter and compare the result to a noisy lossless VHS capture with the same sharpening.

the underlying concept that DV is the cause of any significant mosquito noise artifact is very hard to believe.

If you're talking in general, it's very easy to prove for yourself that DV creates mosquito noise on challenging non-VHS material.

[2Bdecided]

It's pretty similar to a series of ~100kB 720x480 JPEGs. Pixel-sharp detailed images can't be visually losslessly encoded like that, especially if you examine the results upscaled. If there's complex motion or noise, DVD looks even worse.

VHS material is far from pixel sharp, so it's interesting if there's any problem, and surprising if there's a visible difference* after restoration and encoding to the final format.

* - OK, if you freeze frame, upscale and switch back and forth you'll probably see some kind of difference - different MPEG-2 or x264 encoder settings will give a difference under those conditions.

Cheers,
David.

[Brad]

Presently I have an EP capture of a TV show's end credits: static white text with drop shadow on a motion background.

Lossless > DV > x264: clear mosquito noise
Lossless > DV: clear mosquito noise
Lossless > x264: mosquito noise...?

I haven't found animation that trips it nearly as much. I suppose more vertical filtering is used there to avoid twitter.