Tweaking VHS captures

[sanlyn]

Hm, I just scrolled up to post #1. There's convertToYV12 before applying filters. I've always capped YUY2 with huff, RGB when uncompressed.

Should that be applied after filters?

In this case, no. Go to YV12 first, because your source video was in RGB.
* ColorYUV works only in YV12 or YUY2.
* MCTemporalDenoise works only in YV12.
* Crop() works in YV12,YUY2,RGB32,RGB24. In YUV your crop values must be even numbers. See the AviSynth doc on Crop() colorspace restrictions.

poisondeathray and/or jagabo or whoever, correct me if I'm wrong, but:

When you ran that initial script and then saved it in VirtualDub with "full processing mode" enabled, VirtualDub converted your YV12 to RGB32 before saving it. The problem with the "automatic" RGB conversion is that VirtualDub by default treats interlaced YV12 as bottom field first. So if you want your final output saved in RGB32 (and your source is interlaced, top field first) you need:

AssumeTFF().ConvertToRGB32(matrix="[put matrix name here]",interlaced=true)

[sanlyn]

VirtualDub converts incoming YUV video to RGB for filtering and display. Like most other programs it uses the rec.601 matrix. . . .

Got it. Thank you. I've been using Rec601 for SD video, and setting TMPGenc (and CCE when I use it) appropriately as you suggest.

Exception: If I convert something to Rec601 and see a lot of crushed or blown-out stuff, I'm assuming the source video either isn't Rec601-friendly, or was captured or produced with wild levels, so I make adjustments. I realize that even if the ultimate target is 16-235, there is allowance for overshoot. I think too much overshoot looks rather "raw" on most LCD TV's. Occasionally a video looks that way on a TV, so I readjust.

[sanlyn]

Usually I interpret a histogram as saying "in-range", "out of range", or "wiped out" as shown below:



A: In VirtualDub I'd say this image would be "mostly" in-range of 16-235. It's a bit bright, and there's a bit of clipping with the brightest blues. I'd adjust the brightest red and blue because those highlights might look a little "hot" on TV. It depends on the video; some high-key stuff is done that way on purpose. The far left-hand spikes are the frame borders.

B: blacks crushed.

C: highlights blown away.

[jagabo]

In TMPGEnc Plus the option to treat the incoming RGB as studio RGB is called "Output YUV data as Basic YCbCr not CCIR601".

The trouble with interlaced YV12 in VirtualDub is that it converts to RGB assuming progressive YV12. This screws up the chroma channels. It does not effect the field order, which remains the same as whatever the source is. There is no way to flag the field order in RGB AVI, or even that the video is interlaced, so the encoder has to be told the source is interlaced video and the correct field order. In TMPGEnc Plus, for example, the Setting -> Advanced (tab) -> Video Source Setting is where you specify the source properties.

[jagabo]

Some things to note about the ColorTools histograms:

1) The 0-255 and 16-235 Source Attributes setting only effect the gray scale graph. That graph shows what the luma levels will be after conversion to YUV. 0-255 specifies the rec.601 matrix will be used to generate the graph (the graph can only extend from 16 to 235). 16-235 specifies that the pc.601 matrix will be used, the graph can extend from 0 to 255 but the values should be restrained between the 16 and 235 markers.

2) The 16 and 235 markers on the red, green, and blue graphs don't really mean anything. Red, green, and blue data can extend all the way down to 0 and all the way up to 255. If you see a big peak right at the edge it's likely the video is clipping.

[sanlyn]

In TMPGEnc Plus the option to treat the incoming RGB as studio RGB is called "Output YUV data as Basic YCbCr not CCIR601".

Nope. Not using that option.

Your info on ColorTools options reflects my own understanding of them (So I guess I'm on track so far). I also check in Photoshop (caps from VirtualDub) and AviSynth, just to be sure. Learning to use AfterEffects as well. All these 'grams and 'scopes seem to say the same things about adjustments I try to make and sources I'm watching.

Of course, I got most of those tips from many of your previous posts

[sanlyn]

These are my settings in VirtualDub capture when I use DirectShow:

Device: (obviously, many use different devices)

[Attachment 8932 - Click to enlarge]

Compression: (not using ffdshow)

[Attachment 8933 - Click to enlarge]

"Set custom format..."

[Attachment 8934 - Click to enlarge]

If your "device" won't work with YUY2, you'll get an error when you click "OK". You'll have to use whatever your device supports.

[sanlyn]

Now...whatever became of StillCher4? It might be helpful if cherbette could load a sample before any levels or color adjustments.

[jagabo]

2) The 16 and 235 markers on the red, green, and blue graphs don't really mean anything. Red, green, and blue data can extend all the way down to 0 and all the way up to 255. If you see a big peak right at the edge it's likely the video is clipping.

I should have indicated that was for computer RGB. With studio RGB you want the graphs to stay between the 16 and 235 markers.

[jagabo]

These are my settings in VirtualDub capture when I use DirectShow:

Compression: (not using ffdshow)

[Attachment 8933 - Click to enlarge]

"Set custom format..."

[Attachment 8934 - Click to enlarge]

Since you're capturing YUY2 you should use HuffYUV in YUY2 mode. Don't enable the "Always suggest RGB format for output" option. You want YUY2 to go in, and YUY2 to come out, not RGB.

[sanlyn]

That's where I am, jagabo. Thanks for checking me on that.

[Cherbette]

That's where I am, jagabo. Thanks for checking me on that.

So...I'll 'fess up to maybe being a bit strict on 16-235 levels. I break the rules often enough to be less fussy.

Now, if we could get an unfiltered capture of StillCher4 to see how it looks au naturel. . .

About to up it for you guys now

[jagabo]

So...I'll 'fess up to maybe being a bit strict on 16-235 levels.

As you should be. But it's something of a moot point with VirtualDub's rec.601 scaling. You can't produce anything with Y<16 or Y>235 no matter what you do in VirtualDub's computer RGB.

[Cherbette]

Here's the script I used:
AviSource("StillCher2.avi")
AssumeTFF()
ConvertToYV12
Crop(24,40,-4,-24)
ColorYUV(gain_y=40, off_y=-28, gamma_y=60, cont_u=-90, cont_v=-40)
MCTemporalDenoise(settings="low", edgeclean=true, ecrad=3, stabilize=true, maxr=2, interlaced=true)
AddBorders(16, 32, 12, 32)

I'm not sure if this part posted by jagabo was supposed to be left-in or not because in post #158 of the previous thread he made some changes to avoid messing with the b/w portions of the special:
ConvertToRGB(interlaced=true)
RGBAdjust(g=0.98, b=1.2)

I left it out.

Raw transfer: http://www.mediafire.com/?atq8w6h9p3tq06i
Filtered: http://www.megaupload.com/?d=69W05MC8

(for some reason the filtered kept getting hung up in mediafire so I had 2 use megaupload. hopefully this is non-issue)

[sanlyn]

Got it, cherbette. But the "raw" version isn't raw, it's been converted to RGB. That might make a few corrections difficult. Will try it anyway.

[Cherbette]

Got it, cherbette. But the "raw" version isn't raw, it's been converted to RGB. That might make a few corrections difficult. Will try it anyway.

I did "direct stream" and marked the in/out points in virtualdub to clip out that portion. What am I doing wrong? Does it need to be done in Avisynth?

[Gavino]

The problem with the "automatic" RGB conversion is that VirtualDub by default treats interlaced YV12 as bottom field first. So if you want your final output saved in RGB32 (and your source is interlaced, top field first) you need:
AssumeTFF().ConvertToRGB32(matrix="[put matrix name here]",interlaced=true)

jagabo basically covered this in post #34, but just to be clear:
The conversion of interlaced YV12 to RGB is independent of field order, so AssumeTFF() is unnecessary.
The real problem is that VirtualDub always treats YV12 as progressive.

[jagabo]

The conversion of interlaced YV12 to RGB is independent of field order, so AssumeTFF() is unnecessary.

Yes. I usually use AssumeTFF() or AssumeBFF() at the top of the script because sometimes I'll be trying different filters and want to make sure the field order is set correctly for any filters that might need it.

[jagabo]

Got it, cherbette. But the "raw" version isn't raw, it's been converted to RGB. That might make a few corrections difficult. Will try it anyway.

LOL. I'm getting YUY2 with AviSource() now. I switched to HuffYUV 2.1.1.

Cherbette, your video is interlaced -- be sure to flag it as such in ConvertToYV12(). And wasn't that script for another video? Why are you using the exact same parameters in this video? That is what I mean when I say you should start thinking for yourself. Look at what each filter does and decide what parameters you should use for each video or each shot. You could at least start by looking at the levels!

[Cherbette]

Got it, cherbette. But the "raw" version isn't raw, it's been converted to RGB. That might make a few corrections difficult. Will try it anyway.

LOL. I'm getting YUY2 with AviSource() now. I switched to HuffYUV 2.1.1.

Cherbette, your video is interlaced -- be sure to flag it as such in ConvertToYV12(). And wasn't that script for another video? Why are you using the exact same parameters in this video? That is what I mean when I say you should start thinking for yourself. Look at what each filter does and decide what parameters you should use for each video or each shot. You could at least start by looking at the levels!

It's all from the same video. StillCher2.avi is the file name of the full length 60 minute documentary. All I've uploaded are a few 10 second snippets. Wouldn't I apply the script I posted above over the entire 60 minute special? I am starting to think for myself and trying to learn each filter and it's settings. I apply the script, preview it in Virtualdub, and try my best to alter it with what I've learned. What do you mean by looking at the levels? You mean using something like ColorTools? If so, I've been trying to learn it as well and have been using it to look at the histograms/waveforms. I've been tinkering around with Avspmod for the last day or so as well.


I added "interlaced=true" to the coverttoyv12 command just now as you informed:

AviSource("StillCher2.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
Crop(24,40,-4,-24)
ColorYUV(gain_y=40, off_y=-28, gamma_y=60, cont_u=-90, cont_v=-40)
MCTemporalDenoise(settings="low", edgeclean=true, ecrad=3, stabilize=true, maxr=2, interlaced=true)
AddBorders(16, 32, 12, 32)


And I'm still confused as to why the unfiltered AVI I posted of the StillCher4 snippet is in RGB when I selected "Direct Stream" in VirtualDub?

[sanlyn]

Got it, cherbette. But the "raw" version isn't raw, it's been converted to RGB. That might make a few corrections difficult. Will try it anyway.

LOL. I'm getting YUY2 with AviSource() now. I switched to HuffYUV 2.1.1.

This is what I get with ffdshow's huffyuv turned off on my PC:


[Attachment 8938 - Click to enlarge]

This is with my ffdshow's huffyuv turned ON:


[Attachment 8939 - Click to enlarge]

I'll keep doing what I did before: turn ffdshow's huff decoder on when I first open these, recompress with huffyuv2,2, then turn off ffdshow decoder afterward.

[sanlyn]

jagabo basically covered this in post #34, but just to be clear:
The conversion of interlaced YV12 to RGB is independent of field order, so AssumeTFF() is unnecessary.
The real problem is that VirtualDub always treats YV12 as progressive.

Point taken, gavino. Thank you.

[Cherbette]

What's the latest version of Huffyuv?

[sanlyn]

It's comin' along, cherbette. Always a hassle the first time around. My first 5-minute DVD looked GREAT on my 'puter (only took me 3 days!). Then, the first time I put the shiny new disc in my DVD player, the player thought about it for a few seconds and then spit it right back out at me.

I'm still confused as to why the unfiltered AVI I posted of the StillCher4 snippet is in RGB when I selected "Direct Stream" in VirtualDub?

You were correct. I adjusted my own ffdshow setup and I see that your "raw" is YUY2, not RGB.

Cherbette, your video is interlaced -- be sure to flag it as such in ConvertToYV12(). And wasn't that script for another video? Why are you using the exact same parameters in this video? That is what I mean when I say you should start thinking for yourself. Look at what each filter does and decide what parameters you should use for each video or each shot. You could at least start by looking at the levels!

It's all from the same video. StillCher2.avi is the file name of the full length 60 minute documentary. All I've uploaded are a few 10 second snippets. Wouldn't I apply the script I posted above over the entire 60 minute special?

Well, no. VHS playback is just too unstable and quirky. And VHS mastering is far from consistent. Sometimes you can do it with many movie scenes in the same sequence, but you still run into shots or sections where -- by golly -- it's the same movie, same scene, but it looks as if it's from a different video! That's just the nature of VHS. It's not usually necessary to make completely different settings for each piece of capture, but there are times when I've had to make a separate capture for a couple of minutes of really whacko VHS from the same tape; the problems in those few minutes were just completely off the wall from the rest of the tape. It's not you, it's VHS. As I said, documentaries are even worse; techs will patch in anything, from anywhere, to fill the time.

What do you mean by looking at the levels? You mean using something like ColorTools? If so, I've been trying to learn it as well and have been using it to look at the histograms/waveforms.

If you're correcting stuff in YUV, use AviSynth's 'grams and waveforms; they'll be looking at your YUV, not at VirtualDub's RGB. If you've come out of YUV and started correcting in RGB, use VirtualDub's tools. I'd use the tool that's appropriate for the colorspace you're working with.

[Cherbette]

Well, no. VHS playback is just too unstable and quirky. And VHS mastering is far from consistent. Sometimes you can do it with many movie scenes in the same sequence, but you still run into shots or sections where -- by golly -- it's the same movie, same scene, but it looks as if it's from a different video! That's just the nature of VHS. It's not usually necessary to make completely different settings for each piece of capture, but there are times when I've had to make a separate capture for a couple of minutes of really whacko VHS from the same tape; the problems in those few minutes were just completely off the wall from the rest of the tape. It's not you, it's VHS. As I said, documentaries are even worse; techs will patch in anything, from anywhere, to fill the time.

Well in that case this may be a long process haha

[sanlyn]

Well in that case this may be a long process haha

Sometimes it is.

You've apparently been using ffdshow's version of huffyuv to encode and decode. That can be complicated. The two huffyuv's are not compatible. To use the "original" huffyuv, you'd have to disable huffyuv in ffdshow (both encode and decode). If you do so, it means that in order to read your older ffdshow captures, you have to temporarily turn on ffdshow's decoder to open the file properly (leave the encoder disabled). Then after you save (i.e., encode again) the video in the original huffyuv, turn ffdshow's huffyuv decoder off again.

So, think about that first. If you continue to use ffdshow's huffyuv, users who don't have ffdshow on their computers will likely have problems with those files.

Anyone who needs to correct me on this huffyuv thingie, please do. I ran into this problem the first time I installed ffdshow. I've found many posts by those who had to do what I described above.

[jagabo]

My favorite HuffYUV is still v2.1.1 from neuron2.net.
http://neuron2.net/www.math.berkeley.edu/benrg/huffyuv.html

I've had problems with v2.2.0 in the past. I don't remember what those problems were though. I think it had something to do with compatibility with v2.1.1 and RGB modes.

[poisondeathray]

Yes v2.2.0 is documented to cause problems

My favorite is Ut Video Codec. It has separate fourcc for different modes RGB, YV12, YUY2 so there is no mixup, and is significantly faster than HFYU and even a bit faster than HYMT (the multithreaded Huffyuv) on multicore systems despite similar compression. Editing HD footage is a smoother experience, and export rendering is about 10-15% faster (because it decodes faster) with it (usually not an issue with VHS or SD but it makes a noticable difference on HD footage)



You guys still don't believe that "StillCher4.avi" is RGB(A) ?

Just about every program I have says it's RGBA (or BRGA) . If you want more proof, ConvertToYUY2(interlaced=true) (should be a null-op if you decode YUY2), and use vdub in fast recompress mode, and use huffyuv again. Lossless YUY2 compression right? You will see the filesize will be reduced to 59.9MB from 107MB. I am 99.9% certain it's RGB+A now.

You can do this yourself, but if you want samples of known huffyuv YUY2 and RGB+/-A samples let me know. I think some of you have decoder/codec issues




Jagabo is right: it's not a big deal on this specific sample. But these observations might indicate other capture or codec problems. For example other parts of the video might be inadvertently clipped. Also, why waste the extra storage space? Decompression speed is also slower for RGB, and editing is slower (again not an issue for SD footage, but still...)

[jagabo]

If you want more proof, ConvertToYUY2(interlaced=true) (should be a null-op if you decode YUY2), and use vdub in fast recompress mode, and use huffyuv again. Lossless YUY2 compression right? You will see the filesize will be reduced to 59.9MB from 107MB. I am 99.9% certain it's RGB+A now.

I get the same sized file in all YUY2 modes, same size as the source, 85.6 MB (via File Properties in Explorer).

AviSource("stillcher4.AVI")
#Info says YUY2

Open AVS with VirtualDub, select HuffYUV, Save as AVI.
Or, open file directly in VirtualDub, Direct Stream Copy, Save as AVI.
Or, open file directly in VirtualDub, Fast Recompress, HuffYUV, Save as AVI.
Or, open file directly in VirtualDub, Full processing, HuffYUV, Save as AVI.
(Audio disabled in all, just to be sure.)

I believe the "RGB" that MediaInfo reports comes from a flag in the AVI Header that indicates what colorspace the original video was. That has nothing to do with what colorspace comes out of the decoder. That's up to the negotiation between the decoder and the editor. As to why the flag reads RGB -- I don't know.

<edit>
Yes, I verified that. It's the biBitCount field in the AVI Stream Format header ("strf"). If you change it from 24 (decimal) to 16 (decimal) MediaInfo will report the file as YUV. It makes no difference to the decoding.
</edit>

<edit again>
Am I looking at a different file? I don't have a 107 MB file to start with, I have StillCher4 raw transfer.avi, 85.6 MB, from this post:
https://forum.videohelp.com/threads/339411-Tweaking-VHS-captures?p=2110336&viewfull=1#post2110336
</edit again>

[poisondeathray]

But you started out with 107MB , correct? (just checking maybe file got corrupted during download)

Wouldn't you would expect it to be about the same size if it was truly YUY2 ? 85.6MB is still significantly smaller

If I open it directly, I get 107MB again, even using direct stream copy (probably because vdub is decoding it as RGB)



Another question is why is my result 59.9MB when it's YUY2 then (file=> file information reports YUY2 as well as info() ) ?

Hmmm...