Adjusting capture card levels, not sure if it's right.

[ENunn]

I'm messing around with my capture card's (IO-DATA GV-USB2) proc-amp with GraphStudio. I have it running through AviSynth and the histogram. I'm able to get some good settings for most of my VHS tapes but I've noticed that there's a some extra stuff in the 236-255 range that still almost clips.



I adjusted everything to compensate for that, but it looks like it washes everything else out.



Should I not worry about that stuff? I want to get it just right before I start transfering this stuff.

[Sharc]

Seems about right to me. I don't see a problem in the 236 .... 255 range with clipping for the 2 screenshots which you posted.
The black seems to be elevated in the 2nd screenshot (-> washing out), I would leave the settings of the first screenshot and
perhaps elevate the whites with something like

YLevels(16,1.0,235,16,250)

You don't have to worry much about peaks extending occasionally into the 0...16 or 235....255 range. These are often just halo artefacts along edges.

[ENunn]

Little update on this. I ended up over brightening but under contrasted as well. Gives me more flexibility in post. I'm very pleased with the end results, however I've noticed the histogram has invisible lines.


What does this mean? I don't notice anything wrong in the final video. I have my brightness and contrast set to 40 above default and 40 below default respectively, 168/65.

[davexnet]

It might be better if you could get it a little closer.
Can you post a sample video with the settings set as above?

[jagabo]

The lines are rounding errors from integer math adjustments of the levels. They might show up as posterization artifacts on smooth shallow gradients.

In your first image the black level is a tiny bit low but you can still recover the blacks in post. The bright levels look low in both the first and second image. In the second image. White text like that in the second image is usually at full brightness (though not always).

[ENunn]

It might be better if you could get it a little closer.
Can you post a sample video with the settings set as above?

Sure thing. I have tweaked, tweaked with the histogram, and untweaked attached.

The lines are rounding errors from integer math adjustments of the levels. They might show up as posterization artifacts on smooth shallow gradients.

Thanks for the info. Honestly I really don't notice any for some reason. I know it's best to set levels before you capture but I have to do it for every tape transfer and since it depends on the tape and the content it'll be varying. That's why I chose to up the brightness and lower the contrast then expand it back out in post. It gives me some headroom.

In your first image the black level is a tiny bit low but you can still recover the blacks in post. The bright levels look low in both the first and second image. In the second image. White text like that in the second image is usually at full brightness (though not always).

Thanks for the tips! I remember using the RGBAdjust filter and it did help a little. However I don't want any lost detail, and it looks like adding that is clipping some of the whites. I'm not too sure though.

[Sharc]

.... That's why I chose to up the brightness and lower the contrast then expand it back out in post. It gives me some headroom.

Your capture settings seem to be somewhat over-cautious to me, regarding the levels safety margin (e.g. the black level of the 'unfiltered.avi' is unnecessarily high), therefore the captured picture looks washed.
But yes, you can tweak it with post processing. You could also apply 'dithering' to mask the lines in the histogram. Taking your 'unfiltered.avi', something like

Tweak(hue=0,sat=1.0,bright=-55,cont=1.5,coring=false,dither=true)

.

[jagabo]

Yes, given that you're adjusting the contrast by a factor of ~1.6 you only have about 138 different values between the black and white levels; rather than the 220 units you would have with a perfect cap with black at Y=16 and white at Y=235. If you had a very clean source the posterization would be visible in shallow gradients. But the noise inherent in VHS masks it.

[Skiller]

however I've noticed the histogram has invisible lines.

[Attachment 59692 - Click to enlarge]

It's called banding and although it is not very visible with noisy analog sources, it's something to be avoided like the plague.
Why is it there? Well it is there because all the so called "Pro-Amp" of your capture device does is it takes the already digitized video frames and applies a levels correction; and it does so at only 8 Bits precision.

Only few capture devices have proper Proc-Amps which alter the actual signal voltage interpretation inside the DAC during the sampling of the analog signal to digital – not afterwards in the driver software.

Therefore I recommend to always leave the Proc-Amp settings at default (or whatever settings do not touch the levels) and correct afterwards in AviSynth in high bitdepth. With these "fake" Proc-Amps there is nothing to be gained from them anyways. Any clipped brights for example are already gone at this point and cannot be recovered no matter the settings of the Proc-Amp.

[ENunn]

It's called banding and although it is not very visible with noisy analog sources, it's something to be avoided like the plague.

Fair :P I don't really notice any banding however. Adding dither=true fixed that up, and honestly I still cannot notice any differences between undithered and dithered.

Therefore I recommend to always leave the Proc-Amp settings at default (or whatever settings do not touch the levels) and correct afterwards in AviSynth in high bitdepth. With these "fake" Proc-Amps there is nothing to be gained from them anyways. Any clipped brights for example are already gone at this point and cannot be recovered no matter the settings of the Proc-Amp.

That's what I did at first, not in high bitdepth though. My capture card can only do 8-bit. If there was a 10-bit version of the GV-USB2 I'd pick it up in an instant :P

I could hardly tell the difference between tweaked default proc-amp vs tweaked modified proc-amp, maybe the modified proc-amp had more detail added, however I can see why jagabo mentioned poisterization. I think I can notice it in one spot? It's hard to tell.

Tweaked Default Proc-Amp


Tweaked Modified Proc-Amp


I guess I gotta experiment, or just leave it at defaults. I looked at the histogram on the modified proc-amp settings before I tweaked it and it didn't look like anything was lost compared to the defaults, but it could be just doing that to trick me or whatever.

[Skiller]

My capture card can only do 8-bit.

That's perfectly fine. Capturing at 8 Bits is good enough and usually the only option anyways. It's just that any tweaking should better be done at higher bitdepth (and then dithering down back to 8 Bits) to avoid the creation of banding.

I think I can notice it in one spot? It's hard to tell.

Both screenshots exhibit banding diagonally across the whole screen. However, with these early computer graphics you can never be 100% sure it wasn't originally rendered like this without dithering. I would look for a "natural" gradient (such as the sky).

[ENunn]

That's perfectly fine. Capturing at 8 Bits is good enough and usually the only option anyways. It's just that any tweaking should better be done at higher bitdepth (and then dithering down back to 8 Bits) to avoid the creation of banding.

Is seeing the invisible lines normal when I convert to 10-bit then tweak? This is with the modified proc-amp. Setting dither=true in Tweak mostly fixes it.

I would look for a "natural" gradient (such as the sky).

Here you go!

Untweaked Default Proc-Amp


Tweaked Default Proc-Amp Converted to 10-Bits

converttoyuv422(matrix="rec601", interlaced=true)
convertbits(10)
tweak(bright=3, cont=0.92, hue=0.0, sat=1.00, coring=false, dither=true)

Untweaked Modified Proc-Amp


Tweaked Modified Proc-Amp Converted to 10-Bits

converttoyuv422(matrix="rec601", interlaced=true)
convertbits(10)
tweak(bright=-30, cont=1.20, hue=0.0, sat=1.00, coring=false, dither=true)

[Sharc]

I think your last picture should be
"Tweaked Modified Proc-Amp Converted to 10-bits", no?

Looking at your 1st Forrest Gump picture I see no technical reason why you should modify or tweak anything, other than perhaps for personal taste.

[lollo]

In addition, if a restoration is planned, even a light filtering (i.e. GamMatch) or a denoise (i.e. TemporalDegrain2) fixes the original gaps in the histograms,
probably because internally to the filters there is a convertion to a higher number of bits and a dithering operation.

An example here: https://forum.videohelp.com/threads/401349-GV-USB2-Capture-Stick-What-s-In-It#post2615773

[jagabo]

even a light filtering (i.e. GamMatch) or a denoise (i.e. TemporalDegrain2) fixes the original gaps in the histograms,
probably because internally to the filters there is a convertion to a higher number of bits and a dithering operation.

Noise reduction is basically averaging nearby (spacial and/or temporal) pixels together -- that's what fills in the histogram/waveform gaps. For example between 100 and 102 there's a gap. Averaging those together gives you 101, filling in that gap.

[Skiller]

I would look for a "natural" gradient (such as the sky).

Here you go!

I should have been more specific. Banding can be seen best in dark, smooth areas. A bright sky full of clouds is not smooth enough.
A clear sky, preferably at night or dusk/dawn would exhibit banding easily if there is any. But really, any dark to medium bright texture-less area does it. Another instant where it can be spotted is during fades to and from black.

Anyways, visually the two "Tweaked Default Proc-Amp" screenshot look about right to me.

[jagabo]

VHS noise will largely mask small histogram gaps caused by integer rounding. The far left crop is the original with no histogram gaps. In the center left Tweak has caused histogram gaps but the video shows little posterization -- the noise in VHS has hidden it. In the center right Tweak has caused histogram gaps but KNLMeansCL has smoothed them away, there's very little posterization. In the right KNLMeansCL smoothed the video first (as if you had a noiseless video to start with) then Tweak caused histogram gaps, you clearly see posterization in the sky.

[lollo]

even a light filtering (i.e. GamMatch) or a denoise (i.e. TemporalDegrain2) fixes the original gaps in the histograms,
probably because internally to the filters there is a convertion to a higher number of bits and a dithering operation.

Noise reduction is basically averaging nearby (spacial and/or temporal) pixels together -- that's what fills in the histogram/waveform gaps. For example between 100 and 102 there's a gap. Averaging those together gives you 101, filling in that gap.

True for denoising. However, GamMatch (working in RGB domain, so a conversion is needed) does not average any pixel in my understanding, and the effect is similar, although less effective.
I tried to separate the effects of RGB conversion from the filter.


[Attachment 59726 - Click to enlarge]


[Attachment 59727 - Click to enlarge]

[ENunn]

I think your last picture should be
"Tweaked Modified Proc-Amp Converted to 10-bits", no?

Whoops! Fixed.

Looking at your 1st Forrest Gump picture I see no technical reason why you should modify or tweak anything, other than perhaps for personal taste.

Yeah, personal preference :P

In addition, if a restoration is planned, even a light filtering (i.e. GamMatch) or a denoise (i.e. TemporalDegrain2) fixes the original gaps in the histograms,
probably because internally to the filters there is a convertion to a higher number of bits and a dithering operation.

Lucky me, I always do some processing (denoising, etc.) on my VHS captures.

Anyways, visually the two "Tweaked Default Proc-Amp" screenshot look about right to me.

Great! I don't do this type of stuff to anything else, only my VHS captures, so at least I got some reassurance.

[jagabo]

However, GamMatch (working in RGB domain, so a conversion is needed) does not average any pixel in my understanding, and the effect is similar, although less effective.

Full range 8 bit RGB has about six times as many colors (finer gradations) as 8 bit limited range YUV. So making adjustments in RGB creates in-between colors that aren't in the YUV. Hence the filling of the histogram gaps upon returning to YUV.

[Sharc]

Only few capture devices have proper Proc-Amps which alter the actual signal voltage interpretation inside the DAC during the sampling of the analog signal to digital – not afterwards in the driver software.

Therefore I recommend to always leave the Proc-Amp settings at default (or whatever settings do not touch the levels) and correct afterwards in AviSynth in high bitdepth. With these "fake" Proc-Amps there is nothing to be gained from them anyways. Any clipped brights for example are already gone at this point and cannot be recovered no matter the settings of the Proc-Amp.

Agree, when the capture device is an external USB converter for example one has no control over the device-internal A/D conversion (analog signal level, gain etc.) with the "fake" Proc-Amp driver tweaks. However, I imagine that it makes nevertheless sense to adjust the "fake" Proc-Amp settings in order to avoid generating Y'CbCr triplets which fall a priori outside the legal RGB range, hence getting a color distorted picture on the RGB monitor (which we then try to re-adjust afterwards in SW post processing.)
Am I totally mislead?

[lollo]

However, I imagine that it makes nevertheless sense to adjust the "fake" Proc-Amp settings

I do it. Even because the gaps in the Histograms are present also at default setting, then it makes no sense to search a "fake" setting to avoid them rather than search a setting to avoid incorrect levels.

[Skiller]

I do it. Even because the gaps in the Histograms are present also at default setting, then it makes no sense to search a "fake" setting to avoid them rather than search a setting to avoid incorrect levels.

Years ago, with my SAA7134 based PCI capture card, I found the "neutral" Proc-Amp settings by selecting some random channel on it's analog tuner (since there was nothing connected, it resulted in classic tuner static) and enabling the histogram in VirtualDub's capture mode. Like this, while playing with the sliders for brightness and contrast, I could see the spikes and gaps in the histogram move and change until there were none left. This did, of course, produce slightly off levels in captures, but that could easily be corrected in post in high bitdeph, avoiding the banding that the Proc-Amp would produce at any other combination of settings. Otherwise, after the correction, it was exactly the same as if I captured with the right levels by adjusting the Proc-Amp in the first place. It still did capture everything that there is (after the DAC).

Therefore...

However, I imagine that it makes nevertheless sense to adjust the "fake" Proc-Amp settings in order to avoid generating Y'CbCr triplets which fall a priori outside the legal RGB range, hence getting a color distorted picture on the RGB monitor (which we then try to re-adjust afterwards in SW post processing.)
Am I totally mislead?

If you are going to do a levels correction in post, then I'd say my priority during capture would be to avoid the banding. Yes, the capture file itself will have incorrect levels but unless I would not want to do any post processing after capture, I do not see a problem with this. As long as there is no conversion to RGB before doing a levels correction in the capture's native Y'CbCr color space there really is nothing the fake Proc-Amp could do that SmoothLevels() in AviSynth for example could not (but better).

[lollo]

Yes, the capture file itself will have incorrect levels but unless I would not want to do any post processing after capture

Yes, that's my strategy. It is a question of preferences. I want a capture that without any restoration is at his best, and I prefere a spiky histogram to incorrect levels, because I do not see this defect in real time vision, while I do see a level problem; but YMMV.

And, if a do a restoration, I can fix spiky histogram as well as on the opposite strategy the levels can be fixed.

[Skiller]

For that approach I can fully understand your reasoning.

Also, you actually only have gaps rather than spikes in your histogram. It means the levels were expanded at some point (rather than squashed), so that everything is still there, nothing was thrown away. It's not nearly as bad as the opposite case (spiky histogram) where actual data was lost.

[lollo]

We are using opposite terminology.

I call "spiky" a histogram showing a uniform distribution of the levels, but where a saw tooth pattern is "added" on top of that.
An example here, that is almost always produced by my Hauppauge USB-Live2:



I refere to a "gap" histogram when it shows holes in the distribution.
An example here, provided from a user of a IO Data GV-USB2 in another post:



I agree with you that the last should be avoided

[Sharc]

If you are going to do a levels correction in post, then I'd say my priority during capture would be to avoid the banding. Yes, the capture file itself will have incorrect levels but unless I would not want to do any post processing after capture, I do not see a problem with this. As long as there is no conversion to RGB before doing a levels correction in the capture's native Y'CbCr color space there really is nothing the fake Proc-Amp could do that SmoothLevels() in AviSynth for example could not (but better).

Yes. As long as I can edit and level-adjust the capture's native Y'CbCr in post with AviSynth in YUV I see no problem. Without AviSynth (e.g. when using Vdub or NLE's which convert and operate in RGB) I would still have to rely on the "fake" Proc-Amp driver tweaks to prevent incorrect (illegal) RGB conversions, as I understand.
It's not really a problem though because I normally have AviSynth in place anyway, so I could in this case skip the "fake Proc-Amp" adjustments and do all levels adjustments in AviSynth as you suggest.

Btw, smoothadjust.dll v3.20 (SmoothLevels, SmoothTweak) seems to have a problem with >8bit sources (using convertbits(12) for example), or require some additional preprocessing ....
Edit:
I got the solution for SmoothAdjust here: https://forum.doom9.org/showthread.php?p=1946997#post1946997

[Skiller]

We are using opposite terminology.

Well, maybe. I'm not sure we are talking about the same thing so let me show an example.


First, we start with this gradient (made in Photoshop), free of banding, resulting in a smooth histogram:


[Attachment 59758 - Click to enlarge]



Now, let's expand this using Tweak(Cont=1.05, coring=false):


[Attachment 59759 - Click to enlarge]

It results in gaps/holes in the histogram, but none of the original data is actually lost. It's missing high bitdepth dithering to fill the gaps.



And if we do the opposite thing and squash the original using Tweak(Cont=0.95, coring=false), this happens:


[Attachment 59760 - Click to enlarge]


Due to the squashing there are now spikes and some data is lost.



Looking at your last screenshots, I'm not sure what to think of the tooth pattern added on top of the histogram. I don't recall having seen it before, nor do I have a clue how that would happen. Maybe the "classic" mode of the histogram provides more detail to answer that question.
Your last screenshot looks exactly like the expanded example above.

[lollo]

Thanks for the examples, in fact we used the same words. What was unclear to me is when you said that my histogram showed gaps, while there are no gaps.

I don't recall having seen it before, nor do I have a clue how that would happen. Maybe the "classic" mode of the histogram provides more detail to answer that question.

I do not have a valid theory either. Same frame as before with "classic" histogram:

[Skiller]

What was unclear to me is when you said that my histogram showed gaps, while there are no gaps.

You are right. That statement of mine applies only to the histogram of the OP in post #3. I mixed it up, sorry.


Just had bit of playing around with that screenshot and possibly the histogram is "overlaid" like that simply because the video is of course interlaced and the two fields represent different points in time (the background and rain are changed between the two).
If you put SeparateFields() beforehand that would eliminate it if interlacing is indeed the culprit here.