Confusion with colorspaces

[nick1977]

I've been playing around with AVISynth as I've need to do some chromashifting.

The footage is an MKV ripped from a homerecorded PAL DVD.

I think I must have made some mistakes either in virtualdub or in avisynth as the video has the blacks coming out grey.
I've tried various settings for colorspace in both Virtualdub and avisynth but nothing seems to sort it.
The only thing that does work is using the levels filter in virtualdub to adjust input levels to 16-235 but I can't think this should be necessary.

My avisynth script is basic for now and just has:

LWLibavVideoSource("c:\vidtemp\DW\dimensions.mkv")
ConvertToYUY2(interlaced=true)
Chromashift(C=-4)

I've tried removing the ConverttoYUY2 line and it makes no difference.
If I open the file directly in Virtualdub it also makes no difference.
I have recently changed both the in and out colorspaces in Virtualdub to YUY2 but playing with various options doesn't seem to resolve the issue.

Can anyone suggest anything?

Thanks

[Cornucopia]

Blacks as grays doesn't usually have anything to do with colorspaces or chromashifting, although incorrect interpretation of full vs. limited range, especially when converting colorspaces, MIGHT have something to do with it.
It DOES sound like its a levels (contrast/brightness) issue, which may be due to wrong range setting.

FYI,
MOST (digital) cameras record in component YUV color spaces, and the better ones are usually set to record limited range (aka 16-235), so that there is legal overshoot & undershoot which could be adjusted without loss in post if necessary.
Analog devices like VHS output Analog Composite (Y+U+V) or Analog S-Video (Y, U+V) - the range is set when you digitize it (manually or automatically depending on device, app, driver, codec, settings), along with separation of color primaries into component format.
DVDs have ALL their material in Component format (Y, U, V), which should already be limited range (MPEG2 8bit 4:2:0), on disc. MakeMKV usually keeps it as things already are, just re-encapsulates it. What it is after somebody converts it is anyone's guess.


Scott

[nick1977]

Thanks Scott, that's useful.

As this is directly from a home recorded DVD(Panasonic DVD recorder) into an mkv using MakeMKV, I'm not sure where the range error is occurring. Is there anyway to force avisynth to set the range correctly, given what my source is?

Alternatively, what should I have the input colorspace set to in Virtualdub?

At least I can then work on correcting things knowing that I'm not just correcting something the software has caused.

Thanks

[poisondeathray]

Check the actual levels using histogram() (It's a Y waveform)

This will be independent of however you have the display configured, because it reads the file's actual levels (e.g. you might have some display calibrated differently , or some program or display drivers setup differently)

LWLibavVideoSource("c:\vidtemp\DW\dimensions.mkv")
Histogram

http://avisynth.nl/index.php/Histogram

[nick1977]

Thanks. I did that. The Histogram shows two brown strips each side. Occasionally (but rarely) that turns yellow with some activity. It's only on one or two bits though and not always when I would expect complete black (for example). I'm not sure what this actually means in practice though!

[Sharc]

...Occasionally (but rarely) that turns yellow with some activity. It's only on one or two bits though and not always when I would expect complete black (for example). I'm not sure what this actually means in practice though!

The white wavy stuff should fluctuate but stay between the brown bars. Occasionally (rarely) moving into the brown bars and turning yellow is ok, so as you describe it doesn't sound odd.
You could upload a short sample here - preferably a scene with darks and bright in the picture - so members could take a look and advise.

Edit: You find a tutorial on how to read the waveform monitor here:
https://forum.videohelp.com/threads/340804-colorspace-conversation-elaboration#post2121568
The graph is placed on top of the picture in this tutorial. You can do this with

turnright().histogram().turnleft()

[poisondeathray]

The "brown stripes" are the "illegal" areas Y 0-15 , Y 236-255

Small excursions are allowed, but you want most of the levels between Y 16-235

The waveform is turned on the side. A more traditional view would be where the top are brighter levels, bottom are darker

LWLibavVideoSource("c:\vidtemp\DW\dimensions.mkv")
TurnRight()
Histogram()
TurnLeft()

[nick1977]

Here is a clip created using copy streams in avidemux

https://drive.google.com/file/d/146sDT55QwubKhNiyfKhKpEMiQxUGyrMi/view?usp=sharing

I would like to re-iterate I think this is something in settings rather than the video.

I was tinkering with the colorspace settings in Virtualdub and huffyUV which I export to before running through handbrake.
I edited this same video a few years ago (most likely with just Virtualdub rather than AVISynth) but since then I have built a new PC and moved all software across.

I'm pretty sure it's just something I have set incorrectly, but if I can be proven wrong I don't mind - I just want to get to the bottom of it!

Thanks

[Sharc]

Video is interlaced, TFF.
Main issue with this encode as I see it is over-denoised (wax faces, loss of details) and oversharpening (producing halos) rather than the videolevels or "colorspace".
Did you include some heavy denoising + sharpening in your workflow? Is this the unprocessed rip of the recorded DVD?.
One may lower the darks and rise the brights a few notches though, and reduce the color saturation to taste.

[nick1977]

It's the raw capture (i.e. unprocessed rip) from my DVD. The DVD was created on a Panasonic DMR-ES10 Recorder on maximum bitrate using a composite feed from my VCR.
The VHS was a 2nd generation copy which probably doesn't help and it was originally recorded from a set top TV aerial!

[Sharc]

Ok. This explains a lot about the quality.

Here with few levels tweaks. Doesn't improve much though.

ffms2("dimensions_sample.mkv")
assumetff()
qtgmc(preset="fast")
SmoothTweak(hue1=0, hue2=0, brightness=0,contrast=1.1,saturation=0.9,HQ=true,TVrange=True, Limiter=False)
SmoothLevels(input_low=16,gamma=1.1,input_high=235,output_low=4,output_high=240,HQ=true,TVrange=True,Limiter=0)
Dehalo_Alpha()
cas(0.6)

[nick1977]

Thank you very much. I'll try that bit of script later.
Only question I have at this is point is whether those filters are still OK if I don't deinterlace. I'm intending to keep the footage interlaced so it can be burned to DVD if required.
Can I just remove the QTGMC line or do I need to make some additional adjustments to the script?

[Sharc]

Many filters - like Dehalo_Alpha - are designed for progressive video. So if you want it interlaced you would have to re-interlace the progressive video at the end by appendig at the end of the script

SeparateFields().SelectEvery(4,0,3).Weave() #re-interlace
ConvertToYV12(interlaced=true)

Added:
And for DVD you would have to encode it as mpeg2, and if you want a real DVD you need to author it accordingly (folder and file structure) with a DVD authoring tool (Avs2DVD, DVDStyler, DVDauthorGUI ....)

[nick1977]

Thanks but if I have to re-interlace at the end would it not be better to use some other means of making it progressive rather than introducing fake data? In Virtualdub I use the unfold field and then refold at the end of processing. I'm not sure how you do this in Avisynth?

Also, I know it needs to MPEG2 at the end but there is also nothing really wrong with me using x264 at a high bitrate, leaving the option to re-encode to MPEG2 later on.basically deinterlacing and reinterlacing seems a bad idea to me unless someone would like to allay my concerns.

[Sharc]

Thanks but if I have to re-interlace at the end would it not be better to use some other means of making it progressive rather than introducing fake data? In Virtualdub I use the unfold field and then refold at the end of processing. I'm not sure how you do this in Avisynth?

Separatefields()
e=selecteven().<your filter(s)>
o=selectodd().<your filter(s)>
interleave(e,o)
weave()

Also, I know it needs to MPEG2 at the end but there is also nothing really wrong with me using x264 at a high bitrate, leaving the option to re-encode to MPEG2 later on

Yes, but every re-encode deteriorates the quality

basically deinterlacing and reinterlacing seems a bad idea to me unless someone would like to allay my concerns.

Not really usually. Exceptions depend on the source. To preserve the original fields you could use for deinterlacing

QTGMC(preset="fast",lossless=1)

This would use the original fields for reinterlacing.

You have to try which method yields the best result.

[nick1977]

Thanks. I've gone with the unfold method as that seemed to work better. Appreciate your help.