VDub Color Space problem processing Nexus 5x video

[shekh]

Notice pixel format is set automatically to YUV420-FR.

I noticed one issue here: it is not set automatically to -FR, to do my encode I selected it in the pixel format dialog (https://sourceforge.net/p/vdfiltermod/wiki/compression/)

Yes VLC says "full scale", but I did not compare screenshots from VLC, I compared videos in VD.

Are you actually using "x264vfw" or "x264 8 bit" encoder?

[RandomVHUser]

Notice pixel format is set automatically to YUV420-FR.

I noticed one issue here: it is not set automatically to -FR, to do my encode I selected it in the pixel format dialog (https://sourceforge.net/p/vdfiltermod/wiki/compression/)

Yes VLC says "full scale", but I did not compare screenshots from VLC, I compared videos in VD.

Are you actually using "x264vfw" or "x264 8 bit" encoder?

x264vfw, what are the options/exact video compression pixel format you're selecting? I can't seem to select 709 or 601 4:2:0 YUV full, the encode fails, stating
"Unable to initialize video compression: the selected output format is not compatible with the Windows video codec API."

[shekh]

x264vfw, but it is difficult.

Before I explained steps with "polished" encoder, again same picture https://sourceforge.net/p/vdfiltermod/wiki/compression/

for x264vfw there is same bottleneck as for external encoder, you have to
convert format manually by "convert" filter
alias format to 601
set VUI options manually in x264vfw (--colormatrix bt709)

This is not tested, my best guess. This is why I rewrapped x264vfw as proprietary codec, to get rid of these difficulties.

Mistake, I forget what I do. With any VFW it is actually better (no need for alias format etc): conversion works normally, only VUI is not set automatically.

[RandomVHUser]

x264vfw, but it is difficult.

Before I explained steps with "polished" encoder, again same picture https://sourceforge.net/p/vdfiltermod/wiki/compression/

for x264vfw there is same bottleneck as for external encoder, you have to
convert format manually by "convert" filter
alias format to 601
set VUI options manually in x264vfw (--colormatrix bt709)

This is not tested, my best guess. This is why I rewrapped x264vfw as proprietary codec, to get rid of these difficulties.

I reviewed that page more thoroughly and was able to encode with the built in x264 8 bit like on the page, the colors are correct now, thanks for all the help

Also, is there any benefit to encoding to 10bit? I believe my 4k videos are 8bit anyway, but if I wanted to downconvert to 1080p, I could get a better luminescence range by converting 8bit 4k to 10bit 1080p, correct?

[shekh]

del

[shekh]

I reviewed that page more thoroughly and was able to encode with the built in x264 8 bit like on the page, the colors are correct now, thanks for all the help

That`s a relief

Also, is there any benefit to encoding to 10bit? I believe my 4k videos are 8bit anyway, but if I wanted to downconvert to 1080p, I could get a better luminescence range by converting 8bit 4k to 10bit 1080p, correct?

I dont think so. Even if this may add any detail you have to "convert format" to YUV420P16, then do 16bit resize which does not exist yet

May be useful to transcode format which already has 10bits or more.

[poisondeathray]

It sounds like you got things sorted out but I'll answer these anyways

For your 5second clip, it should look like the lighter, washed out, low contrast version in your gif when displayed by a media player. Full range YUV converted to RGB full range for display.

The darker image is technically "wrong" for that video, data is lost Y<16, Y>235. Notice the details in the tree bark etc.. are crushed and missing

The darker image is how the source is shown in VLC. The output should visually match the source if no settings are changed? And why would the source be labeled "YUV full scale" and the output not state full scale? Are you saying VLC/AviDemux is improperly rendering the video as full range?

You're not "viewing" the source. You're viewing an RGB representation of the source. The source is in compressed AVC in YUV colorspace, which gets decoded then converted to RGB for display. How that gets converted to RGB (ie YUV<=>RGB conversions) is the major cause of various inconsistencies and display issues

The "darker" higher contrast version that you "see" is full range data converted with the "wrong" limited range matrix to RGB. Yes, the data is still there if you stay in YUV, but you don't "see" it. If you make adjustments in YUV, it's recoverable. BUT - If you convert it to RGB with a limited range matrix, it gets permanently clipped (e.g. if you use some RGB filters in vdub). Full range data should be using a full range matrix to convert to RGB. That way all the data is "seen" . So the "washed" out image actually has higher dynamic range. You can think of it as having more data - there is more bright and dark details.

What software are you using to verify the specs? I can run it against my original, but they both should be the same.

I'm using mediainfo (view=>text) to see the flags quickly , avisynth with histogram() to see the waveform monitor (to see the actual Y' range)