Virtualdub bad encode levels

[solouplay]

Hi! I'm having some problems encoding a video with VirtualDub 32-bit.
I'm using the x264vfw encoder.

Exporting the video with "keep input colorspace" makes the video look good. Black is black (full range)
But using the "Convert to YUV 4:2:0" creates bad black-levels (limited range)
Bad black levels: http://www.ladda-upp.se/files/2016/b206656.png
Good black levels: http://www.ladda-upp.se/files/2016/b206657.png

Any ideas on how to fix the problem? :)

[sneaker]

I don't understand. You already said there's no problem when using "keep input colorspace". Is there any reason why you can't use that option?

[JVRaines]

Some players don't support colorspaces like yuv422. Yuv420p gives the most compatibility. I had a similar problem with Handbrake: specifying the fullrange option had no effect and my levels were always squashed to studio range. There is no such problem using x264 via FFmpeg.

[solouplay]

I don't understand. You already said there's no problem when using "keep input colorspace". Is there any reason why you can't use that option?

I think YUV colorspace offers better compression? Maybe I'm way off?:)

[sneaker]

But what is your input?

Anyways, VirtualDub's filter system allows you to convert colorspaces. "Convert format" and "alias format" filters are usually the most interesting. That way you can have your desired color format before it reaches the encoder.

[solouplay]

Some players don't support colorspaces like yuv422. Yuv420p gives the most compatibility. I had a similar problem with Handbrake: specifying the fullrange option had no effect and my levels were always squashed to studio range. There is no such problem using x264 via FFmpeg.

Thanks for you answer :)

But what is your input?

Anyways, VirtualDub's filter system allows you to convert colorspaces. "Convert format" and "alias format" filters are usually the most interesting. That way you can have your desired color format before it reaches the encoder.

Oh! I think I got it now!
I was tampering with the Video->Color depth, and that's probably why it didn't work correctly... tampering with the filters works perfectly, thank you! :)

[hello_hello]

My apologies. I misread the opening post and assumed you were using the command line encoder and VirtualDub's external encoder feature, so most of this post doesn't apply, but I'll leave it here anyway. I spent a bit of time playing around and it might come in handy for someone. There's some info on how levels should display on a PC vs a standalone player which might be helpful.

Some players don't support colorspaces like yuv422. Yuv420p gives the most compatibility. I had a similar problem with Handbrake: specifying the fullrange option had no effect and my levels were always squashed to studio range. There is no such problem using x264 via FFmpeg.

The thing is though, YV12 is supposed to be limited range. When playing YV12 video back on a PC the player usually needs to expand the levels to full range so it displays correctly (assuming the monitor is full range). RGB is already full range and already displays correctly on a PC. A well behaved converter should automatically expand or reduce the levels as required when converting between YUV and RGB.
There should be no change in levels going from yuv422 to yuv420. What player are you using? What happens when you play the encoded version on a standalone device. How do the levels look? A TV generally expects limited range by default.

From the VirtualDub help file:
"Conversions between YCbCr formats are done directly in YCbCr space without an RGB intermediate step. Chroma is subsampled or supersampled as necessary using bilinear filtering."

While I can see the level difference in your pics, theoretically what's happening shouldn't be happening.

I found this in the help file too, under the external encoders section:

"The color space is 4:2:0 YCbCr using Rec. 601 limited-range, 8-bit per channel encoding. The normal excursions are 16-235 for luma and 16-240 for chroma, but values from 0-255 may be encoded. Chroma positioning is per MPEG-2 conventions, with chroma samples coaligned with luma samples horizontally and centered vertically. Frames are always encoded as progressive scan."

And then found it not to be completely accurate.
When using Fast Recompress with a YV422 source the output appeared to be YV422.
When using Normal Recompress with a YV422 source the left pane allows you to choose how the video is decoded (Video/Colour Depth) and that's how it's sent to the external encoder. The output choices on the left have no effect.
When using Full Recompress with a YV422 source the left pane allows you to choose how the video is decoded (Video/Colour Depth) but the output is always YV12. Even if you use the Convert Filter to convert to a different format, the output appears to always be YV12 (you can probably still use the convert filter to change the levels). I couldn't get VirtualDub to mess with the levels and I tried all sorts of input/output combinations. I wonder if the problem's a playback one, not an encoding one? What about the x264 command line? I'm not sure what happens when you feed it video it's not expecting (ie if you specify a particular input colour space and give it something else).

You can redirect the x264 log to VirtualDub's log in the external encoder configuration. Then if you enable VirtualDub's log you might see something odd.... or not... you can only try. I cut the external encoder command line down to this. Just stuff for VirtualDub, I think. x264 can do what it likes.

--input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o "%(tempvideofile)" -

I enabled x264's logging, opened a video containing YV16 video and put VirtualDub into full processing mode, leaving the input and output colour spaces on auto. Here's the log file. I think I used a resized down version of the "bad" screen shot as the image, which I converted to YV16 and then to AVI. I opened the AVI in VirtualDub and re-encoded it. MPC-HC is displaying the encoded version.