Colors slightly washed out after uncompressed render in Vegas

[mr-scarface]

So I rendered a video in Vegas using the uncompressed format. When I play it in MPC-HC or VLC, the colors are a bit washed out and it's definitely noticeable.
But when I play the original source file through MPC-HC or VLC, the colors are fine. Only the uncompressed render has the color issue. However, when I import the uncompressed render back into Vegas, it looks fine inside Vegas.

There's an option in Vegas under 'Preview Device' called 'Adjust levels from studio RGB to computer RGB', which is checked. If I uncheck it, you get that same washed out color look inside Vegas.

FYI the codec for the uncompressed render is - "24 bits rgb (rv24)"

I tried adjusting render settings in MPC-HC to get the colors to look right but nothing worked - I tried adjusting output range, 10-bit RGB, and RGB output levels under 'video decoder'.

Would appreciate any help.

Thanks

[Sharc]

What format is your source? YUV (YCbCr) or RGB color space? Maybe its out of gamut when converted from YUV to RGB by VEGAS.
Can you upload a sample of your source?

[mr-scarface]

I can't upload a sample but VLC says - Decoded format: Planar 4:2:0 YUV.
MPC-HC says: yuv420p

[jagabo]

There's an option in Vegas under 'Preview Device' called 'Adjust levels from studio RGB to computer RGB', which is checked. If I uncheck it, you get that same washed out color look inside Vegas.

That's your problem. In YUV black is defined as Y=16 and white as Y=235. On a computer black is defined as RGB=0 and white as RGB=255. Studio RGB (aka limited range RGB) has black at RGB=16 and white as RGB=235. Studio RGB is a special editing mode designed to retain super-blacks and super-whites, it's not meant for viewing. You need to convince Vegas to convert to computer RGB (aka full range RGB) rather than studio RGB.

Or, if your processing that limited range RGB with some other program, do the contrast stretch there.

[Skiller]

Vegas works in RGB internally, therefore I always supply it with RGB to have control over how this is done (don't trust Vegas).
To avoid an intermediate render to RGB, I like to use Avisynth Virtual File System.
Upon mounting an AviSynth scipt file (avs) with it, it turns it temporarily into a "fake" avi that Vegas will import as if it was an uncompressed RGB avi. Very handy.

[poisondeathray]

Typical workflow in vegas it to apply a "studio RGB to computer RGB" filter (under the levels filter preset) just before exporting

The discrepancies occur because vegas handles YUV => RGB conversions using either studio range or computer range , and it varies depending on the input source. Basically, most native camera formats get studio level RGB treatment (0-255 YUV => 0-255 RGB) . Internally, you're usually working in studio level RGB (16-235), but most computer monitors work with computer RGB (RGB 0-255) . You can read the "Glenn Chan" articles on Vegas, they explain it in more detail

There's an option in Vegas under 'Preview Device' called 'Adjust levels from studio RGB to computer RGB', which is checked. If I uncheck it, you get that same washed out color look inside Vegas.

That just affects the preview, not the actual data on the timeline. You need to apply the levels preset to enable actual change

[mr-scarface]

Typical workflow in vegas it to apply a "studio RGB to computer RGB" filter (under the levels filter preset) just before exporting

Thanks. This worked.
However, since this is technically an effect being added, could this potentially mess with the picture quality in any way? It looks fine at first glance, but I wonder if this is the way forward for me. I specifically render in uncompressed to keep the quality pristine.

The other way I managed to do this is to change the pixel format in the project settings to '32-bit floating point (full range)' then render it - then the colors look fine on the rendered file. The only weird thing is, it messes up the colors on the timeline preview inside Vegas, like it is oversaturated.

I read on the vegas forums that you should be working in 32-bit floating point (full range) if keeping perfect quality is your aim. I probably need to research color theory and learn more about this subject, but I wonder what is the best way forward for now. Will look into the other suggestion on this thread too.

edit: I was actually able to fix the oversaturation issue on the timeline preview by changing the setting for 'View transform' from 'sRGB (ACES)' to 'Off'. Then I noticed the colors look a bit deeper and richer on the vegas preview when in 32-bit mode - moreso than then actual source file itself. I wonder if this 32-bit mode is a better representation of what the colors should actually look like. FYI the source is a recorded stream using Bandicam's screen capture.

[jagabo]

However, since this is technically an effect being added, could this potentially mess with the picture quality in any way?

It's the standard way of converting YUV to RGB. Every other program, every media player, every graphics card, every TV does it that way. Limited range YUV to full range RGB.

[poisondeathray]

Thanks. This worked.
However, since this is technically an effect being added, could this potentially mess with the picture quality in any way? It looks fine at first glance, but I wonder if this is the way forward for me. I specifically render in uncompressed to keep the quality pristine.

The other way I managed to do this is to change the pixel format in the project settings to '32-bit floating point (full range)' then render it - then the colors look fine on the rendered file. The only weird thing is, it messes up the colors on the timeline preview inside Vegas, like it is oversaturated.

I read on the vegas forums that you should be working in 32-bit floating point (full range) if keeping perfect quality is your aim. I probably need to research color theory and learn more about this subject, but I wonder what is the best way forward for now. Will look into the other suggestion on this thread too.

edit: I was actually able to fix the oversaturation issue on the timeline preview by changing the setting for 'View transform' from 'sRGB (ACES)' to 'Off'. Then I noticed the colors look a bit deeper and richer on the vegas preview when in 32-bit mode - moreso than then actual source file itself. I wonder if this 32-bit mode is a better representation of what the colors should actually look like. FYI the source is a recorded stream using Bandicam's screen capture.

"what the colors should look like" is dependent on the situation. The typical situation is what jagabo wrote above - limited range YUV to full range RGB . Your original recording is YUV , not RGB. So when you "see" something on the display it has been converted to RGB. There are different ways of converting to RGB



In terms of levels preset vs. 32bit. Negligible quality loss between those 2 methods - nothing you would see with naked eye; nothing like compression losses

A slightly larger difference occurs when you're applying effects like blurs, curves, grading. Vegas works in linear when 32bit is used. It "degammas" or linearizes the input, and all the intermediate calculations are done in the 32bit precision , so there is no clipping .

You're actually losing more quality compared to the orginal source (in terms of pixel values) just by importing it, because of how vegas operates internally in RGB , and your original asset is 8bit YUV 4:2:0 . It upsamples the chroma using a non reversible algorithm. Nothing you would see with the naked eye, unless you look closely on test patterns.

A completely lossless in/out workflow for a YUV 4:2:0 asset is possible, but complicated and requires some usage of other programs. It requires converting everything to EXR float but chroma upsampling using nearest neighbor, 32bit in vegas, EXR export, and back to YUV 4:2:0 using nearest neighbor algorithm

FYI the source is a recorded stream using Bandicam's screen capture.

If you wanted easier completely lossless workflow, you would record in RGB , because that's what the screen display is . Not sure about newer Bandicam versions, but older versions were not able to. Internally it worked in YUV4:2:2, so even if you used a lossless RGB codec, the quality loss is measureable and if you pixel peeked

[abolibibelot]

Vegas works in RGB internally, therefore I always supply it with RGB to have control over how this is done (don't trust Vegas).
To avoid an intermediate render to RGB, I like to use Avisynth Virtual File System.
Upon mounting an AviSynth scipt file (avs) with it, it turns it temporarily into a "fake" avi that Vegas will import as if it was an uncompressed RGB avi. Very handy.

I've attempted this in the past, but had too many issues to continue.
It may work better with current versions of Vegas and other reputed editing softwares, but it's probably bound to fail with ever-so-slightly complex scripts, particularly if using filters with a “look ahead” kind of processing (although I'll admit that this is merely a half-educated guess).

[Sharc]

Vegas works in RGB internally, therefore I always supply it with RGB to have control over how this is done (don't trust Vegas).
To avoid an intermediate render to RGB, I like to use Avisynth Virtual File System.
Upon mounting an AviSynth scipt file (avs) with it, it turns it temporarily into a "fake" avi that Vegas will import as if it was an uncompressed RGB avi. Very handy.

I've attempted this in the past, but had too many issues to continue.
It may work better with current versions of Vegas and other reputed editing softwares, but it's probably bound to fail with ever-so-slightly complex scripts, particularly if using filters with a “look ahead” kind of processing (although I'll admit that this is merely a half-educated guess).

I have been using the Avisynth Virtual File System successfully with Shotcut.
Doing the filtering of the lossless YUV captures (*.avi) in Avisynth - ConvertToRGB32 - Mounting the script file with Avisynth Virtual File System - import (drag and drop) the virtual .avi onto the timeline of Shotcut for further editing and final encoding.
Didn't have any issues so far.

[poisondeathray]

I have been using the Avisynth Virtual File System successfully with Shotcut.
Doing the filtering of the lossless YUV captures (*.avi) in Avisynth - ConvertToRGB32 - Mounting the script file with Avisynth Virtual File System - import (drag and drop) the virtual .avi onto the timeline of Shotcut for further editing and final encoding.
Didn't have any issues so far.

What was the reason for ConvertToRGB32 ? Shotcut (unlike vegas) has a YUV capable timeline. Shotcut uses libavcodec (same libraries as ffmpeg) - pixel formats and fourcc like YV12, YV16 are accepted natively

Vegas works in RGB internally, therefore I always supply it with RGB to have control over how this is done (don't trust Vegas).
To avoid an intermediate render to RGB, I like to use Avisynth Virtual File System.
Upon mounting an AviSynth scipt file (avs) with it, it turns it temporarily into a "fake" avi that Vegas will import as if it was an uncompressed RGB avi. Very handy.

I've attempted this in the past, but had too many issues to continue.
It may work better with current versions of Vegas and other reputed editing softwares, but it's probably bound to fail with ever-so-slightly complex scripts, particularly if using filters with a “look ahead” kind of processing (although I'll admit that this is merely a half-educated guess).

It doesn't "fail" per se... but "complex" scripts are slow to begin with . avfs adds an additional layer of overhead and latency, so it is slower.

It will "fail" - for scripts that "fail" in avspmod or vdub2 (used as preview) to begin when using non linear access. e.g. srestore . That' s not avfs's "fault" . Frame accurate source filters and scripts otherwise work ok .

vapoursynth avfs has fourcc emulation now - IYUV for 8bit420, UYVY for 8bit422, v210 for 10bit422 - preferred fourcc's for vegas get "studio RGB" level treatment with these fourcc's , so no more clipping . Premiere and other window's based NLE's also treat those fourcc's as YUV instead of RGB - so they are truly lossless now. Resolve Studio imports v210 and it gets passthough treatment (also supports RGB30)

[Sharc]

I have been using the Avisynth Virtual File System successfully with Shotcut.
Doing the filtering of the lossless YUV captures (*.avi) in Avisynth - ConvertToRGB32 - Mounting the script file with Avisynth Virtual File System - import (drag and drop) the virtual .avi onto the timeline of Shotcut for further editing and final encoding.
Didn't have any issues so far.

What was the reason for ConvertToRGB32 ? Shotcut (unlike vegas) has a YUV capable timeline. Shotcut uses libavcodec (same libraries as ffmpeg) - pixel formats and fourcc like YV12, YV16 are accepted natively

Thanks for the hint. I once found that it produced less artifacts for interlaced sources and interlaced encoding. Maybe it was my imagination only, and I can really skip it for Shotcut and leave it as YV16. Source is huffyuv YUV 4:2:2 interlaced, converted to YV16 for subsequent filtering in Avisynth.

[Sharc]

I have been using the Avisynth Virtual File System successfully with Shotcut.
Doing the filtering of the lossless YUV captures (*.avi) in Avisynth - ConvertToRGB32 - Mounting the script file with Avisynth Virtual File System - import (drag and drop) the virtual .avi onto the timeline of Shotcut for further editing and final encoding.
Didn't have any issues so far.

What was the reason for ConvertToRGB32 ? Shotcut (unlike vegas) has a YUV capable timeline. Shotcut uses libavcodec (same libraries as ffmpeg) - pixel formats and fourcc like YV12, YV16 are accepted natively

Thanks for the hint. I once found that it produced less artifacts for interlaced sources and interlaced encoding. Maybe it was my imagination only, and I can really skip it for Shotcut and leave it as YV16. Source is huffyuv YUV 4:2:2 interlaced, converted to YV16 for subsequent filtering in Avisynth.

Now I found that Avisynth Virtual File System seems not to work with YV16. So I need to convert to either YV12 or RGB32, otherwise the virtual .avi is not getting created by the VFS. As subsequent encoding will be in YV12 it will be the obvious choice I think, rather than RGB32 (unless it is for VEGAS).

[poisondeathray]

Now I found that Avisynth Virtual File System seems not to work with YV16. So I need to convert to either YV12 or RGB32, otherwise the virtual .avi is not getting created by the VFS. As subsequent encoding will be in YV12 it will be the obvious choice I think, rather than RGB32 (unless it is for VEGAS).

What version ?

The newest versions work with YV16 or YUY2 avs scripts . You can extract avfs.exe from portable vapoursynth archive

Really old avfs version didn't support newer 2.6 planar pixel types like YV16

[Sharc]

Now I found that Avisynth Virtual File System seems not to work with YV16. So I need to convert to either YV12 or RGB32, otherwise the virtual .avi is not getting created by the VFS. As subsequent encoding will be in YV12 it will be the obvious choice I think, rather than RGB32 (unless it is for VEGAS).

What version ?

The newest versions work with YV16 or YUY2 avs scripts . You can extract avfs.exe from portable vapoursynth archive

Really old avfs version didn't support newer 2.6 planar pixel types like YV16

Yep! That was it. Problem solved. Thanks a lot

[mr-scarface]

However, since this is technically an effect being added, could this potentially mess with the picture quality in any way?

It's the standard way of converting YUV to RGB. Every other program, every media player, every graphics card, every TV does it that way. Limited range YUV to full range RGB.

So when I do edits in Avidemux, does it also basically do the same thing? I've noticed when I render in (FF)HuffYUV (in AviDemux), the rendered file doesn't have the color issue that Vegas does. Is it because it's automatically doing that conversion whereas with Vegas you need to apply it manually?

[mr-scarface]

Thanks. This worked.
However, since this is technically an effect being added, could this potentially mess with the picture quality in any way? It looks fine at first glance, but I wonder if this is the way forward for me. I specifically render in uncompressed to keep the quality pristine.

The other way I managed to do this is to change the pixel format in the project settings to '32-bit floating point (full range)' then render it - then the colors look fine on the rendered file. The only weird thing is, it messes up the colors on the timeline preview inside Vegas, like it is oversaturated.

I read on the vegas forums that you should be working in 32-bit floating point (full range) if keeping perfect quality is your aim. I probably need to research color theory and learn more about this subject, but I wonder what is the best way forward for now. Will look into the other suggestion on this thread too.

edit: I was actually able to fix the oversaturation issue on the timeline preview by changing the setting for 'View transform' from 'sRGB (ACES)' to 'Off'. Then I noticed the colors look a bit deeper and richer on the vegas preview when in 32-bit mode - moreso than then actual source file itself. I wonder if this 32-bit mode is a better representation of what the colors should actually look like. FYI the source is a recorded stream using Bandicam's screen capture.

"what the colors should look like" is dependent on the situation. The typical situation is what jagabo wrote above - limited range YUV to full range RGB . Your original recording is YUV , not RGB. So when you "see" something on the display it has been converted to RGB. There are different ways of converting to RGB



In terms of levels preset vs. 32bit. Negligible quality loss between those 2 methods - nothing you would see with naked eye; nothing like compression losses

A slightly larger difference occurs when you're applying effects like blurs, curves, grading. Vegas works in linear when 32bit is used. It "degammas" or linearizes the input, and all the intermediate calculations are done in the 32bit precision , so there is no clipping .

You're actually losing more quality compared to the orginal source (in terms of pixel values) just by importing it, because of how vegas operates internally in RGB , and your original asset is 8bit YUV 4:2:0 . It upsamples the chroma using a non reversible algorithm. Nothing you would see with the naked eye, unless you look closely on test patterns.

A completely lossless in/out workflow for a YUV 4:2:0 asset is possible, but complicated and requires some usage of other programs. It requires converting everything to EXR float but chroma upsampling using nearest neighbor, 32bit in vegas, EXR export, and back to YUV 4:2:0 using nearest neighbor algorithm

The completely lossless workflow you mentioned seems like too much of a headache for now. In that case, would you recommend just applying the filter preset instead of working in 32-bit mode? My current PC struggles in 32-bit mode but I'll be upgrading soon so that won't be an issue.

By the way, in addition to 'Uncompressed', I noticed Vegas has Sony YUV codec, 10-bit Sony YUV codec, and also Lagarith Lossless Codec. Are those potentially better than Uncompressed while being lossless? My Uncompressed renders cause some laggy playback in media players. I noticed (FF)HuffYUV renders in Avidemux don't have that laggy playback issue though.

Also, how does Premiere pro generally fare in this aspect compared to Vegas?

If you wanted easier completely lossless workflow, you would record in RGB , because that's what the screen display is . Not sure about newer Bandicam versions, but older versions were not able to. Internally it worked in YUV4:2:2, so even if you used a lossless RGB codec, the quality loss is measureable and if you pixel peeked

I do know Bandicam has an RGB mode, but I think the file sizes will be insane in that format. It's something I'll have to look into.

[poisondeathray]

would you recommend just applying the filter preset instead of working in 32-bit mode? My current PC struggles in 32-bit mode but I'll be upgrading soon so that won't be an issue.

In general, I would just use the preset. If you're using a workflow that benefits from 32bit mode (some filters, compositing, grading) , I would probably use 32bit mode. Why don't you do some short tests and see if you can tell the difference? On just simple in/out it's pretty much negligible in terms of differences to naked eye (but measureable)

By the way, in addition to 'Uncompressed', I noticed Vegas has Sony YUV codec, 10-bit Sony YUV codec, and also Lagarith Lossless Codec. Are those potentially better than Uncompressed while being lossless? My Uncompressed renders cause some laggy playback in media players. I noticed (FF)HuffYUV renders in Avidemux don't have that laggy playback issue though.

Those options don't really help you, because they are for exports. The underlying issue occurs upon import and decoding for the timeline with RGB conversion (ie. problem occurs before you export)

Also, how does Premiere pro generally fare in this aspect compared to Vegas?

PP has native YUV timeline, so no problem

If you wanted easier completely lossless workflow, you would record in RGB , because that's what the screen display is . Not sure about newer Bandicam versions, but older versions were not able to. Internally it worked in YUV4:2:2, so even if you used a lossless RGB codec, the quality loss is measureable and if you pixel peeked

I do know Bandicam has an RGB mode, but I think the file sizes will be insane in that format. It's something I'll have to look into.

Probably not worth it, unless it recorded true lossless RGB in the first place.

[jagabo]

My Uncompressed renders cause some laggy playback in media players.

This is usually because the video can't be read from the hard drive fast enough. Don't be misled by "SATA 6Gb/s" claims from the manufacturers. That's the speed of the SATA interface, not how fast data can actually be read from the platters. The latter is far lower and varies depending where on the drive the data is written. The inner cylinders are far slower than the outer cylinders. Sector remapping can slow things down. And large files are usually fragmented, slowing the transfer rate.

I noticed (FF)HuffYUV renders in Avidemux don't have that laggy playback issue though.

Compressing the data by about half reduces the bandwidth requirement enough for it to be comfortably read from the hard drive.