Superblacks and superwhites question

[SameSelf]

So for a long time, I have been meaning to learn a little more about all this colorspace stuff. Especially in regards to broadcast legal [16,235] versus ComputerRGB [0,255]. So I made the following little chip card in AE which I assume is Computer RGB colorspace. The top row fades from grayscale RGB (0,0,0) to (28,28,28) in (4,4,4) increments. The black band in the middle is simply (0,0,0). The white band below it is (255,255,255). And the bottom chips fade from RGB (227,227,227) to (255,255,255) again in (4,4,4) increments. This is the screen shot from within AE.

AE Screen Shot


Then I dynamically link my composition in PP. Here is the YC Waveform. And everything looks good.

PP YC Waveform


Next, I frameserved out of PP using DMFS and set the output to YUY2. Here is the Avisynth script that I load into x264:

AVISource("E:\signpost.avi", audio=false).AssumeFPS(30000,1001)
ConvertToYV12(interlaced=false, matrix="PC.709")

After encoding using x264, I uploaded to Youtube because I wanted to see if Youtube expands out the colorspace thereby clipping the blacks and whites. Here is a screenshot of the YT upload. IMO, the white chips look fine. But the black chips looked start to lose resolution at around RGB (8,8,8) and even RGB (12,12,12).

Youtube Screenshot


So what do you guys think? Is there a problem anywhere in this chain? Thanks.

[SameSelf]

OK, I took a look at how the encoded video looks using MPC. Here is the screenshot. So on further inspection, it appears that YT is faithfully reproducing the upload. The encode looks to be lacking resolution. Thoughts?

[jagabo]

Your PP YC Waveform shows limited range YUV (IRE0 = Y16, IRE100 = Y235). It's perfectly normal for full range RGB (0-255) to be converted to limited range YUV (Y=16-235, U,V=16-240). Then when converting back to RGB for display it is expanded back to full range RGB again. DVDs, Blu-ray discs, broadcast TV, Youtube videos, etc. use limited range YUV.

Your ConvertToYV12(interlaced=false, matrix="PC.709") should have been "rec709" (HD) or "rec601" (SD).

[SameSelf]

Brilliant jagabo! I should have known I was getting it wrong somewhere. I did notice that the resolution in PP seemed less than what I could see in AE. Now I know why. PP squeezes full range RGB (0-255) to YUV (Y=16-235, U,V=16-240). Thanks very much for that. Also, I had no idea U,V went slightly higher than Y.

So, I fixed my Avisynth script as you suggested:

AVISource("E:\signpost.avi", audio=false).AssumeFPS(30000,1001)
ConvertToYV12(interlaced=false, matrix="rec709")

After encoding again, sadly, the results from the MPC screenshot don't look any different. I am a little confused now.

[jagabo]

When you frame serve from PP does it output RGB? If it's putting out YUV then ConvertToYV12(interlaced=false, matrix="rec709") won't change levels. If that's the case then PP is converting to full range YUV. Use Info() in your script to see what it's receiving.

AVISource("E:\signpost.avi", audio=false).AssumeFPS(30000,1001)
Info()

Try using ImageSource() to create a video from your image then convert to YV12.

ImageSource("image.png", start=0, end=300, fps=29.97)
ConvertToYV12(interlaced=false, matrix="rec709")

[poisondeathray]

It looks like the test image wasn't done correctly, or maybe you posted the wrong one in the 1st post - double check the values - they appear to be off, and scale at different rates

[SameSelf]

I assume you are referring to the AE screenshot. It wouldn't surprise me if I screwed something up. Here is how I made my chip chart:

In a new comp, I simply added a new solid for each square and set the solid's color to the decimal equivalent RGB value. For example, RGB (8,8,8) was set to RGB (0.0313, 0.0313, 0.0313) as I assumed the scale extended from 0 to 255. Similarly, the RGB (227, 227, 227) was set to RGB (0.8902, 0.8902, 0.8902). Maybe that is where I screwed up. But I assumed that AE was full range RGB.

I am certain my monitor is not calibrated properly, so that could also be the difference you are seeing.

But, regardless of whether I set up the chart properly in AE, I can see the gradations in the blacks all the way across in AE. But they look crushed in the final encode. Would you agree with this assessment? Which tells me that something is wrong in my workflow. I am trying to figure out what it is.

[poisondeathray]

Potential problems are your method of screenshot taking. Your encode is in YUV, but MPC is displaying in RGB. How it converts to RGB for display can be affected by many things. There might be causing other conversions depending on how you have it set up. Renderer settings for example can change the displayed levels. Some use the GPU settings (e.g. overlay) . Or youtube with GPU on vs. off. Then the GPU settings can affect what you "see." Some are completely different eg. VMR9 vs haali or EVR or overlay usually very different. Individual player settings can adjust the image as well.

If you're used to 8bit values, just make an 8bit test pattern. 0-255 in RGB. "limited range" refers to what values in YUV are "mappped" to RGB for display. As mentioned above, limited range conversion is standard, and takes Y' 16-235 , CbCr 16-240 and "maps" to RGB 0,0,0-255,255,255 in 8bit values. This is the same as the "Rec" matrices in avisynth.

When working in RGB, stay in RGB the whole way through. Don't make unnecessary colorspace conversions, and you will get better control if you decide when and how to convert it. So I would use RGB in debugmode with what you are doing. You only use YUY2 , when you have a native YUV timeline (e.g. a premiere only project with YUV assets and YUV filters). AE is working in RGB, dynamic link will be in RGB so it makes sense to frameserve RGB. Control the conversion at the very end with avisynth. You would use ConvertToYV12(matrix="rec709") from RGB frameserver. Or use a dithered conversion if you want higher quality (It won't make much a difference on youtube). I'm one of the ones that can't get dmfs to work on anything newer than CS5, so I can't check your workflow.

If you upload a tiny sample of your final YUV encode, along with your source image - someone can test if it's correct or not, or if it's a display configuration issue on your end

-------------------

Youtube actually reads flags now. If you encode full range YUV, and flag it full range, it will acknowledge it and clamp the re-encode to legal range.

But if you use your own server and hosting, you can actually display full range YUV with flags (essentially the equivalent of avisynth "PC" matrix is applied, 0-255 in YCbCr is "mapped" to 0-255 in RGB)

[jagabo]

Be careful with using a media player to test your levels. The player can be set to change the brightness and contrast. And the graphics card's video processing amp can change brightness and contrast too. Make sure that neither is molesting the signal. And you need to make sure you monitor is set up properly to display levels correctly. Make sure that all auto contrast/brightness/color/sharpness/etc. features are disabled everywhere.

See the following post for a DV video with a levels chart.

https://forum.videohelp.com/threads/326496-file-in-Virtualdub-has-strange-colors-when-o...=1#post2022085

And this post for an MPG version of that chart:

https://forum.videohelp.com/threads/353338-Avisynth-ColorYUV-Gamma-is-doubling-the-enco...=1#post2220162

Those video include super blacks and super brights for test purposes. So the numbers in the chart represent Y values, not the expected RGB values when played. The first post explains what you should see. The DV video has a small error, the 0 bar is actually 1.

If you want to see the Y values without the contrast stretch to full range RGB use ConvertToRGB(matrix="pc.601"). To see a waveform monitor view use TurnRight().Histogram().TurnLeft().

[SameSelf]

Thanks so much pdr and jagabo. I am slowly digesting all this information. I will post back later this evening with the actual mp4 for people to take a look at themselves.

[jagabo]

Here's an updated MPG file with a levels chart. It shows the Y values and the expected RGB values after a standard rec.601 conversion. The values you get on-screen may vary by one unit because of rounding issues.

Here's the base image (not what you're supposed to see when playing the video):



And the script that fed the image to HcEnc:

ImageSource("grayblocks5.png", start=0, end=240)
AssumeFPS(24000,1001)
ConvertToYV12(interlaced=false, matrix="pc.601")

And what it should look like when playing the video:

[SameSelf]

OK, after doing some more research, I determined that one of the important things to do is set the color management in AE properly. I have no idea if this makes any difference, but I had the Working Space set to "HDTV (Rec. 709)" with 32 bpc (float). None of the options (Linearize Working Space, Blend Colors Using 1.0 Gamma, etc.) were checked.

And, as pdr said, players, like MPC, are not color calibrated or expect YUV but rather work in RGB. So I did the most obvious thing and re-imported my encoded mp4 back into AE. Lo and behold, the gradation is present even though I couldn't see it in MPC or YT. So I am going to chalk this up to improper calibration. Plus my monitor has never been calibrated. So it sounds like I am sort of wasting my time until I get a calibrator anyway.

Once again, thank you, thank you so much for all the insight. I will be spending a lot of time over the next few days or weeks pondering everything that has been said here. FWIW, here is my encode:

I should also add, when I played the attached video on my TV, I could see all the gradations.

[vhelp]

What about adjusting an LCD laptop monitor ?

The following was snipped from another discussion that I probably should have thrown here.

vhelp wrote--The one thing that bothered me most is the video display on my dell laptop. Its horrible. All the colors are washed out when I view a video, whether in virtualdub, ffplay, or MPC-HC. I can't seem to figure out how to match it to my desktop pc, work or home. Maybe someone know something about this and can help? Anyway. I gotta get out of here and get some dinner. Its deli night.

The one thing that bothered me most is the video display on my dell laptop. Its horrible. All the colors are washed out when I view a video, whether in virtualdub, ffplay, or MPC-HC.

In VirtualDub go to Options -> Preferences -> Display and disable everything there. Press Save. Exit and restart VirtualDub. Open a video. Do you still have washed out colors?

Or in VLC set Tools -> Preferences -> Video -> Output -> Windows GDI Video Output. Exit and restart VLC. Colors still washed out?

If colors look ok with those settings you just need to adjust your graphics card's video proc amp settings.

El Heggunte wrote--How old is your Dell laptop? Are you sure that its monitor is not a 6-bit device?

Looks like you are correct. I need to adjust the pro amp on the monitor. But its difficult when you are trying to adjust it for vhs granularity since that is the bulk of my video sources. I'm nearly certain that the laptop is from 2010, but the link (below) says the cpu is from 2012. So I'll have to find the receipt to verify. I scanned it into my computer, just have to find out which one since the the filename was some odd string of chars Anyway. The monitor on the laptop, for video work does not match anywhere near the desktop monitor. I would say the desktop monitor is more contrast and black level is more darker. Its kind of hard to describe. But about the best I can describe: if you open two video windows, and play them at the same time, one will be lighter color or washed out look. Darks will be muddy like. Something like that. It may be related the Gamma and Color Heat levels for this kinds of monitors on laptops. I checked my settings, and the gamma is at 1.0 value. Its not that critical. I just copy the video to a mem stick and walk it over to my desktop pc to review the video.

http://ark.intel.com/products/53442/Intel-Core-i3-2370M-Processor-3M-Cache-2_40-GHz?q=i3-2370M

Dell inspiron laptop model # 5520

Looks like you are correct. I need to adjust the pro amp on the monitor. But its difficult when you are trying to adjust it for vhs granularity since that is the bulk of my video sources.

You have this backwards. You adjust the monitor to standard video levels. Then you adjust your VHS caps to those standard levels.

The monitor on the laptop, for video work does not match anywhere near the desktop monitor. I would say the desktop monitor is more contrast and black level is more darker.

Start by adjusting the black/white levels. This post has a levels test pattern you can use for that:

https://forum.videohelp.com/threads/374734-Color-Levels-Question?p=2414529&viewfull=1#post2414529

Download the MPG file and play it. Make sure the player isn't messing up the video -- disable any proc amp or other filters of the player. Then adjust the graphics card's video proc amp until blacks are as black as your display can get, and whites are are as bright as you want the monitor to display (generally you do this by adjusting the backlight). Then adjust the proc amp controls so the video chart looks like the second image in the above post.

Note that cheap LCD monitors can't display very dark blacks. You may have to settle for a dark grey.

And you usually want to turn off all "auto" adjustments of the graphics card and player. That is, turn off anything like auto contrast, auto color, auto skin tone, etc. Also disable any noise reduction, deblocking, etc. All those things do is mess up your video. You want to see what your video really looks like, not some caricature created by the player or graphics card.

Note that cheap LCD monitors can't display very dark blacks. You may have to settle for a dark grey.

This laptop was not intended to be a video workstation at first, but slowly became that over time, though I still use the main desktop xp system for that. But you may be right, that I may have to settle for the dark grey on this cheap monitor. But if I can try a few adjustments I will.

I read about those hardware color adjustment devices that people are using for LCD's, I will do some research and see if there is anything I can do to adjust mine, or else I may consider a new LCD unless the issue is with my laptops hardware pro amps and its inherent limitations. Thanks for chiming in on this issue in the other thread, much appreciated.

[jagabo]

And, as pdr said, players, like MPC, are not color calibrated or expect YUV but rather work in RGB.

Players are usually set up to send YUV from the video directly to the graphics card (unless you enable the players own proc amp) so it's the graphics card's video proc amp that does the conversion to RGB for display on the monitor.

[SameSelf]

Wow, I forgot all about this thread. So much has changed in a year. I have upgraded my video editing workstation to a color managed workflow. Therefore, I am pleased to say these seem like fairly petty problems now .

As far as doing any sort of color critical work on a laptop, how is that even possible? It is imperative that you have HW that can isolate the color management of your video from the OS and gpu. Also, you need a monitor that can be calibrated with custom LUTs and operate natively in YUV space, preferably 10-bit. So a dual monitor setup becomes unavoidable.