Player Encoding

[chris319]

Please, no more discussion of my choice of test pattern colors. Thank you.

The table below shows the results of my testing of the color reproduction of five video players on videos encoded to bt.601 and bt.709.

	                      bt,601	bt.709
Firefox (YouTube)	OK	No
Chrome (YouTube)	OK	No
Opera (YouTube)	OK	No
Edge (YouTube)	        No	OK
VLC (Local)	        No	OK

The operating system is Windows 10. Below are the command lines used to encode the test pattern bmp file to mp4 using ffmpeg.

ffmpeg -y -loop 1 -i test_pattern601.bmp -t 10 -vf scale=out_color_matrix=bt601 test_pattern601.mp4

ffmpeg -y -loop 1 -i test_pattern709.bmp -t 10 -vf scale=out_color_matrix=bt709 test_pattern709.mp4

"OK" means the colors are reproduced with digital values +/- 2 of the original values. "No" means the colors are reproduced with digital values exceeding +/- 2 of the original values.

I am told by one of the Firefox programmers that Firefox uses GPU/hardware color decoding.

YouTube links:

https://www.youtube.com/watch?v=9DycR9-zeXw

https://www.youtube.com/watch?v=oA3-TFJv1xI

[jagabo]

Yes, different browsers/players/drivers interpret colors differently, especially when not flagged, but even if flagged.

[chris319]

In your test pattern mp4 file, MediaInfo has as the last line "Matrix coefficients BT.709". In my files, encoded with ffmpeg, MediaInfo shows no such information.

What are you using to encode your mp4 videos?

[jagabo]

I used the x264 CLI encoder. The commands are:

--colormatrix=bt470bg --colormatrix=bt709 --colormatrix=bt709

Only the last one is really necessary.

You can pass the flags along to the x264 encoder in ffmpeg by using the x264opts. I think this is right but I don't use ffmpeg much:

-x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

And the values for rec.601

-x264opts colorprim=bt470bg:transfer=bt470bg:colormatrix=bt470bg

Youtube simply takes the video you uploaded and reencodes it, marking it as rec.709. So if you upload rec.601 it will be incorrectly flagged as rec.709. But many players don't pay any attention to the flags so it may or may not be displayed correctly.

[chris319]

x264 returns:

x264 [error]: not compiled with MP4 output support

Youtube simply takes the video you uploaded and reencodes it, marking it as rec.709.

If that were the case, I should be able to upload a 709 file to YouTube and have it play with correct colors, which it does not. See table above.

[jagabo]

x264 returns:

x264 [error]: not compiled with MP4 output support

Output to another container like mkv. Or use a build of x264 with mp4 output support.

Youtube simply takes the video you uploaded and reencodes it, marking it as rec.709.

If that were the case, I should be able to upload a 709 file to YouTube and have it play with correct colors, which it does not. See table above.

As I told you, many players ignore the colormatrix flag. Unfortunately, there's no way of guaranteeing proper colors across all platforms. The best you can do is use rec.601 for SD, rec.709 for HD, and flag which matrix was used.

[poisondeathray]

Likely you're using the HTML5 player . Known issue with youtube , firefox , chrome etc.. all use Rec601 despite HD/709 prevalence these days . You can force youtube to use a flash player and to receive an AVC stream in your browser configuration

The problem is there are many variables that may cause the end user to "see" something different

There are different client configurations for youtube , any given browser and os , hardware, drivers etc... Even the same youtube address might be sending a different video depending on requested resolution, platform. You might get an VP8/9 vs. someone else getting AVC stream for example. How that is handled locally in HW depends if your actual HW has acceleration for that format (AVC is more common than VPx, one reason many user force AVC streams - HW acceleration) - but that also governs how YUV is converted to RGB

Despite all the "pros" of HTML5, this is one area that flash is better than current HTML5 implementation (and flash has a lot of "cons") . Flash is able to abide by colormatrix flags . So on websites that you control yourself, you have higher likelihood of proper colors

There is no way of guaranteeing anything, even on the same platform, browser or even browser version. With software players it's even worse - you have dozens of renderer choices

[chris319]

Can either of you back the assertion that YouTube resamples to a specific color space? It seems to be common knowledge that YouTube resamples all videos but how do we know it alters the color space?

[jagabo]

It doesn't alter the colorspace if you upload YUV (aside from subsampling the chroma). It simply flags it as rec.709 when it reencodes. Just download one of your own videos and examine it. The video will still have rec.601 colors but MediaInfo will report rec.709.

[chris319]

It doesn't alter the colorspace if you upload YUV (aside from subsampling the chroma). It simply flags it as rec.709 when it reencodes.

Then it doesn't really "re-encode" as you originally said, it merely re-flags and does subsampling. Yes, it actually does re-flag as 709, even a 601 video or one with no coefficients field at all.

The coefficients field is thus useless because a) some encoders don't even bother with it, and b) YouTube will re-flag it as 709 which is incorrect if it's a 601 video. Firefox, Opera and Chrome will play a 601 video correctly despite it having been flagged by YouTube as 709.

This behavior can be changed with DXVA in Windows and VDPAU in Linux. The players/browsers should standardize on 709 because a) it's how videos come flagged out of YouTube, and b) sRGB is supposedly the web standard and it uses 709 coefficients. So if a user uploads a 601 video he'll just have to live with the color errors.

[jagabo]

Then it doesn't really "re-encode"

"Re-encode" as in "re-compress", not "convert the colorspace".

[chris319]

If all it's doing is re-sampling to 4-2-0, one wonders if it resamples if a video comes in already at 4-2-0.

[jagabo]

Why don't you ask them? Of course, all the resized videos are resampled. And you could find out for yourself with test patterns.

[poisondeathray]

If all it's doing is re-sampling to 4-2-0, one wonders if it resamples if a video comes in already at 4-2-0.

What do you mean by "resampling", specifically ?


It re-encodes, yes . You can prove this by downloading the video(s) . You can examine the deterioration and artifacts of n+1 generation of lossy re-encoding. You can choose specific versions with youtube-dl .

4:2:0 input is passed through as 4:2:0 prior to re-encoding . There is no evidence of additional chroma manipulation (e.g. perhaps an intermediate YUV444 or RGB stage) , because you can see that additional deterioration manifest on test patterns

[chris319]

What do you mean by "resampling", specifically ?

Converting from 4-4-4 or 4-2-2 to 4-2-0. Apparently the color space is not altered.

you can see that additional deterioration manifest on test patterns

What kind of test pattern will show this off?

[poisondeathray]

What do you mean by "resampling", specifically ?

Converting from 4-4-4 or 4-2-2 to 4-2-0. Apparently the color space is not altered.

Yes, YUV444, YUV422 and YUV420 are all YUV (or technically "YCbCr") . Technically it's not "color space" either, it's "color model" but people use them interchangeably

4:4:4 , 4:2:2 and 4:2:0 only describe chroma subsampling.
https://en.wikipedia.org/wiki/Chroma_subsampling

A color space change would be something like YUV => RGB . Youtube always serves YUV420 . So if you upload something other than that, it gets converted to 420 before the re-encoding

you can see that additional deterioration manifest on test patterns

What kind of test pattern will show this off?

Test patterns with sharp distinct colored borders. Chroma subsampling is just resizing of the chroma planes Cb,Cr . When you down/up scale, the lines get blurrier and blurrier, the extent depending on what algorithm is used. The more times, the more blurry. There are many examples and tests in various threads you can see (or just do it yourself)

When it gets converted back to RGB for display, you are essentially upsampling the chroma . RGB is never subsampled

[chris319]

Test patterns with sharp distinct colored borders.

This suggests a redesign of my test pattern. Move the 235 white chip all the way to the right and have the yellow chip be the leftmost chip.

https://www.youtube.com/watch?v=oA3-TFJv1xI

[jagabo]

Try a pattern of alternating single pixel thick light and dark lines in the chroma, flat medium intensity luma. If the chroma comes back very messed (bad loss of saturation, moire patterns) up it probably went through some spacial resampling.

[chris319]

In drawing the raw bmp image, I tried alternating green and magenta lines. On my monitor they blurred together into gray, as does your video file when played back with Windows Media Player. Any ideas?

Here it is with alternating green and black lines:

https://www.youtube.com/watch?v=J8fF9Qg8tBs&feature=youtu.be

[jagabo]

Remember in YV12 the chroma planes are half the width and half the height of the luma plane. So the color stripes need to be 2 pixels wide in 4:4:4. In RGB you can paint 2 pixels thick lines so that the chroma of YV12 shows up as alternating lines. But the RGB 4:4:4 to YUV 4:2:0 subsampling is hard to control. You may not get nice sharp lines. I created my sample in AviSynth where I could create perfect lines in the U and V planes.

Here are the Y, U and V channels from your video (downloaded the MP4 from youtube):


[Attachment 40918 - Click to enlarge]

You can see that the striped color patch has no stripes in the chroma.

In my video the stripes are obvious in the chroma (and the luma is completely flat):


[Attachment 40919 - Click to enlarge]

Be sure to view it pixel-for-pixel to avoid avoid moire artifacts and blurring.

[chris319]

Here is how you originally told me to draw it, and that's how I drew it.

alternating single pixel thick light and dark lines in the chroma, flat medium intensity luma

Is this how you want me to draw it now?

the color stripes need to be 2 pixels wide in 4:4:4. In RGB you can paint 2 pixels thick lines so that the chroma of YV12 shows up as alternating lines.

[jagabo]

That's a start but you have to control how the RGB 4:4:4 is resampled to YUV 4:2:0. Different programs will use different scaling algorithms. Try forcing a point resize if you can. And it will help to use RGB colors that map to the same Y value. If you're going to use (R,G,B) 254,1,254 for the magenta then use 1,102,1 for the green. That should work for a rec.709 conversion.

Another thing to consider: some scaling problems may not show up in a small patch. I recommend you use a patch (with vertical lines) that covers the width of the frame and another (with horizontal lines) that covers the height of the frame.

[chris319]

That's a start but you have to control how the RGB 4:4:4 is resampled to YUV 4:2:0.

All I'm doing is creating a .bmp image that will be useful to any YouTube or other user. This talk about controlling the resampling is moot.

Where do you get the value of 102 for green?

Why not just look at the boundary between two saturated color patches? You're going to have to magnify the image to see any error anyway.

[jagabo]

Here is how you originally told me to draw it, and that's how I drew it.

alternating single pixel thick light and dark lines in the chroma, flat medium intensity luma

No, you drew single pixel thick lines in the luma, not the chroma.

That's a start but you have to control how the RGB 4:4:4 is resampled to YUV 4:2:0.

All I'm doing is creating a .bmp image that will be useful to any YouTube or other user.

And generally failing.

This talk about controlling the resampling is moot.

Not, it's not. The resampling method determines how sharp the boundary between colors will be, how much the saturation changes at those boundaries, and whether there will be any overshoots. See if you can convert to YV12 and get the colors of the alternating green/magenta bars to match the colors of the green/magenta patches at the lower left:


[Attachment 40923 - Click to enlarge]

Compare what you get to the attached rec.709 YV12 video. Exactly what you see on-screen will vary depend on how the chroma is upscaled from YUV 4:2:0 to RGB 4:4:4 during playback. On my computer the alternating green/magenta lines of the video have essentially the same RGB values as the green/magenta patches.

Where do you get the value of 102 for green?

The green I gave has the same luminance as the magenta after a rec.709 conversion. Ie, if you look at the luma plane the green/magenta areas will all have the same shade of grey.

Why not just look at the boundary between two saturated color patches? You're going to have to magnify the image to see any error anyway.

Because it's much harder to see at a single boundary (or just a few). Try resizing the attached video a little bit during playback -- eg enlarge it from 1280x720 to ~1288x725. Can you see much of a difference between the color patches at the top? How much do you see in the alternating green/magenta bars at the bottom and right.

[chris319]

And generally failing.

Don't be snide.