YCbCr 16-235 to RGB 0-255 and PAL/NTSC differences (7.5 IRE)

[intracube]

Partly following on from this thread:
https://forum.videohelp.com/threads/333622-Canon-T3i

Jagabo posted some test videos with greyscale patterns for monitor calibration. I don't have the links handy, but they are useful in determining what is going on. I'm sure he'll post the links
EDIT: here it is , and the link in that post
https://forum.videohelp.com/threads/313178-A-problem-with-dynamic-ranges?p=1965119&view...=1#post1965119

Thanks poisondeathray/jagabo, that's really useful. It looks exactly as it should with every media player I've tried:


I also found a THX calibration test on the 'DVD Demystified' disc; and I think I'm getting correct results with that too:


What confused me was one of the other calibration charts on the DVD Demystified disc:

*ignore the slightly ribbed pattern on the background - it appeared when I saved the screen-shot as a jpeg.
I would have expected to see the background (Y=16) to appear black which would then make the leftmost vertical bar invisible.

Could this be an authoring problem with the disc rather than my computer?

Lastly, some clarification on 'PAL' and 'NTSC' differences. I thought that while both share the same white level (Y=235), PAL uses Y=0 for black. The Y=16 black level for NTSC relates to the 7.5 IRE 'setup'?
If the above is true, what's used to distinguish between the two formats. Wouldn't converting 16-235 YCbCr to 0-255 RGB result in crushed blacks for PAL content??

EDIT: looks like 'PAL' and 'NTSC' DV/DVD both use the 16-235 range:
http://forums.creativecow.net/archivethread/8/459306
http://www.glennchan.info/articles/technical/setup/75IREsetup.html
http://www.tri-sysdesigns.com/Articles/BlackisBlack.html

[jagabo]

What confused me was one of the other calibration charts on the DVD Demystified disc:

[Attachment 6317 - Click to enlarge]
*ignore the slightly ribbed pattern on the background - it appeared when I saved the screen-shot as a jpeg.
I would have expected to see the background (Y=16) to appear black which would then make the leftmost vertical bar invisible.

Could this be an authoring problem with the disc rather than my computer?

It looks like an authoring problem. Or maybe they mean for the chart to demonstrate something else. Another possibility is that you used a different player. Or had two players running at the same time. In that last case, the two players may look different even playing the same video because one is using the graphics card's video processing ability (video overlay, basically a hole in the Desktop through which you see YUV video) and the other is using the Desktop (software is converting to YUV to RGB and displaying RGB on the Desktop).

Lastly, some clarification on 'PAL' and 'NTSC' differences. I thought that while both share the same white level (Y=235), PAL uses Y=0 for black. The Y=16 black level for NTSC relates to the 7.5 IRE 'setup'?
If the above is true, what's used to distinguish between the two formats. Wouldn't converting 16-235 YCbCr to 0-255 RGB result in crushed blacks for PAL content??

EDIT: looks like 'PAL' and 'NTSC' DV/DVD both use the 16-235 range:
http://forums.creativecow.net/archivethread/8/459306
http://www.glennchan.info/articles/technical/setup/75IREsetup.html
http://www.tri-sysdesigns.com/Articles/BlackisBlack.html

Yes, in the digital domain both PAL and NTSC use Y=16 to Y=235. IRE setup is only an analog issue.

And beware of Quicktime which screws up all of this.

[intracube]

It looks like an authoring problem. Or maybe they mean for the chart to demonstrate something else.

As the chart is alongside other calibration images (colourbars, overscan, etc), I think it's intended to be used to set up a TV screen - rather than to conceptually show the 16-235 levels.

Another possibility is that you used a different player. Or had two players running at the same time.

I used one player at a time (xine) which showed up the error with the DVD Demystified chart, then immediately opened the grayblocks2.m2v video - which looked correct.

I also tried other players; ffplay, mplayer - grayblocks2.m2v looked correct with both, while the DVD Demystified chart looked wrong. Although there are other test charts on the same disc (THX calibration charts) that look correct.

In that last case, the two players may look different even playing the same video because one is using the graphics card's video processing ability (video overlay, basically a hole in the Desktop through which you see YUV video) and the other is using the Desktop (software is converting to YUV to RGB and displaying RGB on the Desktop).

Sounds like you're describing the XVideo extension:
http://en.wikipedia.org/wiki/X_video_extension

With mplayer it's easy to select a different video output driver, for example:
mplayer -vo gl grayblocks2.m2v
mplayer's 'gl' driver has some useful options, including the ability to force the colourspace and levels conversion:

colorspace=<n>
    0: MPlayer's default YUV to RGB conversion
    1: YUV to RGB according to BT.601
    2: YUV to RGB according to BT.709
    3: YUV to RGB according to SMPT-240M
    4: YUV to RGB according to EBU
    5: XYZ to RGB
  levelconv=<n>
    0: YUV to RGB converting TV to PC levels
    1: YUV to RGB converting PC to TV levels
    2: YUV to RGB without converting levels

So running the following:
mplayer -vo gl:levelconv=2 grayblocks2.m2v
deliberately disables the levels conversion, and gives a similar output to the second image in this post:
https://forum.videohelp.com/threads/313178-A-problem-with-dynamic-ranges?p=1965119&view...=1#post1965119
setting levelconv=0 produces the correct result, as expected.

But I normally use the 'xv' driver, and I'm now confident it's getting the levels right automatically. However unlike the 'gl' driver, I couldn't find a way to override the levels for diagnostic purposes. So they both have their uses.

EDIT: looks like 'PAL' and 'NTSC' DV/DVD both use the 16-235 range:
http://forums.creativecow.net/archivethread/8/459306
http://www.glennchan.info/articles/technical/setup/75IREsetup.html
http://www.tri-sysdesigns.com/Articles/BlackisBlack.html

Yes, in the digital domain both PAL and NTSC use Y=16 to Y=235. IRE setup is only an analog issue.

And beware of Quicktime which screws up all of this.

Yep, I remember reading about that in this thread:
https://forum.videohelp.com/threads/330410-Youtube-changed-my-colors!-What-to-do

Separate to the issue of levels, on page 2 of the thead, poisondeathray comments that YouTube 'will always use BT.709' colour.
If my starting point is SD video (from DV/DVD/DVB) which has legal Y values of 16-235, should I keep the levels as they are, and convert from BT.601 to BT.709 prior to uploading to YouTube?

The differences between 601/709 might be subtle, but it'd be good to get that right as well.

Although, it doesn't look like ffmpeg allows these to be changed currently:
http://forum.doom9.org/archive/index.php/t-158840.html

[poisondeathray]

Those switches werey broken in ffmpeg when I last tested (few months ago), but you can use colormatrix filter in avisynth to convert between Y'CbCr 709 and 601 . Note those ffmpeg switches refer to Y'CbCr <=> RGB conversions (that's the only time the "matrix" is ever used). That's why the colormatrix filter is unique , it allows you to swap in Y'CbCr without having to take a trip into RGB land and incurring those losses

EDIT: see next post

It might have changed, I did those tests a while back, and it was independent of resolution (both HD & SD get the BT 709 treatment)

YT ingnores flags and switches on the encoder or muxer (embedded in stream) also - you can flag a stream fullrange , or limited range, or BT601 or 709 (or others even), but it ignores everything. It re-encodes everything, and converts to RGB for display using BT709 only.


However, flash, if you use your own implementation or something that doesn't re-encode - allows for full range flag (so full range Y'CbCr material is decoded to RGB as full range - Y'CbCr[0,255] => RGB [0,255] . This can result in less banding even in 8-bit format . You can have a look at these 2 videos. The 1st is "normal" YUV data Y'[16,235] displayed as normal range RGB. The 2nd is full range YUV data Y' [0,255] displayed as full range RGB [0,255]

(you might not be able to see the "banding" depending on the quality or bit depth of your display panel)


ConvertToYV12() No flags
http://www.blip.tv/file/4409086




ConvertToYV12(matrix="PC.709") Fullrange flag
http://www.blip.tv/file/4409087

[poisondeathray]

Sorry I made a mistake - If you use avisynth's colormatrix filter to convert to BT709 - colors will look "right" when youtube converts to RGB for display for SD source - I just confirmed this

colormatrix(mode="rec.601->rec.709" , clamp=0)

But I'm not sure how to do that in linux? Maybe it will work with wine ?



Avidemux has a coloryuv filter, that has a BT709 option but it doesn't work properly (when examining the exported file). Also the preview in avidemux is very very off. I remember reporting that in the avidemux forum over a year ago, but it still hasn't been fixed

[_Al_]

Be carefull while exporting from Sony Vegas through DebugMode frameserver, it changes colorspace.

It needs to be fixed , for RGB input:
ConvertToYV12(matrix="PC.709")
or this line for YUV input:
ColorYUV(levels="TV->PC")



otherwise it is going to be clipped entering encoder to 16-135:

[intracube]

The blip.tv examples are interesting. Of course a smooth gradient will show up limitations in colour depth much more readily than general content.

Sorry I made a mistake - If you use avisynth's colormatrix filter to convert to BT709 - colors will look "right" when youtube converts to RGB for display for SD source - I just confirmed this

colormatrix(mode="rec.601->rec.709" , clamp=0)

That's what I was wondering

Using the image in this post as an example:
https://forum.videohelp.com/threads/329866-incorrect-collor-display-in-video-playback?p...=1#post2045830
If a standard def video (DV, DVD, etc) using BT.601 colours is uploaded to YT, it will look like the rightmost example when played back (I think) - dull greens and bright reds. But the black/white level should be ok, as that's a separate issue.

But I'm not sure how to do that in linux? Maybe it will work with wine ?

Possibly, but I've had mixed results using wine in the past with other programs. Some folks have got avisynth and wine working together - so at least there's hope:
http://ubuntuforums.org/archive/index.php/t-1333264.html

AviSynth v3 should support Linux natively:
http://avisynth.org/mediawiki/AviSynth_v3
but the project looks like it's been stalled for quite some time

The mlt framework might allow for 601/709 conversions:
http://forum.doom9.org/archive/index.php/t-158840.html
need to look into it further...

Also it looks like Blender does handle levels properly on imported video (and also when rendering/exporting) - the faulty DVD Demystified test chart confused me.

So what I said in this post:
https://forum.videohelp.com/threads/331639-How-can-I-achieve-a-Technicolor-effect-in-Vi...=1#post2056079
..was wrong. Oh well.

[Toastie]

I have a video I know is NTSC interlaced ITU-R BT.470-2 (ITU-R BT.601) mpeg-2. I just want to reduce the filesize .m2v down a bit so I can remmux with the audio and author it to fit on a DVD-5. I used DGIndex to create the .d2v file which is referenced by MPEG2Source= in my script like this:

LoadPlugin("C:\Program Files (x86)\DGMPGDec\DGDecode.dll")
MPEG2Source("D:\DVDs\2011-03-12\myvideo.d2v")
ConvertToYUY2(interlaced=true)

I don't care about converting to RGB. I jsut want to make sure I pass the correct colorspace/colormatrix info to CCE so it doesn't mess things up a bit or have uneeded conversions take place.

I'm a little confused on what to do. I was looking at the documentation that comes with the colormatrix filter and it says:

"In case you captured something or you have a XviD/DivX (both are encoded Rec.601 coefficients), and you want to encode it to mpeg-2 using CCE (which assumes Rec.709 coefficients), you should use the following script (progressive material):

ColorMatrix(clip, mode="Rec.601->Rec.709")

Since have I interlaced:

ColorMatrix(mode="Rec.601->Rec.709", interlaced=true)

would seem like the thing to do to feed CCE 709 which it expects by default. Problem is that it is doing a conversion and then CCE is going to switch things back to 601 for the output file it creates. I've done some experiments and CCE always seems to create 601 m2v files, regardless of the colormatrix settings. 601 is what I want I think since this is SD video, not HD.

Isn't there a way to tell CCE that you have 601 and to leave it alone and not mess with it by incorrectly assumeing its 709 that needs to be converted to 601??

I don't know that the output file is going to really look different one way or the other. I just want to do this with a clean minimal conversions path from source to destination.


Thanks!

[poisondeathray]

The information in the documentation is incorrect. Many years ago some people thought SD DVD used Rec709 when being converted from the RGB master. This is not the case, and the documentation has not been updated yet

So leave your script as is, and it will work fine, you stay in YUV , no RGB conversion or uncessary transformations

[jagabo]

CCE (which assumes Rec.709 coefficients)

CCE doesn't flag the color matrix at all. The player, seeing no matrix flagged, will assume a color matrix. In the case of DVD that will be rec.601. In general, unflagged SD video will be assumed as rec.601, unflagged hd video as rec.709.

[poisondeathray]

And to clarify for the earlier posts about flash in general: it depends if the client computer is using hardware acceleration or not for flash:


HW on : YUV=>RGB with Rec601 (always, regardless of flags)

HW off , no flags : YUV=>RGB with Rec709
HW off , flags specified : YUV=> RGB with flag

Both range flags, and colormatrix are accepted when HW is off


*Note flags don't affect youtube , which re-encode and disregards flags. You need a non re-encoding web host like your own site to use flags properly



And intracube, FFMBC has implemented a colormatrix filter . I don't know if it's build into commonly distributed generic FFMPEG binaries

I haven't tested it , but I think the syntax is
-vf colormatrix=bt709:bt601
-vf colormatrix=bt601:bt709

Changelog:
Convert HD YUV BT709 to SD BT601 and vice versa
New filter colormatrix to convert between bt 601 and bt 709.

http://code.google.com/p/ffmbc/

[intracube]

And intracube, FFMBC has implemented a colormatrix filter . I don't know if it's build into commonly distributed generic FFMPEG binaries

I haven't tested it , but I think the syntax is
-vf colormatrix=bt709:bt601
-vf colormatrix=bt601:bt709

Thanks pdr. The colormatrix filter isn't available on the recent FFMPEG build that I'm using. Fingers crossed it'll get added at some point.