Colorimetry

[txporter]

As I get deeper into video manipulation, I start to see more and more things that I missed or glossed over on my multiple first passes through learning various program. The most recent one to confuse me is Colorimetry between HD and SD. I have learned that HD material uses the Rec. 709 (BT.709) colorspace, while SD material tends to use Rec. 601 (BT.601). From what I have seen posted, it seems that I should change colorspace from Rec. 709 to 601 when resizing HD material to SD.

I tried to do a little reading on wiki (709 and 601), on the ColorMatrix plugin itself, and some of the links off of the wiki articles. I like this one particularly.

Based on reading the ColorMatrix info, it sounds like DVDs are usually stored as Rec. 709 (first line under Description of filter)? I expected to see that they were usually Rec. 601. I tried loading a video file in GSpot to see the Colorimetry info like mentioned on the ColorMatrix page, but I don't see that info anywhere.

What is my goal supposed to be? Do I want to output the video with the same colorimetry as it starts with (if not converting between HD and SD resolutions)?

It looks like I can have ColorMatrix read this info itself by either using hints from MPEG2Source (info=3 + hints=true on CM), or by feeding the d2v file itself to ColorMatrix. If I do this, does this simply ensure that whatever colorimetry that is used in the source video is also used in the output video? In other words, I would still need to convert from 709 to 601 if resizing from HD to SD resolutions?

I think I am making this more complicated than I need to, but I am having trouble understanding it. Any help is appreciated.

[jagabo]

DgIndex will show you the colorimetry in MPEG 2 videos. If the colorimetry isn't specified it assumes 601 for SD, 709 for HD (and indicates this with an asterisk by the field). Most DVDs are 601, a few 709. If you use an MPEG encoder that supports colorimetry flagging (HcEnc, for example) you can use whichever you want. Otherwise you should use 601.

[txporter]

Ok, thanks for the tip on DGIndex. For the most part, I do use HCenc but I haven't been setting the *COLOUR parameter in the ini file.

These are my most common conversions (1-3 with HCenc, 4-5 with xvid_encraw):
1. DVD video --> SD MPEG2
2. 720p capture --> 720p MPEG2 (manipulated to drop dup frames)
3. 1080i capture --> 720p MPEG2 (inverse telecine and resize to save space)
4. DVD video --> SD MPEG4-ASP/AVI
5. 720p/1080i capture --> SD MPEG4-ASP/AVI

It sounds like I probably don't need to worry about ColorMatrix with any of those conversion except rare DVDs for #1 & #4 and all conversion for #5, correct?

I did check a couple of clips that I have here on my laptop. Two DVD rips are BT-420-2 B,G (BT.601) and two HD clips (one 720p clip and one 1080i clip) both showing BT.709.

[jagabo]

I did check a couple of clips that I have here on my laptop. Two DVD rips are BT-420-2 B,G (BT.601) and two HD clips (one 720p clip and one 1080i clip) both showing BT.709.

If there is an asterisk next to the report in DgIndex the colorimetry was not flagged in the video, it's just a guess. Depending on how you handled the conversion it may or may not be correct.

For example, if you opened an HD MPEG video in VirtualDub for filtering it would convert to RGB with a rec.601 matrix -- which would be wrong.

[txporter]

For example, if you opened an HD MPEG video in VirtualDub for filtering it would convert to RGB with a rec.601 matrix -- which would be wrong.

Does avisynth do the same thing (always convert to Rec. 601)?

[jagabo]

For example, if you opened an HD MPEG video in VirtualDub for filtering it would convert to RGB with a rec.601 matrix -- which would be wrong.

Does avisynth do the same thing (always convert to Rec. 601)?

If you left the video in YUY2 or YV12 it wouldn't be a problem. If you converted to RGB the default is to use a rec.601 matrix. If the video was rec.709 the correct command would be ConvertToRGB(matrix="rec709").

[manono]

To make it simple, for 5 convert to BT.601 in the script. Leave the rest alone. Leave HCEnc at default, which I'm assuming makes the output colorimetry the same as the input colorimetry.

Of course, with downloaded stuff you never know what they did to it, but most of those guys wouldn't know enough to mess it up when going from HD to HD. Going from HD to SD is another matter, and I would expect a lot of downloaded stuff not to have been done properly.

Based on reading the ColorMatrix info, it sounds like DVDs are usually stored as Rec. 709 (first line under Description of filter)?

Parts of the ColorMatrix doc are dead wrong and that's one of them.

[txporter]

To make it simple, for 5 convert to BT.601 in the script. Leave the rest alone. Leave HCEnc at default, which I'm assuming makes the output colorimetry the same as the input colorimetry.

Of course, with downloaded stuff you never know what they did to it, but most of those guys wouldn't know enough to mess it up when going from HD to HD. Going from HD to SD is another matter, and I would expect a lot of downloaded stuff not to have been done properly.

Based on reading the ColorMatrix info, it sounds like DVDs are usually stored as Rec. 709 (first line under Description of filter)?

Parts of the ColorMatrix doc are dead wrong and that's one of them.

Thanks, Manono. That sounds like the right thing for me to do. I did a little testing with HCenc yesterday. It does keep the input colorimetry unless I resize from HD to SD. HCenc changes the input Rec.709 to Rec.601 for me. Or at least that is what DGindex reports. However, it is a guess again (*) so maybe that is incorrect. At any rate, I think your suggestion is a good one and the one that I am going to use.

I don't mess with any downloaded stuff (torrents, etc). I am just manipulating Tivo recorded material and DVD rips. Seems like the files on some of those torrent sites can have all sorts of problems.

[jagabo]

I did a little testing with HCenc yesterday. It does keep the input colorimetry unless I resize from HD to SD. HCenc changes the input Rec.709 to Rec.601 for me. Or at least that is what DGindex reports.

HcEnc has no idea what colorimetry the input file is. By default it doesn't flag the colorimetry. The asterisk in DgIndex indicates it's assuming the video is rec.601 because it's SD and not flagged. If you downsized an HD rec.709 source and gave it to HcEnc and didn't specify the rec.709 flag the video will be played back incorrectly.

[txporter]

Good to know. Thanks, jagabo.

[PAL sucks]

I don't know if I have to start a new thread but this is the best one I found searching.

Using avisynth and cce sp2, what are the most correct colorimetry changes needed in order to not fudge a reencode?

For HD to SD, would this be what to use:
colormatrix(mode="Rec.709->Rec.601", inputfr=true, clamp=0)

or could 709 uncompressed levels be used when viewing on hdtv? Would they look horrible on a crt tv? Do modern dvd players handle levels accordingly or treat everything as compressed?

Does cce care if it's not rgb? It just passes through whatever is given to it for yuy2?

I am under the impression that legal SD requires 601 with compressed luma levels because that is what I see using histogram when checking dvds beforehand (although I am just going by the luma readings alone in dark scenes and borders cause I just started paying attention to this stuff). There is usually a long flat line butted up against the illegal zone but for HD that zone is filled with action. Does (tv not pc)601 also mean compressed luma or can you have rec.601 with full luma? (None of my reencodes used a conversion to RGB, btw.)

So for SD to mpeg2, no change?
HD to SD, convert and crush luma too? Or is this just a recommendation for tube tvs?

I have read several pages and the most assertive statement I found was an afterdawn (I think) guide stating that cce expects 601. I have also seen posts saying it expects 709 but that was an older thread.

Of course, I am not sure what exactly that means and am guessing 601 is compressed luma and chroma and 709 is full expansion and players expand 601 on playback, I know pc specs are different. I also read that no one has access to dvd specs and maybe that's why there are so many posts about it.

[Gavino]

Of course, I am not sure what exactly that means and am guessing 601 is compressed luma and chroma and 709 is full expansion and players expand 601 on playback, I know pc specs are different.

Both 601 and 709 (as Rec.nnn or TV mode) specify what you call 'compressed' luma and chroma, meaning that they use a compressed range (16-235 for luma, 16-240 for chroma) to represent RGB 0-255.
The difference between them is to do with the RGB<->Y'CbCr conversion matrices, not the range of levels, which is a separate issue (TV v PC range).

[jagabo]

Gavino already said some of this, but in general:

The conversion matrices only determine how RGB is converted to YUV and vice versa. ColorMatrix()'s mode="Rec.709->Rec.601", is simply a shortcut that converts a YUV 709 source to YUV 601 without the intermediate RGB step.

Digital video (both rec.601 and rec.709) uses YUV with luma values from 16 to 235. and chroma values from 16 to 240. Computer monitors use RGB values from 0 to 255.

The default in most programs is to stretch the luma from Y=16-235 to RGB=0-255 when converting YUV to RGB. Conversely when converting RGB to YUV they compress the luma from RGB=0-255 to Y=16-235.

Use rec.601 for SD, rec.709 for HD, unless otherwise flagged.

What shows up on the screen (what you actually see), on old CRT displays as well as modern LCD and Plasma displays, is RGB.

[PAL sucks]

Well that cleared things up. Always convert luma to 16-235 for TV if uncompressed or if someone extended them.