TMPGEnc CCIR601 option REVEALED!

[MrMoody]

Everyone who's used TMPGEnc much knows there is a mysterious option labeled "Output YUV data as Basic YCbCr not CCIR601"

(EDIT)
This thread explores theories and tests of this option in an effort to determine when you should use it.

Conclusion
If your input is YUV the box should NOT be ticked. This includes DVDs, all mpg files, and any file that uses MPEG (3ivx, Divx, Xvid, MP4x, WMV, MOV, HuffYUV encoded as YUV, etc).

If your input is RGB, the box needs to be ticked only if the RGB values all fall within the range 8-235, and not ticked if they are full range of 0-255. RGB includes BMPs, JPGs (still images should always be 0-255 and not need the tick however), HuffYUV encoded as RGB, most if not all DV, etc.

If your input is a frameserver index/signpost/script file (like d2v, vdr, avs) then it will depend on the functioning of the frameserver and the options you have set in it.

To find out if you need to tick the option, open Settings, go to the Advanced tab, and double-click Custom Color Correction. Tick "Show Histgram" (sic) and a "Histgram" window will appear. At the bottom, select YCbCr. If your histogram looks like this (no gap at the left side), you do NOT need the "Output YUV ..." option enabled:

But if your histogram looks like this (obvious gap at the left side), you do need the option enabled:

[FulciLives]

The proof is in the pudding.

Give me the pudding

- John "FulciLives" Coleman

[lordsmurf]

"The proof is in the pudding" = "The alcohol is in the dessert".
Sounds good to me. 8)


Sadly, everybody has tested this... with differing results.

[FulciLives]

"The proof is in the pudding" = "The alcohol is in the dessert".
Sounds good to me. 8)


Sadly, everybody has tested this... with differing results.

I think the problem is that this "dreaded option" as I like to call it acts differently based on the codec and/or format of the video source as well as the luminace level of the source (i.e., 0-255 vs 16-235).

Unless you are dealing with DV AVI using the Canopus DV codec you are best to leave TMPGEnc Plus alone and go with CCE where you know exactly WHAT is going on.

LordSmurf ... is that a flaming dessert?

Nothing more fun at a party than lighting those alcoholic desserts on fire

- John "FulciLives" Coleman

[MrMoody]

Here is the original image, range 0-255:

MPEG-2 encoded from BMP, box UNchecked:

Box CHECKED (note this combo clips the values and ruins the image) :

Input image modified to be within 8-235:

Encoded with box UNchecked:

Box CHECKED:


If anyone can give an example where CHECKED does NOT increase the contrast or UNCHECKED changes it either way, give it up. I want this nailed down.

[adam]

MrMoody, your results are exactly as I would expect, however you are misinterpreting what they mean. Neither setting is better or worse for pc playback. Mpeg must use some form of YUV colorspace which in and of itself does not support the full 0-255 range. Thus when you play your mpeg back via a decoder it is going to expand the range back out to 0-255 anyway. This CCIR601 setting is strictly used to instruct the encoder whether or not to compress your luminence scale. Let me explain by describing what happened in your test.

Your source is 0-255 and since your output is going to mpeg, its going to wind up 8 or 16-235. By leaving this option unchecked TMPGenc remaps your digital 0 to digital 16. Nothing is lost during the conversion to mpeg as your blackest black is stored at 16 now, the same as YUV. Same thing on the opposite end of the scale with whites.

By checking this option you are instructing TMPGenc that your source already conforms to CCIR601 in that your luminence ranges already contain headroom and footroom. You are telling it that you have no luminence data below 16 or above 235. Thus when it converts to mpeg it does not do any remapping. If your source is actually 0-255 then anything below 16 or above 235 is hard clipped. Those luma values are gone and that's why your output looks so bad.

I don't think this setting is mysterious at all. The tooltip accompanying it describes exactly what it does. If your source is CCIR601 (16-235), then enable it so that your luminence ranges are not further compressed (ie: 16 gets remapped to 32). If your source is 0-255 and does not yet conform to CCIR601, leave the option unchecked so that TMPGenc will make it so. If you use the wrong setting for your source you will either hard clip some luminence values or you will over compress your luminence scale. Both will look bad and somewhat similar.

Lordsmurf, if I remember correctly a very large number of us reached the same consensus regarding this setting. I remember your results were opposite of ours but then we discovered that your source was actually opposite of what you thought, (0-255 versus 16-235 or vice versa, I forget) thus your results were the same as ours the whole time.

This setting is no different then the luminence levels setting in most other encoders, they usually just label the options 0-255 and 16-235 instead of using TMPGenc's typical Engrish.

[MrMoody]

Ah! I think I get it. That's why the last picture looks like the first original: it's actually correct based on the input.

So when your input is MPEG or YUV colorspace, you should CHECK the box, but if you're using anything RGB (including conversion from YUV to RGB by Avisynth etc) then it should be UNCHECKED. Right?

[lordsmurf]

It doesn't matter anyway.
Your DVD player will still do what it wants.

It's just better to switch encoders.

[adam]

Ah! I think I get it. That's why the last picture looks like the first original: it's actually correct based on the input.

So when your input is MPEG or YUV colorspace, you should CHECK the box, but if you're using anything RGB (including conversion from YUV to RGB by Avisynth etc) then it should be UNCHECKED. Right?

Well, being RGB doesn't mean it has to be 0-255. Basically if your source is YUV you know its 16-235. Anything else and you've got to test it via a histogram or something. Actually, everything gets converted to RGB before TMPGenc accesses it anyway, so there are no hard and fast rules. Like I said, I think its easy to tell what to set this to depending on what your source is...the hard part is knowing what your source actually is.

It doesn't matter anyway.
Your DVD player will still do what it wants.

This is not something that can vary from one player to the next. A digital luminence value has a set analogue equivalent. The player cannot change this. If you hard clip luminence values that data is simply discarded. Its no longer present in your source so there's no way a dvd player could compensate for that, and it will look bad on any player. If you overcompress your luminence values then you are limiting the dynamic range of your luma. If your player has a super black option your could correct it, but then you wouldn't be getting super black anymore either and this wouldn't even work for PAL.

Simply put, there is only one correct way to set your luminence levels and if you don't do it then it will adversely affect playback on all players and decoders. This is something that must be set correctly in all encoders, not just TMPGenc. The only time such setting doesn't matter is if your source stays YUV the entire time, in which case there's no way you could screw it up.

[Shadowmistress]

This thread is getting too technical again.

Remember that the point of the conversion is so that we can watch it on TV (8-235).

First thing, MrMoody's pic is 0-255. If he were to keep that luma range and stick it on tv it would automatically be compressed to 8-235 by the tv. If he was to convert it to 8-235 it would still play at 8-235 on tv. Is there a difference? IMO minimal. Therefore, the box should be UNCHECKED .

MrMoody's pic is 0-255. If he intended it to stay on a computer monitor for all time, then the box should be UNCHECKED so no change would be made.

IF MrMoody's pic was 8-235 and he wanted to play it on tv, the box should be UNCHECKED so that no change was made.

IF Mrmoody's pic was 8-235 and he intended it to stay on computer monitor for all time, only then could the box be CHECKED. But remember that the monitor will take an 8-235 pic and expand it automatically to 0-255 anyway. Is there a difference? IMO minimal. Personally I'd leave it UNCHECKED in this instance also, that way I don't have to go messing about with this feature!

Adam, I agree with everything you're saying about how it works. But unless you're trying to compensate for a previous encode messing up the luma scale, I think it's safe to do your encodes and leave the luma scale alone.

If your source is CCIR601 (16-235), then enable it so that your luminence ranges are not further compressed (ie: 16 gets remapped to 32).

I don't believe that happens. I think tmpgenc leaves the source alone if the box stays unchecked. But go dig out that hex viewer thingy you're talking about and show me that Tmpgenc will take a 8-235 source and compress it to 16 or 32 - when the box is left unchecked.

That's my two cents worth.

P.S. Remember that we are viewing all of MrMoody's pics on a monitor and therefore they have *all been adjusted to 0-255.

[jdizzy40]

NIce Kitty! Can't tell the real difference though!

[MrMoody]

@shadowmistress:

Actually, adam has convinced me he's right. Once he explained it in terms of colorspace I got it. I think the thing that's fooling you is also what fooled me: The computer MPEG-2 decoder is expanding the 8-235 stream back to 0-255 for display.

If your source is CCIR601 (16-235), then enable it so that your luminence ranges are not further compressed (ie: 16 gets remapped to 32).

I don't believe that happens. I think tmpgenc leaves the source alone if the box stays unchecked.

I think it will. This is easily tested; be back soon.

[BJ_M]

It doesn't matter anyway.
Your DVD player will still do what it wants.

It's just better to switch encoders.

its true on dvd players - they are not even consistent output from the same brand when played back to a scope ..

[Shadowmistress]

Here's my pudding:

I think the thing that's fooling you is also what fooled me: The computer MPEG-2 decoder is expanding the 8-235 stream back to 0-255 for display.

No. That's a conflict of interest. The mpeg encoder wants to make it look good on your tv so it will do what you tell it to do and nothing else. The MONITOR is changing it to 0-255 for display.

But without the hexviewer results this is all just speculation anyway.

[MrMoody]

Trying to figure out black boxes will make your brain hurt. I'm beginning to see why trying to figure this out made Fulci and Smurf tear their hair out.

I've had no success getting TMPGEnc to compress the first clip further. I tried: inputting the first clip directly, inputting through AVISynth+DGDecode, and then again adding ConverttoRGB24. The only time I've gotten a compressed image is the clip in the 5th picture above, by feeding in a BMP with 8-235 range, then the decoder output is compressed to 8-235 when it should have theoretically decoded as 0-255.

Seems like when feeding in MPEG-2 (even through a frameserver) TMPGEnc somehow senses the clip is already CCIR601 and BEHAVES as though the box is checked. You can force it to assume the clip IS YUV but you can't make it assume it ISN'T if it knows better.

If anyone can come up with an example of an input movie clip in any format where UNCHECKED results in luminance compression vs playback of the original clip I sure would like to hear what you did. Until then, I feel safe leaving the box unchecked. To see a histogram, load your clip into VirtualDub, go into Filters, add a Levels filter, hit Show Preview, and then Sample Video. A luminance compressed image will have no black bars at either end, like below:

I really think the only time you need to CHECK the box is when you have an input clip that looks like this in Virtual dub to start with. A properly encoded MPEG-2, which is CCIR601, fills the histogram when looking at it this way.

[BJ_M]

5th down image

[BJ_M]

6th down image

[BJ_M]

4th down image

[BJ_M]

2nd down image

[BJ_M]

3rd down image

[BJ_M]

first image

[Shadowmistress]

Thanks for proving it. Unchecked box does nothing!
Check out the bottom right box of BJ_M's evaluations.

[daamon]

I've been following this thread with interest... Not contributing coz I'm learning rather than being able to add.

So why does the MPEG2 picture look a bit washed out (look at the arm of the chair, near the top of the picture) when compared to the original? (Pic 2 compared to pic 1 in the post immediately above).

Other dark areas show the same (slightly) washed out look...

[FulciLives]

My head hurts.

Use CCE and leave it YUV/YUY2 through out from capture to AviSynth scripting to CCE encoding.

The RGB environment of TMPGEnc Plus has to be a negative thing any way you slice and dice it.

- John "FulciLives" Coleman

[daamon]

My head hurts.

I'm not surprised if your avatar is anything like the real thing!!!

[MrMoody]

My head hurts.

Well, no wonder. Your skin's off!

Use CCE

But then how do we set this setting:

The RGB environment of TMPGEnc Plus

I believe you that it uses RGB but where is this documented/shown?

[Shadowmistress]

So why does the MPEG2 picture look a bit washed out ... when compared to the original?

Wipe the dust off your monitor :P

[daamon]

LOL - Very good. But that'd mean that there was magic dust on my monitor that scrolled with the image... Gotcha...

Seriously though, I'm curious.

[FulciLives]

My head hurts.

Well, no wonder. Your skin's off!

Use CCE

But then how do we set this setting:

In CCE if the source is 16-235 (as it should be with a YUV/YUY2 capture) you select the 0-255 option. If your source is 0-255 then you select 16-235 (I think) but I only do YUV/YUY2 captures.

As far as TMPGEnc Plus and VirtualDub being RGB programs ... ummm ... that is what everyone says

- John "FulciLives" Coleman

[Shadowmistress]

Fulcilives, if you take an 8-235 capture and convert it to 0-255, your tv will promptly convert it back to 8-235 once you play it. The two conversions cancel eachother out.

If you run it through tmpgenc and leave the box unchecked, there are no conversions and it stays 8-235 on source and on tv.

If you use CCE where you don't know which is the "leave it alone" setting, God help ya!

So why does the MPEG2 picture look a bit washed out ... when compared to the original?

Ok, in all fairness, I had to really squint to see the difference in pictures. Then the reason hit me.
Picture #1 is a native 0-255. Monitor displays it at 0-255.
Picture #2 has been converted by tmpgenc to 8-235 and then converted back to 0-255 by your Monitor. These two conversions sorta cancel eachother out, but thats why you see the minimal quality difference.

It's like resizing a flick from 704x480 down to 352x240, then resizing back up to 704x480. You will lose quality. The Monitor is resizing picture #2 but it shows a drop in quality.

When leaving the box unchecked in tmpgenc, it will produce the colorspace at 8-235 which is suitable for tv.