TMPGEnc CCIR601 option REVEALED!

[Shadowmistress]

Oh shit, I just realized that I completely contradicted myself on that last part of my statement with what I said in red next to Pic #2. Here's the explanation:

Although Pic #2 is now a native 8-235, the monitor has rescaled it to 0-255. BJ_M did the histogram on the pictures after they hit the monitor. So technically the historgram is right because it's accounting for what the monitor did to the picture.

Hey BJ! What's that program called that you're using?


My contension all along was that leaving the box unchecked would produce a 8-235 scale, regardless of what type of source you feed it.

[BJ_M]

the monitor does nothing to the histogram -- in fact, that system doesnt even have a monitor and i operate it with ultravnc ...

a tv will not cancel out or change or de-core levels --

ntsc is IRE7.5 and if 0-255 is used (which will look good on a pc monitor) , it will not look good on a tv ..


NTSC is based on 75% color saturation and broadcast will allow ussually up to 121 IRE - and hard clip it ..

a NTSC image will look 'grey' on a pc monitor ..

in both cases - the tv and pc monitor are assumed to be properly set up -- one also is lin. gamma and the other not -- and the color temps are usually different .

judging an image on a pc that will be shown on a ntsc monitor is pretty well a waste of time unless you use a scope (even then - with editing - you should view on true ntsc monitor ) ..



I used Sony Vegas for the scope pictures ...

[andie41]

If the "output YUV as basic YCbCr..." is unchecked, TMPGenc will compress the colorspace by 16. If you are using 0-255 and outputing to NTSC video, this is fine - you will then have 16-235.

However, if you capture, for example, analog video at 16-235, and produce mpeg in TMPGenc (or other encoder) and leave the "output YUV as basic YCbCr" box unchecked, the video will be further compressed to approx 32-220 - not good.

An additional problem is that some, most, my, NTSC DVD players add 7.5 IRE setup. So, if you began with 16, did not check the "output YUV as basic YCbCr..." in TMPGenc and make a DVD and play it in a NTSC DVD player, you could end up with black level at approx 48- where are my blacks.

[FulciLives]

Fulcilives, if you take an 8-235 capture and convert it to 0-255, your tv will promptly convert it back to 8-235 once you play it. The two conversions cancel eachother out.

If you run it through tmpgenc and leave the box unchecked, there are no conversions and it stays 8-235 on source and on tv.

If you use CCE where you don't know which is the "leave it alone" setting, God help ya!

I think it was Adam that told me when using CCE to select 0-255 when you have a YUV/YUY2 capture ... such a capture will always be 16-235

I seem to recall him saying in CCE you always pick the opposite option of the source (if source is 16-235 then you pick the 0-255 option).

Actually now that I think of it ... I don't think that the 0-255 / 16-235 option in CCE even does anything if you are inputing a "proper" YUV/YUY2 16-235 video clip.

I'm sure either Adam or maybe even BJ_M can clear up the why or what to do when it is an RGB source but I do know that my captures turn out looking like the original in terms of luminance.

As I have said in the past (if not this thread then many others) I always do a YUV/YUY2 capture (PICVideo MJPEG but now HuffyUV with my new fast ass computer) ... use AviSynth with no colorspace changes ... then encode with CCE SP using the 0-255 option selected.

Works for me

Not only can I do more than a 2-pass VBR but it is MUCH faster than using TMPGEnc Plus. I would say that the quality is just a tad better as well though TMGPEnc Plus is very nearly as good.

- John "FulciLives" Coleman

[MrMoody]

adam's theories notwithstanding, as far as I can determine and in every test I've tried, the YCbCr option in TMPGEnc behaves just like a luma levels filter set for input values of 8-235 and output of 0-255. Uncheck it, it always does nothing. Check it, and it always behaves the same as the levels filter in VirtualDub set to those numbers.

Reward: First person that posts or PMs me a reproducible example that does NOT behave this way, I will Paypal you $5 so you can buy yourself a beverage. I think my money is safe ...

Here's my operating theory. If TMPGEnc thinks in RGB, it always operates correctly when converting from YUV input to RGB internally, so checking the box is not needed and in fact checking it will always ruin (proper) YUV input. This option is merely to correct RGB input from codecs/capture drivers that produce a compressed RGB range instead of the full range it expects ... and in that case you're better off to feed it YUV if you can so you can leave it unchecked (just like CCE, surprise).

[MrMoody]

With CCE in quick tests, on YUV input the 16-235/0-255 option has no effect either way as FulciLives said. With RGB input, 16-235 leaves it unchanged, 0-255 acts like TMPGEnc's checkbox, again correcting for compressed range in the RGB values.

[andie41]

Uncheck it, it always does nothing. Check it, and it always behaves the same as the levels filter in VirtualDub set to those numbers

Unchecked, the colorspace is compressed, by a value of 16. It may be difficult to see this effect, depending upon one's equipment. Using NTSC colorbars and viewing these color bars, before and after, using Photoshop, waveform monitors in programs such as Vegas video, and with hardware waveform monitors, this effect is readily apparent - 0-255 becomes 16-235 (good), 16-235 becomes approx 32-220 (not so good).

[MrMoody]

Unchecked, the colorspace is compressed, by a value of 16. ... 0-255 becomes 16-235 (good), 16-235 becomes approx 32-220 (not so good).

We're saying the same thing, assuming you're putting RGB into it.

Box unchecked:
YUV input (16-235)>internally convert to RGB(0-255)>compress to MPEG(16-235).
RGB input (0-255)>compress to MPEG(16-235).
RGB input (16-235)>wrong, double compressed!

Box checked:
YUV input > wrong, clips the output. CCE won't do this.
RGB input (0-255)>same here. CCE WILL do this.
RGB input (16-235)>compress to MPEG(16-235).

The only way to get double range compression is to feed compressed range RGB into it without the box checked.

I defy you to make it double range compress a YUV input. I say you can't. And the only way to do it more than once from RGB is to find an improper decoder that outputs RGB as 16-235 from MPEG-2 (DGDecode doesn't).

[BJ_M]

Uncheck it, it always does nothing. Check it, and it always behaves the same as the levels filter in VirtualDub set to those numbers

Unchecked, the colorspace is compressed, by a value of 16. It may be difficult to see this effect, depending upon one's equipment. Using NTSC colorbars and viewing these color bars, before and after, using Photoshop, waveform monitors in programs such as Vegas video, and with hardware waveform monitors, this effect is readily apparent - 0-255 becomes 16-235 (good), 16-235 becomes approx 32-220 (not so good).

go back to the first page -- i already did scope the images ...

[adam]

No, the reason it is the same is because he has posted a bmp still decoded from the mpeg file. When it was decoded the luminence levels were expanded to 0-255 during the conversion to RGB. I can tell that this is what you meant when you said it was adjusted for the monitor, and this is going to happen most of the time. Yes the result is the same because TMPGenc remapped the luminence levels (compressed) before converting to mpeg. Therefore we can reverse this when we decode it for pc playback. This is the correct way to handle 0-255 footage.

Your luminence levels get compressed to 16-235 when you convert to mpeg...to be sure. That is all YUV supports. So if we then look at a .bmp still taken from that mpeg, and it shows 0-255 luma values, naturally its luminence levels were expanded during decoding. So it cannot be said that leaving the option unchecked did nothing simply because we wind up where we started. You've got to take into consideration the decoding that we are using to preview.

Again, mpeg uses YUV and YUV only supports 8,16-235. When you convert to mpeg it is being compressed to 16-235 whether you know it or not. The only question is how you get there. With a 0-255 source there are two options. You can either hard clip everything below 16 and above 235 or you can remap 0 to 16, and 255 to 235, (aka compress) and then chop off everything above and below...which is now nothing.

Looking at the pic you referenced compared to the one above it, the fact that they are the same is evidence that TMPGenc DID compress the luminence levels. If it hadn't then the outer levels would have been hard clipped and then once decoded to 0-255 on the pc it would have looked different.

FulciLives, if you are loading a YUV source into CCE it doesn't matter what you set the luminence levels to. Since it is a YUV->YUV conversion the luminence levels will never change either way. With an RGB source, its the same as with any other encoder (IMO.) If your source is 0-255 you want to use the 16-235 option to compress it, and if its 16-235 you wan to use the 0-255 option to make sure that you don't further compress it.

[adam]

With CCE in quick tests, on YUV input the 16-235/0-255 option has no effect either way as FulciLives said. With RGB input, 16-235 leaves it unchanged, 0-255 acts like TMPGEnc's checkbox, again correcting for compressed range in the RGB values.

This is the same as how I described above in your TMPGenc example. Your source is 0-255. When you choose 16-235 it remaps before converting. Nothing is lost. Now when you decode back to RGB it is expanded to 0-255 again and it looks identical to your source. It is not that "nothing" has changed, its that everything was preserved and thus looks the same at playback.

By selecting the 0-255 option you did not remap your values. You hard clipped everything outside of 16-235 and that's why it looks "greyer" even after being decoded to 0-255 again. Those outer levels were completely discarded.

[trock]

Probably a dumb question, but is there a free or inexpensive program that can determine what colorspace an existing AVI file is in?

[BSR]

Looks like everyone is getting different results, and I seem to be reading the same stuff over and over and over, just worded differently.

Here are my results from MY process, others may vary.
I use Sony DV cam to capture VHS through passthrough.
I have Sony DV codec installed (RGB output only).
I open clip in TMPGEnc.

With
"output YUV as basic YCbCr..." checked, = my DVD matches the VHS perfectly.
"output YUV as basic YCbCr..."unchecked= DVD is much lighter colored than VHS. Washed out. Blacks not as black.
This confirms what adam was saying, or at least on my setup and in this process.

I test on a TV and switch between DVD player and VHS player.
These are the results with MY 2 dvd players and 2 VHS player. Maybe others are not the same.
I tried doing comparisons on PC monitor, but depending on which program I used, I would get different results and shades luma.

So Test your situation and your process yourself till your happy. It's not hard to make 2 short clips of YOUR process and see which replicates the originals the best by comparing on TV.

To each his own. The topic of the thread interested me, but after reading through it.... my head hurts too.

[adam]

its true on dvd players - they are not even consistent output from the same brand when played back to a scope ..

You are saying that (not counting super/true black options) not all NTSC DVD players output a digital luma value of 16 as 7.5IRE? or for PAL, 16 as 0IRE? I did not know this, but even still I don't see how this makes the luminence levels setting in your encoder any less material, and I don't think you were suggesting this either. Even assuming the DVD player is completely off, hard clipping your luminence levels rather than compressing them, will adversely affect quality in the source itself, regardless of how or where its being played back. And, again assuming the player is totally off, over compressing your luma values will still adversely affect playback on any player because regardless of what digital value corresponds to the analogue equivalent that your player spits out, they are still operating on a set scale. So regardless of what my 16 eventually comes out as, either way I want it to represent the blackest black in my picture. If I overcompress my luma then this is not happening.

[MrMoody]

I use Sony DV cam to capture VHS through passthrough.
I have Sony DV codec installed (RGB output only).
I open clip in TMPGEnc.

With
"output YUV as basic YCbCr..." checked, = my DVD matches the VHS perfectly.
"output YUV as basic YCbCr..."unchecked= DVD is much lighter colored than VHS. Washed out. Blacks not as black.

That's because your Sony DV codec generates RGB that hasn't been expanded, RGB(16-235) as I referred to it above. This is what the checkbox is for. DV is the example given in the TMPGEnc tooltip for checking the box, by the way.

Question for you: How does the codec output to TMPGEnc? Does it make a complete, indepedent AVI (what fourcc code?) or some sort of frameserver pointer file (like a d2v or vdr)? If it's an AVI is it as big as you would expect, or tiny?

I suspect that AVI input should always be unchecked, but if your DV generates an AVI, that blows that theory.

[MrMoody]

I think we've all reached the same point by different paths. First post edited.

[BSR]

The DV generates the AVI(large 15gig or so/hour)
FourCC is dvsd.
AVIcodec reports Sony DV codec for the file.

Yes I assume also that the RGB from Sony is not expanding to 0-255.

[adam]

I agree MrMoody, I think we are all saying the same thing as summarized by what you said:

Box unchecked:
YUV input (16-235)>internally convert to RGB(0-255)>compress to MPEG(16-235).
RGB input (0-255)>compress to MPEG(16-235).
RGB input (16-235)>wrong, double compressed!

Box checked:
YUV input > wrong, clips the output. CCE won't do this.
RGB input (0-255)>same here. CCE WILL do this.
RGB input (16-235)>compress to MPEG(16-235).

[MrMoody]

Thanks. Here's my new theory. DV actually records in RGB, with the values scaled for direct TV playback (16-235). When you download to the PC, it doesn't convert the values, it just puts an AVI wrapper around the data. Then the codec does the same thing too - simply outputs the values from the file with no conversion. This is OK for conversion to another format (as long as you know about it) but means the AVI will not display properly when played back on a PC (your blacks won't be black, or whites white). This also means if you use VirtualDub to make an Xvid or whatever from your huge AVI, you need to apply a levels filter to expand to full range before the Xvid codec compresses it again.

[Shadowmistress]

So bringing the conversation back down to noob level, I guess we've reached a consensus that:

#1. You can only compress the color range if it is improperly identified. If tmpgenc recognizes it correctly it will not keep compressing it further and further when you leave the box unchecked.

#2. If your source is true RGB 0-255, leave the box unchecked as tmpgenc will remap the scale and make your picture look good.

#3. Only check the box if you are compensating for DV as MrMoody mentions above. In that case output the opposite scale of your source.

[adam]

I don't think that consensus can be drawn from what has been discussed here. I personally think the correct consensus is what MrMoody posted before. In any RGB->YUV conversion, the outer levels are clipped, naturally. Unchecking this option remaps first, checking doesn't. That's all a luminence level option in any encoder does.

1) No. Neither TMPGenc, nor any other mpeg encoder that I know of, excersizes any discretion over this whatsoever. When you convert RGB to YUV the outer levels get clipped. If there is no useful luma data there, nothing changes. If there is useful luma data there it will be discarded, that's why you have to remap your values to conform to YUV's scale first. Then when you clip the outer levels, you don't lose anything. If you leave this option unchecked in TMPGenc it will remap your values higher regardless of what they are at present. You will overcompress your luma if your source is 16-235 to begin with. I bet I know the exact reason why you can't reproduce this in your tests.

A. 16-235 source encoded with option unchecked yields roughly a 32-215 mpeg.

B. Load mpeg into TMPGenc and it decodes it to RGB, expanding the luminence range out to 0-255, giving you actual luminence values extending back out to 16-235.

C. Rinse, wash, repeat. You are, in fact, overcompressing your luminence values each time you encode. But then you are reversing this when you decode again. The same thing happens everytime someone looks at a decoded .bmp still from an mpeg and says, "look its 16-235 just like its supposed to be." But no, once decoded its supposed to be 0-255 again.

2) Well yes, except that I wouldn't say its actually doing anything to your picture quality but rather keeping any luminence data from being clipped. Its kinda like pulling your arm back in the window just before you pass another car. You can always stick it back out there again, but if you don't make that adjustment then you will lose the arm entirely.

3) Check the box anytime your actual luminence values conform to CCIR601 (don't extend beyond 16-235). This often applies to DV, but not always, and it can easily apply to any source other than DV.

[FulciLives]

I'm no expert when it comes to working with DV AVI footage. Most of my stuff has been PICVideo MJPEG or HuffyUV.

But I am 99.9% sure that DV AVI is YUV/YUY2

How do I know this?

The Convolution3D filter for AviSynth only works with YUY2 source videos and ... guess what ... it works with DV AVI.

I just brought this up since MrMoody said that his "new theory" has something to do with DV AVI being RGB with a 16-235 luminance range.

If DV AVI was RGB then the Convolution3D filter wouldn't work.

- John "FulciLives" Coleman

[BSR]

DV actually records in RGB, with the values scaled for direct TV playback (16-235).

DV records into YUY2 colorspace. The Sony DV codec on the computer decompresses to RGB. So does the Panasonic DV codec. Canapus can output either RGB or YUY, but has a Chroma upsampling bug(which can be fixed I do believe with an avisynth script.)
Not all DV codecs act the same way with 0-255 and 16-235.
Doom9 has some threads discussing this.

So bringing the conversation back down to noob level, here's my question to BSR.

There's a nice way to put it.

EDIT - deleted responses. Previous post changed... Stepped away from my computer too long.

[MrMoody]

The Convolution3D filter for AviSynth only works with YUY2 source videos and ... guess what ... it works with DV AVI.

I just brought this up since MrMoody said that his "new theory" has something to do with DV AVI being RGB with a 16-235 luminance range.

If DV AVI was RGB then the Convolution3D filter wouldn't work.

We're straying a little here, and I'm sure you're right about DV (which I know nothing about) but I thought AVISource converts everything to YV12 by default, whether the input is RGB or not. I could be wrong ...

[BSR]

AVISource converts everything to YV12 by default, whether the input is RGB or not. I could be wrong ...

I do believe AVIsource() uses the output of the codec you have installed.
Directshowsource() converts your DV to YUY2.

EDIT - fixed quote

I use AVIsource() with convolution3d() and I need to add converttoyuy2(interlaced=true) before the filter.
Even tried the YV12 version of con3D.

[FulciLives]

AVISource converts everything to YV12 by default, whether the input is RGB or not. I could be wrong ...

I do believe AVIsource() uses the output of the codec you have installed.
Directshowsource() converts your DV to YUY2.

EDIT - fixed quote

I use AVIsource() with convolution3d() and I need to add converttoyuy2(interlaced=true) before the filter.
Even tried the YV12 version of con3D.

So it is codec dependant then?

Blah.

I'm sticking with what I know hehehehe

- John "FulciLives" Coleman

[Shadowmistress]

First of all, @Adam: speak english!....no wait, speak noob!

Since you agreed with me on what to do above - my consensus points #2 & #3, (you disagree with how it works but you agree with what to do) I guess point #1 is all that we disagree on.

You said wash, rinse, repeat... so I did. After all, I was curious.

I took 20 frames of a commercial which came in an avi format. (No, I don't know what the original RGB/YUV value is). I put it into tmpgenc, kept the same size and framerate and default settings, and coded all at CBR 7000 kbps. I made a file labelled one, then loaded one into tmpgenc and made file labelled two, loaded two in and made file three... etc. I washed it 20 times.
The results were unexpected.

Here is what happend when I always left the box unchecked.


So then I thought, ok I should be compensating and coded every other clip with the box checked. This theoretically should preserve it to as close to the original. This is what happened.


Then I thought, this can't be right, what happens if I leave the box checked through all the washes? And I got this.


Is there a better preservation technique that I'm missing here? Could it be that it's just signal degredation from being washed 20 times? If you got a hold of the 20th encode of a film, which of these 3 would you prefer it be?

In my opinion, always leaving the box unchecked does the least damage to the scale.

[BJ_M]

resize them smaller or compress them more as jpeg

[canadateck]

Host your images here http://www.exs.cx/

[lordsmurf]

Blah.

I'm sticking with what I know hehehehe

- John "FulciLives" Coleman

I'm with you. I'd rather make video than talk in circles about it.

Senseless setting names, poor translation, end of story.
Switch to something that is written in English, not psuedo-English.