TMPGEnc - "Output YUV data as Basic YCbCr not CCIR601&q

[DJRumpy]

LOL

nimco, your emoticon was perfect. Gave me a good laugh for today. It does look a little overwhelming doesn't it?

If I were you, and I was uncertain as to which setting to use, I would pick a very dark scene, and encode about 2-5 minutes of that scene, using both settings (on and off). See which looks better to your eyes. Look for 'sparkles' in the very dark areas. These sparkles will show up as tiny blocks of gray or greenish gray color.

The setting is source dependant, meaning that it will vary according the the luminance range (0-255, or 16-235), that your input video is using. If you do not have an easy way to verify which luminance range your source is using, then this is the simplest way, requiring nothing more precise than your eyes

[HAGGARD]

Hi all...

I did some tests regarding this topic and I've found out that this option should be DISABLED to leave inputvideo as is!!!
When this option is ENABLED, TMPGEnc will expand luminance to the 0-255 range!!!

I ran the test like this...
Created a still image from Premiere's Bars en Tone (CCIR601)...
Encoded this frame with TMPGEnc with setting enabled
Encoded this frame with TMPGEnc with setting disabled
Opened both m2v files in VDubMod and exported them to BMP
Then opened all files (orginal/setting enabled/setting disabled) in Paint Shop Pro and checked with the colorpicker how luminance/color had changed...

The picture with the option enabled is much brighter than the original, but values below 16 are cutoff to 0. i.e. A darkgrey (16,16,16) is converted to black (0,0,0). In other words loses more information!!!

The picture with the option disable is (almost) identical with the original file, so I would say leave this option disabled...


BUT.... How does a DVD player interpret this information???
Is it capable of playing 0-255 range??? Then it the option in TMPGEnc should be enabled!!!
If it doesn't... Then the option should be disabled!!!
How the hell can I test that?!?

'HAG

[adam]

edited---give me a sec, tables aren't lining up.

[adam]

Listen to what you are saying. The darkest part of your source (the part represented by (16,16,16,16) has now become 0. What have you lost? Nothing. There were no colors beyond this point to lose, all you have done is remap your values to a new scale. The darkest color in your source is still represented in your new image it has just been moved to the outer edge of the scale.

That is not grey at the (16,16,16,16) point. It is black and it should be pure black. But it looks like gray since you are viewing it on your pc monitor. Remember, a tv does not support as wide a range as the monitor does. Black to a tv is grey to a monitor.

So all you have done is taken this scale of representable colors and stretched it out. The blackest black (16) is still your blackest black (0), your whitest white (235) is still your whitest white (255) and everything in between is still stored relative your your outer values. When played on a tv, it will look IDENTICAL to your original 16-235 source.

Let me see if I can illustrate it.

[Nothing] 16------------------------------235 [ Nothing]
16(0)------------------------------235(255)

Essentially all you have done is cut off the null data on either side and expanded the actual colors to fill in the gaps. But the point is the colors themselves have stayed the same, and thus you have preserved the 16-235 nature of your source.

Now, when you disabled this option in TMPGenc, you further compressed your luma, as evidenced by the fact that you said it looks, almost identical to the source. If both use the full 0-255 range, and both are being played back on a pc, then both should look identical, as far as the color range is concerned. How much of the color range you lost varies according to your source. Now if you were to play that clip on a tv then your darkest color would not be as dark as it could be. You lost your true blacks and whites.

Once again, if you are going to do any visual tests regarding this setting you MUST view the output on a tv. If it looks bad on a pc then that probably means you encoded it CORRECTLY!

I will say it again. If your source is 0-255 you should disable this option. If your source is 16-235 you should enable this option. It is completely source dependant, neither setting is "better."

[HAGGARD]

Adam,

After reading your answer to my post...

This means that a DV cam compresses luminance values from 0-255 (what the CCD sees) into 16-235 so it complies with the CCIR601 rules.
Since DVD player electronics should comply to the CCIR601 rules, 0-255 values in luminance in the MPEG data should be compressed so that the analog video luminance fed into the tv is 16-235.
This in other words mean that the MPEG data recorded onto the DVD should be expanded to 0-255 values!!!
I'm starting to get it now!!!

CCD (0-255) -> compres luminance to comply with CCIR601 rules (16-235) -> DVD needs full luminance range (0-255) -> DVD playback compresses luminance range to comply with CCIR601 rules (16-235)...

CCD TAPE DVD TV
(0-255)->(16-235)->(0-255)->(16-235)

I guess that you are ABSOLUTLY RIGHT!!!

From now on I will ENABLE the setting in TMPGEnc!!!

Thank you for making me think clear!!!

'HAG

[DJRumpy]

Since DVD player electronics should comply to the CCIR601 rules, 0-255 values in luminance in the MPEG data should be compressed so that the analog video luminance fed into the tv is 16-235.
This in other words mean that the MPEG data recorded onto the DVD should be expanded to 0-255 values!!!
I'm starting to get it now!!!

I'm not sure where you got that idea. Your television will only output a luminance range of 16-235. Anything outside of those values will be chopped off. If your source is using 0-255, then you should by all means compress it to 16-235.

If we followed your example above for a DVD, with a souce using a true 0-255 luminance scale, then anything between 0-15 and 236-255 would simply be discarded (not a good thing). By 'true', I mean a source that has actual luminance values in the 0-15 and 236-255 range.

Remeber when enabling this option for a 16-235 video, your original color values in a 16-235 luminance range, fit into the 0-255 range. Luminance values for 0-15 are all black, and luminance range values for 236-255 are all white. The values from 16-235 are preserved as is. This is desirable. During playback on an NTSC television, 0-15 are ignored (discarded) as well as 236-255.

If, however, your video is using a 0-255 pc luminance range, and you do NOT compress the luminance scale, then all luminance values from 0-15, and 236-255 will simply be discarded.

This is not a difficult concept once you understand what the setting is for.
If your video is using a 0-255, compress the luminance (Unchecked).
If it's using 16-235, avoid compressing the luminance (Checked).

I would be very curious to see if TMPgenc actually expands the luminance range so that 16 becomes 0, or if it simply places the existing luminance range into the 0-255 range 'as is', where 16 remains 16 and all of the 'new' values (0-15, 236-255) are filled with black and white respectively). I believe the second case is correct.

Adam, have you ever verfied with TMPgenc as to which is the case?

[lordsmurf]

I use the histogram function in Adobe After Effects.

I'm very new to AE 5.5. Read the book, saw nothing. Looked at Adobe help, saw nothing. This isn't the easiest program to figure out. Can you point me in the right direction for running the histogram? It will open my MPEG-2's just fine. Beyond that, I'm just clicking like an idiot.

[HAGGARD]

I would be very curious to see if TMPgenc actually expands the luminance range so that 16 becomes 0, or if it simply places the existing luminance range into the 0-255 range 'as is', where 16 remains 16 and all of the 'new' values (0-15, 236-255) are filled with black and white respectively). I believe the second case is correct.

If we followed your example above for a DVD, with a souce using a true 0-255 luminance scale, then anything between 0-15 and 236-255 would simply be discarded (not a good thing).

DJRumpy,

As discribed in my earlier post about this subject, I tested the output of TMPGEnc with the setting checked and unchecked...
My conclusion is that TMPGEnc Really expands the luminance range!!! e.g. colors between the 16 and 235 ranges are also affected...
So for instance a bright color (any color) becomes even brighter and a dark color also darker... So NOTHING is really discarded!!!

I'm not sure where you got that idea. Your television will only output a luminance range of 16-235. Anything outside of those values will be chopped off. If your source is using 0-255, then you should by all means compress it to 16-235.

I don't agree with what you are saying here...
I know (as a technician) that the values below 16 and above 235 are for different purposes for the electronics in a tv. For example horizonal blanking/vertical blanking/chroma-keying/etc.
The video signal (in my case PAL) is constructed by the decoder in the DVD-player.
The laser from the DVD players reads the MPEG2 data from the disc as if it was a normal PC DVD drive. ie. luminance values between 0 and 255. So the video data actually in the MPEG stream holds luminance values between 0 and 255!!! To make it even more clear... The extra video signals like blanking isn't kept in the MPEG stream, but is constructed by the decoder following the rules of your television standard (ie. PAL or NTSC). So you could look at the decoder as a generator, which generates signals needed for a proper videosignal AND a mixer that mixes the video (in this case DVD movie or whatever) into the generated videosignal...
To comply with CCIR601 rules the video (eg. what you see on your television) must be kept between the 16 and 235 values because else it would disturb blanking signals!!! The videodata itself on the disc is recorded with 0 to 255 values and the videodecoder (in my case an ESS VideoDrive Chip) compresses the luminance range into the 16-235 values...
In other terms... a RGB value of 16,16,16 in the analog signal (I don't know how much volts this is at the moment, but it is at about 72.5% from the signal in the carrier) represents real black and 235,235,235 (at about 10-12%) is real white...

pffff, I hope that you now understand what i mean...

Maybe I'll write a guide about video(signals) in a few days so that I can make it even more clear how video works...

'HAG

p.s. Real nice topic, though...

[lordsmurf]

I'm still gonna go with the "flip a quarter" method until we get this straightened out (and I find time to test).

I also need help from adam in finding the histrogram utility in AE.

[adam]

Lordsmurf: Create a new project, import your asset and drop it on the composition box. Like I said before, you will need to run several tests using strategically chosen scenes. You need to basically find scenes that "should" have true black and true white. In other words look for very dark or very light scenes, and advance in your timeline until you reach that frame.

Then hit Effect/adjust/levels. It will show a histogram for just that frame. If the black wave extends all the way to the outer ends of the line or if it rolls off the edges than it is in the 0-255 range. After running several tests, if the black wave never touches the outer edges, then its in the 16-235 range. If you use avisynth than you can actually just let the video play and the histogram will continually update itself. This is nice, but a static one can tell you just as much and its a little easier to interpret.

[adam]

For those people who are utterly confused at this point, here is just a simple little test that will at least show you the difference that we are talking about when we talk of the 0-255 range (pc) and the 16-235 range (tv.)

Load a vob file from a commercial DVD into dvd2avi. I say use a vob because then you can be sure its in the 16-235 range. Click on video/colorspace and set it to RGB 24-bit. Now click on video/YUV->RGB and change it back and forth between pc scale and tv scale. The difference should be clear. You will see that on your monitor, tv scale looks grey and drab. This is actually the CORRECT color level for your tv. That is why this is all so misleading. The only way to get an accurate visual analysis of your source is to view it on the correct display device.

[DJRumpy]

The luminance scale for NTSC television uses 16-235, regardless of what's read off of the disk. The television doesn't care what the DVD player feeds it. It will only display luminance values between 16-235. The rest is discarded. This is a basic fact due to hardware limitations/hardwiring, just as a PC has 255 luminance levels available to it. No matter what you do to your luminance scale, you will only have 16-235 displayed. If you place color values into the 0-15 range, and into the 236-255 range, they will not be displayed. They will be discarded. I think we both agree on this?

It appears that you are assuming the decoder looks at the mpeg stream, and converts the luminance scale to CCIR-601? It doesn't. It would need to scan the entire disc to make such a decision, since this information is not stored in the MPEG header information, like the flags containing aspect or bitrate information. Every encoder I've ever used offers this setting to adjust the luminance level. Do you think they included it to toture you? No, it's included, because an NTSC dvd requires a luminance scale of 16-235 to produce an NTSC compliant video signal. It it didn't require it, all encoders would simply use the 0-255 luminance range, the DVD would assume all discs used the 255 scale, and we wouldn't be having this discussion.

The laser from the DVD players reads the MPEG2 data from the disc as if it was a normal PC DVD drive. ie. luminance values between 0 and 255. So the video data actually in the MPEG stream holds luminance values between 0 and 255!!!

This isn't a surprise. The MPEG will hold whatever luminance values you encoded into the MPEG stream, hence the reason for the setting in the first place. If I encode a DVD with a 255 luminance scale, the video stream that's decodec will have a 255 luminance scale. The same reason the DVD player doesn't know what luminance level your video contains, is also the same reason we as people don't know. Because you would either have to embedd the information in a header that could be quickly read off the DVD like resolution, DAR, bitrate, etc., or the player would have to scan the entire disc, to determine what the lowest and highest luminance levels were according to the darkest and lightest scenes in the video. Neither of these things happen. It simply decodes the video as encoded, and sends the signal to the television.

Are you confusing PAL with NTSC? PAL is not limited to 235, so any disc you look at would probably have a 255 luminance scale. I don't know, as I don't own any PAL discs, but I would asume that's the case.

[lordsmurf]

Thanks, AE is pretty much just like Photoshop on that aspect then.

It appears ATI MMC v8 grabs 0-255, whereas ATI MMC v7 may have grabbed 16-235. Not too sure. But it's looking that way.

What I find interesting was the 7.7 files always looked a bit washed on tv, whereas the 8.1 files of mine had always been fine. This goes both for direct conversion and TMPGEnc re-encodes.

So that leads me to the next item: the DVD player expects 0-255 and converts to analog 16-235 signal? Interesting. Very interesting. I'd like to hear more on this, HAGGARD.

I also have some Panasonic recorder-made disc that play very gray on tv, and harsh gray on a computer. And it's not the source, it was the encode that matted it out.

I also know of a Mac G4 with a Matrox RT and FCP that encodes very dark from the QT MPEG2 encoder. Again, source fine, not sure what causes this as of yet.

More testing to be done...

I'm converting some things in TMPGEnc today... with.... and without... the option changed on both 16-235 and 0-255 sources.

[vhelp]

Hi all..

I'm a little late in the game here, though I've delt w/ these issues as well 8) :P 8)
So, I guess it's my turn at the wheel here

Here's what I have expert experience of (yeah, right) ...

* If your source is a DVD rip, these DO NOT require the box [ ] checked.
Correct me if I'm wrong, but DVD's are YV12. And, isn't YV12 @ 0-255

* If your source is from a DV CAM (home movies/fottage) then your source
...is going to be @ 16-235, DO CHECK [x] Output YUV.., but this is
...an optional step (to be quite honest w/ you) When viewing [x]'ed, the colors
...will look "richer" as in ChachiFace's experience. ..he seemed to have disappeared.

* If your source is DV (DV codec or device ie, ADVC-100) w/in the DV specs, color
...space will follow that of DV specs, but observe also, the diff.'s in your INPUTS ie,
*** - If your DV'ing from say, Satellite or Cable or VHS, these sources are in the
.......0-255 color space. The DV will continue as it should, but you DO NOT want
.......to [ ] Output YUV.. this field then. Leave it unchecked ie, [ ]
*** - If your source is DV CAM footage, then you DO want to [x] Output YUV..
..that is, check it to see if its TOO dark or just right, or else in most cases,
just leave it also, unchecked ie, [ ]

The key here, is the INPUT source. What is the color space of these sources being
feed into the devices ?? ...and in WHICH device ??
That's what has to be checked first. Once that is observed, then you only have
to note the INPUT's color space, and then proceed at your descression on setting:

- INPUT Clr ---- CHCK --- SOURCE -----------------
* 16-235 ....... [ ] .... Cable
* 16-235 ....... [ ] .... VHS
* 16-235 ....... [ ] .... Satellite
* 16-235 ....... [ ] .... DVD (rip)
* 16-235 ....... [ ] .... DVD (Analog capture card)
* 16-235 ....... [ ] .... DVD (DV cap device ie, ADVC-100, PASS-THRO)
* 16-235 ....... [ ] .... MJPEG = YV12 (??)
* 16-235 ....... [X] .... DV CAM (home footage)

Note 1, That TMPG v2.520 states that it's 8-235, and 0-255.
I think there may be some confusion going on w/ 16-235 vs. 0-255 and what
TMPG's saying in the [ ] Output YUV.. field !! May need to be verified.

Note 2, You also have to factor in a "language" barrier here w/ the creator
of TMPG. He go'es on saying..
"Since DV format movie has already recorded in CCIR601,
better result can be expected if this option is enabled"
I believe that this was ment for an user "option" to push the color space
further, if source was 8-235 (or if you want, 16-235) but that you should not
"assume" that its designed to "correct" or that it "HAS TO BE CHECKED" if
your source is 8-235 (or if you want, 16-235) !!

If you [x] Output YUV.. this on all your sources, especiall off those
that are from DVD rips and Analog Capture cards (ie, Cable/Satellite/VHS) you'll
be creating unecessary contrasted areas and NOT TO MENTION, your bitrate will
be reflected (uaually higher than should be) and might requre you to make TWO
cd projects (boing)

So, please be careful of this switch setting. If anything, DV (via home footage)
will not suffer much from using this option. I know, as I use both options,
and am satisfied w/ either one, though leaving it [ ] Output YUV.. seems
to yield more than enough, in my experinece
Other than that, you're all better off leaving box unchecked !!
.
.
But, it's fun to experiment w/ what works and don't works
-vhelp

[adam]

Vhelp, perhaps you are in a PAL region where NTSC safe colors do not apply, but everything you just said is completely backwards. Every single one of those items you listed in the chart would be in the 16-235 range. Yes DVDs do use YV12, which is a YUV format. If you read the links I posted above, you will see that the YUV format does not even support the 0-255 range, and is instead only limited to 16-235.

An NTSC tv can ONLY display colors in the 16-235 range. Obviously DVDs, VHS, and tv broadcasts are not in the 0-255 range because the tv cannot even play that range. Everything beyond 16 or 235 would be hard clipped.

Once again, if taken off the tv (broadcasts) or formatted for the tv (DVD, Vhs) than it will ALWAYS orginate in the 16-235 range. To properly encode this source you should encode in the 0-255 range which means you should enable the "Output YUV data as Basic YCbCr not CCIR601" option.

There are really only two sources which will originate in the 0-255 range. If you record on a non-DV camcorder or if you create footage on your pc (animation, or pc aided effects) then these will use the full 0-255 range. For these sources you need to compress the ranges to CCIR601, and should leave the "Output YUV data as Basic YCbCr not CCIR601" unchecked.

The final consideration is that many codecs will auto stretch material to 0-255. After all, if it doesn't do this then it won't look right on the pc which is where you are going to be doing your editing. The burden is on you do to the research to determine whether you need to compress your ranges back to spec, or leave them as is. Most of this information, regarding what different codecs do, is available on the internet. Otherwise you must run a histogram or use a little trial and error.

So as I said before, the general rule is that almost all souces will already be in the 16-235 range and you should ENABLE the "Output YUV data as Basic YCbCr not CCIR601" in TMPGenc. You must look at your process and determine if you have one of the rare sources that orignates as 0-255, or if your source was converted to 0-255 at any stage before encoding. When this occurs then you should disable the "Output YUV data as Basic YCbCr not CCIR601" option.

[vhelp]

Hi Adam..

Vhelp, perhaps you are in a PAL region where NTSC safe colors do not apply, but everything you just said is completely backwards. Every single one of those items you listed in the chart would be in the 16-235 range.

Na, I'm in NTSC.
.
.
And, I think you are right. I had the feeling that my numbers would be
sort of reversed or something. All these color space number made my eyes
go

Yes, but the DV is what confused me too. Becuase it (in short) is a lighter
color (not color washout though) Just slightly lighter. That's why I said that
it would be ok to [x] check this box for it, IF your source is from a DV CAM
w/ home footage. But, I did not mean to say, if you recorded to miniDV
tapes w/ your cam. This is completely different. Becuase your 16-235 is
being re-encoded to the DV spec, hence the hart to notice, or slightly'ler
lighter color in your source, when you are recording off of Cable or Satellite, but of which would be fixed w/ a color filter in TMPG.. ie Gamma
etc.

Once again, if taken off the tv (broadcasts) or formatted for the tv (DVD, Vhs) than it will ALWAYS orginate in the 16-235 range. To properly encode this source you should encode in the 0-255 range which means you should enable the "Output YUV data as Basic YCbCr not CCIR601" option.

Yes, I quite agree w/ you here too
But, I really think that this should be an optional check. Because, in the
DV footage I've encoded, onced burned to CD and played on DVD player,
your source color space was as good as you would want. But, if you do
check [x] this box, then you'll have slightely richer color (or aka, darker
color) and that may not be as desireable. But, I would say its safe to say
that it's dependant upon the source quality ie, how much light the user,
hollywood produce took the footage of, to make the final determination

I'll try and revise that table
-vhelp

[adam]

I would not say this setting is optional. If its going to be played on an NTSC tv the output needs to be in the 16-235 range, plain and simple. If you don't compress it, you are going to get hard clipping. If you overcompress it you have lost the dynamic range of your colors. Both are bad, and there really isn't anything subjective about it. For every source there is a single proper setting to use to ensure your output is in the correct range.

In regards to this lighter color when using DV passthrough, this is simply because the source is 16-235 and you are viewing it on the pc which uses the 0-255 range. The color range of the source is unchanged, it just LOOKS lighter on the pc monitor. You get the same effect with DVDs using the little dvd2avi test I mentioned earlier.

If you use DV passthrough to record off your television, it retains the luminence range of your source, which will be 16-235. The proper way to encode this is to use the 0-255 range by enabling the "Output YUV data as Basic YCbCr not CCIR601."

Now regarding the richer colors you are getting with DV using this option, this, if anything, is because of oversaturation issues. This is not because the luminence ranges are too wide, indeed the wider the better. This is because the ratio between R, G, and Y is out of proportion. This is not something you should try to correct by arbitrarily clamping your luminence range. This is something that you need to correct using a color filter, and if your luminence ranges are improperly set, then when using such a filter you are basically flying blind.

Again, I really don't think there is anything subjective about this. Wider dynamic range in regards to your luminence is always preferable. If anything, a darker black or a whiter white encodes better than a less than black or less than white color. The only reason to limit the dynamic range of your colors is when you have to, in other words to comply with CCIR601.

[ronnylov]

It's easy to test. Make one encoding with it checked and another one with it unchecked. You will find that the one with the option checked has a more expanded scale, white is whiter and black is blacker. This is a fact because I have tested it myself.

If you leave it unchecked the scale is more compressed, black is less black (greyish) and white is less white.

I used the compare() function in avisynth and loaded my original hyffyuv capture as clip1 and the encoded clip (via dvd2avi and mpeg2dec3) as clip2. I got less difference between the clips when leving the option unchecked so this is what I will do in the future with my analogue captures. The result on TV also looked more natural with the option unchecked.

CCE does not care if you feed it with YUY2 (via avisynth) you can set the option to whatever you like it will be the same as tmpgenc with the option unchecked. Mainconcept 1.4 behaves like TMPGEnc, if you uncheck it the original and the source was the same. Canopus Procoder behaves like TMPGEnc with the option checked, so the whites were too white on my TV! I had to fix it in avisynth with coloryuv("PC->TV") before feeding it to Procoder because there was no option to uncheck in the encoder settings...

But this may be different with different sources (some need to be expanded other don't). Just remember that if you check the option then the range is expanded. If it looks better expanded then use it, otherwise don't.

By the way I have a PAL TV in Sweden. If I check the option in tmpgenc I get too white whites and too black blacks so I don't think this range is for NTSC only. But it may be different with other capture cards maybe. I have an ATI All-In-Wonder Radeon and ATI optimizes for NTSC mainly...

I have read that Canopus DV codec does shrink the source to TV-scale when the avi is opened in RGB mode. So using Canopus DV codec together with Canopus Procoder fixes the problem (shrinked scale by DV codec and expanded to normal again with Procoder). But using Mainconcept DV codec you won't shrink the scale so then you'll get too bright white and too black black if you encode it with Procoder or tmpgenc with the option checked... But using mainconcept mpeg encoder 1.4 and option unchecked will work with mainconcept dv codec.

I suggest that you all test both options and use whatever looks best. If you want more contrast then check it, otherwise leave it unchecked.

[HAGGARD]

I finally know the answer to this question!!!!

I tested it the other way around...

I ripped a few different movies and checked the luminance values...
E.g. opened some clips into vdubmod and exported a few dark and bright scenes as .bmp then opened these .bmp's in Paint Shop Pro and went over the bitmaps with the colorpicker)
In ALL movies I've checked luminance values are between 0-255!!!!
So it seems that DVD movie's don't follow the CCIR601 rules.
In this case you should leave the 'Output YUV data as Basic YCbCr not CCIR601' setting UNCHECKED!!!

I also checked output of my DV camera (JVC GR-DVL140) and the luminance values are also between 0-255, thus is TMPGEnc wrong with the statement that DV-cams always output following the CCIR601 rules!!!

'HAG

[adam]

Haggard, once again, PAL tv's are not limited to CCIR601 specifications. PAL DVDs use a slightly different format than NTSC DVDs and its possible that they are stored in the 0-255 range. In this case, yes you would want to leave the "Output YUV ..." option unchecked. My experience with PAL DVDs is extremely limited, so I will just leave it at that but I can assure you that all NTSC DVDs are stored in the 16-235 range, and it is NTSC sources that we are really talking about here. If you are in a PAL region than you do not need to concern yourself with CCIR601. This is a stupid NTSC problem.

Regarding DV sources, I believe I already explained this several times. DV camcorders film in 16-235 range but many DV codecs auto expand this to 0-255. You are simply using one of these codecs when you capture to your hard drive.

[ronnylov]

It's not a NTSC problem only. The settings affect PAL encoding too and there are more countries in the world using PAL than NTSC so why should we be talking about NTSC problems mainly?

The stupid thing is that most software does not know if I'm encoding PAL or NTSC. The settings must be correct for each standard and if I select an option because a NTSC guru says so I may destroy my PAL encodings...

Anyway it seems correct that in most cases the setting should not be enabled when I encode to PAL DVD. I don't know about NTSC DVD's, I have never seen one in real life...

[adam]

edited.

[nimco]

I did some experiments.
I was using NTSC DV avi type –2 file captured from miniDV tape. I have a MainConcept codec on my PC, luma was set to 0-255.
My script was:

AviSource("test.avi")
ColorYUV(opt="coring")
ColorYUV(analyze=true)
ConvertToYV12()
Histogram()

It showed that is nothing below 16 and above 235 in this clip, sure I was using opt=”coring” .

I modified an avs above (just to make sure that ConvertTo and Histogram do not change the luma level):

AviSource("test.avi")
ColorYUV(opt="coring")

and encoded it in CCE with the luma setting at 0-255. Same result was at the 16-235 setting. Adam, you are 100% right, when the source is YUY2 it is really does not matter.

I’v got a mpg file.
I opened it in WinDVD and did a snapshot, saved it as bmp file. I checked its RGB levels in EasyPeg (photo editor). They were in the 16-235 range.
Same result if take the mpg above and run a script:

DirectShowSource("test.mpg")
ColorYUV(analyze=true)
ConvertToYV12()
Histogram()

I see that the luminance is in the 16-235 range.

The next step was to see a histogram in Adobe After Effects (I was using a demo).
I proceeded as Adam was suggested. Created a new project, imported my assets, dropped it on the composition box.
Then checked a histogram in Effect/adjust/levels. It showed a range 0-255, very clearly. Also I checked the RGB in the Info window. They also were in the range 0-255.

I think it happened because the After Effects converts mpg to RGB and expands the luminance to 0-255.

Also after authoring and burning mpg to DVDR the video was looking on TV just perfect, correct luma range and correct colors.

It seems to me it is not so simple to do a luminance level testing.
To capture still image from VirtualDub screen might not be so accurate thing also. VD requires RGB and converts it to 0-255 range, correct me if I am wrong.

Any ideas what did I wrong and what is the right way to check the resulting MPG file luminance?

[DJRumpy]

nimco, your like a breath of fresh air

Adam, I gave up 10 posts ago. I must be a glutton for punishment...

Haggard, your experienced enough with this to know that there is never a single perfect setting that applies to all video. Did you ever consider that your tests are skewed, since every movie you examined, even those that are known to use 16-235 (DV) show an expanded luminance range? Perhaps Paint Shop also converts your input to RGB (0-255).

Refer here for info on DV: http://www.sonyusadvcam.com/content/article_39.shtml

Please note, that although it utilizes a 0-255 luminance range, 0 and 255 are reserved, and true black is started at 16, and white is considered to use 235. DV is designed to output to 601 specs. This link also has some good examples of what happens when you 'compress' a luminance range that is already 16-235.

DV cameras capture images using a range of 0 to 255 to define luminance (i.e., black-and-white information) while traditional video standards (ITU-R 601) define luminance using a range of 16 to 235. DV codecs, however, scale the 0-255 range to 16-235 when the images are recompressed.

[HAGGARD]

Haggard, your experienced enough with this to know that there is never a single perfect setting that applies to all video. Did you ever consider that your tests are skewed, since every movie you examined, even those that are known to use 16-235 (DV) show an expanded luminance range? Perhaps Paint Shop also converts your input to RGB (0-255).

DJRumpy,

Sorry to bother you again, but I maybe forgot to mention that I also tested that possibility!!! I encoded a piece of video with the setting checked (eg. 16-235 range)!!! Burned this to DVD, decoded with VDubMod and checked luminance range with PSP colorpicker.
AND VALUES ARE BETWEEN 16-235!!!.
So your theory that PSP or whatever software expands luminance range into 0-255 isn't right!!! (maybe with DV, but that doesn't matter anymore since original movies are encoded in the REAL 0-255 luminance range...

'HAG

p.s. I have based this all on the tests that I have done not on my own opinion...

The original


With Setting disabled


With Setting enabled


Save these 3 images and compare them with each other....!!! (use colorpicker in PSP)

LOOK AT THE BOTTOM RIGHT CORNER WHERE THE GREY BECOMES TOTALLY BLACK WHEN SETTING IS ENABLED!!!

With setting disabled, picture luminance values are (almost) same as original. Values below 16 are still there...

With setting enabled, white is expanded from 235 to almost 255 and the values below 16 are just cut off (so you're losing information)!!!

Don't forget that only the last 2 pictures ran thru encoding/decoding process!!!

So if DJRumpy was right then all pictures should have been (at least almost) the same if ran thru software (VDubMod or PSP) that automatically expands luminance values, but they aren't!!!

p.s.2. These pictures are converted from .bmp to .jpg, but this conversion didn't influenced color or luminance values... I also tested that...LOL

Is this proof or what?!?

Again, you're right about that there is never a single perfect setting that applies to all video... But with testing you know for sure what IS the right setting. (besides, it only takes a few minutes, so why not test before encoding?!)

'HAG

p.s. edited this one 3000 times, but this is really the last edit...

[ronnylov]

Haggard- You live in the Netherlands and I live in Sweden so we use PAL both of us and we came to the same conclusion. But NTSC could be different as mentioned earlier in this thread. I don't know.

Somebody that works with NTSC could also make a test. I would like to know if there is a difference or not because I may need to make a NTSC DVD perhaps some day in the future. Maybe this is automatically taken care of when we change the PAL/NTSC setting in TMPGEnc? Remember to check the results on TV also because if it looks fine on PC it may not on TV because of the two different scales.

What is the reason to use a PC scale and a TV scale? Why can't the DVD-player just adjust black and white levels to apropriate values for each standard (PAL/NTSC) and playback device (PC/TV) during playback but always use the 0-255 scale on the disc? You'll loose dynamics if you don't use the complete byte range.

[nimco]

Haggard, just a thought:

On the Original jpg, the lowest right side quadrate has luma = 16,
And next to the left side quadrate has inside three tiny strips. The most left of them has luma = 7.
At least my photo editor shows that.
With the setting disabled in TMPGEnc you compress luma even further (according to Adam), to above 16 and below 235.
You open your mpg and convert to GRB. Converting to RGB expands luma to your previous 16-235 by following ratio "16-235 to 0-255".
( If your mpg would be 16-235, it could be expanded to 0-255, but it was compressed even further, hence it will be expanded by following the same ratio back to 16-235). And it is on jpg With Setting Disabled.
With the setting enabled you preserve your original luma at 16-235, after converting to RGB it was expanded by following the same ratio. The lowest right side quadrate has obtained luma =0 (it was 16 before), and I see it on jpg "With Setting enabled". But on jpg The Original, on the next to the left quadrate, the most left tiny strip inside had luma = 7. I was mentioned about it above. This one off course is missing, luma wants to be expanded by the same ratio, it means below 0.
The idea is that you call your Original as 16-235 range, but actually it has some even more black areas, as the tiny strip inside, luma=7. And it will be missing.
Correct me if I am wrong. I am her to learn to do conversions properly.

Sorry for my confusing English.

[HAGGARD]

What is the reason to use a PC scale and a TV scale?

Ronnylov,
The reason that there is a PC and TV scale is very simple... a TV needs a few more signals in the whole videosignal to work properly. The signals are commonly known as blanking. This blanking signal is needed to surpress the electron beam in the CRT (Cathode Ray Tube) when it goes from the right side of the CRT to the left after it has written one line on the CRT. Black on a CRT is not really black, but we see it as black. In order to prevent seeing the electron beam jump from right to left, the beam needs to be ULTRAblack. See example (white line with black part in the middle)

Why can't the DVD-player just adjust black and white levels to apropriate values for each standard (PAL/NTSC) and playback device (PC/TV) during playback but always use the 0-255 scale on the disc? You'll loose dynamics if you don't use the complete byte range.

DVD-discs DO use 0-255 luminance range!!! Both PAL and NTSC!!! It's just the decoder that needs to comply with the local television standard!!!
My DVD-player for instance (Cheap H&B DVD-5415S with ESS VideoDrive decoder) has no problem with NTSC disc's... It will read the disc's flawlessly and also play them flawlessly (besides a little jerkyness due to other framerate).

With the setting disabled in TMPGEnc you compress luma even further (according to Adam), to above 16 and below 235.

Nimco,
With the setting DISABLED, the output of the encoder is as-is!!! So almost identical to the input video (A little difference due to compression)...
With the setting ENABLED, The encoder tries to expand the luminance values between 0-255. So the encoder asumes that the values of the input video are between 16-235!!! Values below 16 or above 235 will be cut off (as remarked by you)...

So the setting should be used like this...
If input video has 0-255 range then disable this setting
if input video has 16-235 range then enable this setting.

So you'll have to check which luminance values your input video uses...

'HAG

[lordsmurf]

Most of us are visual learners.

More images as posted above would probably clear up these disagreements once and for all. If I knew where to get testing images on this topic, I'd do it myself.

[ronnylov]

Thanks for the explanation HAGGARD, it makes sense!

So if the source material is in 16-235 scale then you should enable the setting to expand it to DVD range 0-255, otherwise leave it unchecked to .

Now I wonder what is the reason that some source formats use the 16-235 range instead of full 0-255 range? For instance Canopus DV codec compresses the range to 16-235 so this setting should be enabled in tmpgenc to expand back to full range when creating DVD.