VideoHelp Forum
+ Reply to Thread
Results 1 to 19 of 19
Thread
  1. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    Hi,

    I am using TMPGEnc to convert DV-AVI to VideoCD. On DV-AVI, the YUV data is already in the CCIR601 format, so there is no need to convert it again (in the box MPEG Settings/Quantize Matrix/Special Settings/Output YUV...). I am checking on this box, assuming that the YUV data remains unaltered. I assume that every conversion to CCIR601 loses some data (Y is squeezed from 0-255 to 8-235).

    My observation is exactly the opposite: the "conversion to CCIR601" preserves the data, while the "conversion to YCbCr" alters it (maybe Y is expanded from 8-235 to 0-255).

    I noticed this by repeatedly compressing a MPEG file. If I use "output as CCIR601" repeatedly, the pixels' intensity is preserved. If I use "output as YCbCr" repeatedly, the intensity is expanded to "too much white" or "too much black".

    Is this the right behavior? If yes, then TMPGEnc's recommendation written in the yellow tooltip box is wrong (and my first assumptions are also wrong). If not, then I think there's a bug in this conversion.

    I am using TMPGEnc version 2.59.47.155. I suspect the latest version suffers from the same problem, because I didn't see anything mentioned in the latest change log.
    Cosmin
    Quote Quote  
  2. Member adam's Avatar
    Join Date
    Sep 2000
    Location
    United States
    Search Comp PM
    First off, realize that your color levels are being altered regardless of what setting you use. Your source is YUV, and TMPGenc uses VFAPI to access it. VFAPI only supports RGB input, so your file is being converted from YUV to RGB and then back to YUV. There is slight quality loss in these conversions so your encoded output may never fully preserve the color levels of your source.

    Secondly, make sure you are analyzing your output on the proper playback device. PC's use the full 0-255 range, only tv's are limited to 8-235 or 16-235 (depends on region.) If you want to test whether your luminence ranges have been treated properly than you MUST play the source on a tv. You cannot go by what you see on the pc monitor.

    Finally, here is the big kicker. DV is shot in the 16-235 range and already complies with CCIR601, like you said. But what you probably don't realize is that your ranges can be expanded to 0-255 by either your DV codec or your editing software. So really, when deciding whether to use the "output data as YCbCr" all bets are off until you truly determine the luminence range of your source. What DV codec are you using, if you use one, and what editing software do you use? Sometimes the only way to determine what is being done to your source is to do a histogram.

    If your source already complies with CCIR601 than you want to stretch the luminence range out to 0-255 by using the "output data as YCbCr" option. This doesn't affect the luminence ranges, but just prevents them from being further compressed. Are you saying that with this option checked, repeated conversions result in loss of the dynamic range of your colors? Basically, white become less bright and blacks become less dark? This shouldn't happen, and if it is happening than perhaps there is a bug.

    If your source uses the full 0-255 range ie: pc authored material like animation, analog film sources, or anything which has been stretched out to 0-255, than you want to have TMPGenc "output data as CCIR601." It will compress the outer ranges of your luminence scale so that these colors do not get clipped during playback. Regardless of what range your source originates at, repeated conversions of the same source with this option set will result in the repeated diminishing of your dynamic color range. Blacks will begin to look more like grey, and whites will just become duller. Its usually very easy to determine when this has happened with DV footage; anything shot in low light will just appear totally black on the tv. You are saying that when using this setting, after repeated conversions the luminence ranges appear to be unaltered?

    Here is a little tip that might help you in determining whether your luminence ranges are being repeatedly compressed or not. You will need to use a letterboxedUse a letterboxed source, so either use something othe and use the clip frame filter to add a black mask only over half of your letterboxing. The black mask represents "true black" so after repeated conversions your remaining letterboxing, the part not covered by the mask, will appear grey next to the mask. Just make sure and add the mask to your encode everytime you re-encode the source. The mask will always be "true black" so use this as your point of reference.

    If there is a bug here I haven't picked up on it yet. So far this setting has worked as expected for me.
    Quote Quote  
  3. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    Originally Posted by adam
    Secondly, make sure you are analyzing your output on the proper playback device. PC's use the full 0-255 range, only tv's are limited to 8-235 or 16-235 (depends on region.) If you want to test whether your luminence ranges have been treated properly than you MUST play the source on a tv. You cannot go by what you see on the pc monitor.
    Understood. I thought it's enough to analyze the colors on the PC. If they look differently on the PC, it means they are altered. I am using VirtualDub, and the Panasonic DV codec (decoder-only).

    But there is something else that I did: I encoded a MPEG repeatedly. I made it CCIR601 the first time, to be sure that I start with a correct source. Then I re-encoded it to again to MPEG, with and without the CCIR601 option off, using TMPGEnc and nothing else. This way I eliminated the difference between the MPEG decoder, and the DV decoder.

    My complain remains: the MPEG reencoded with CCIR601 looks almost exactly the same as the original MPEG, while the other one has both the white and the black expanded. The colors were clearly altered in the second case, and I concluded that I didn't need an experiment with a TV.

    Originally Posted by adam
    Finally, here is the big kicker. DV is shot in the 16-235 range and already complies with CCIR601, like you said. But what you probably don't realize is that your ranges can be expanded to 0-255 by either your DV codec or your editing software.
    I understand. Since the same thing happens with MPEG only, I conclude that either my MPEG decoder (Windows 2000) is expanding Y from 16-235 to 0-255, or TMPGEnc is indeed buggy. I have DivX-5.02 installed, but I think it shouldn't be responsible for this kind of alteration.

    Originally Posted by adam
    If your source already complies with CCIR601 than you want to stretch the luminence range out to 0-255 by using the "output data as YCbCr" option.
    Or maybe I'm just wrong!

    I thought that the conversion to CCIR601 ("option unchecked") shrinks the dynamics of the colors, to ensure the 8/16-235 range, while the conversion to YCbCr ("option checked") leaves the color intensity "as-is".

    What happens is that the "option unchecked"/CCIR601 leaves the color "as-is", while the "option checked"/YCbCr expands the color - this is why I get "too much white" and "too much black".

    Originally Posted by adam
    Here is a little tip that might help you in determining whether your luminence ranges are being repeatedly compressed or not (...)
    Thanks, Adam, for the tip. I'll try it in a few days. If you wish, I'll let you know what happens.
    Cosmin
    Quote Quote  
  4. Hi Cosmin,
    I've wondered about this setting for a long time and I've also made some experiments... but I'm unable to conclude for sure. What I believe is that checking leaves the sorce as-is and unchecking compresses it to 16-235.

    Did you try adam's suggestion? What did you conclude?

    Thanks!
    Quote Quote  
  5. I have been perplexed by this setting for quite some time. After doing considerable reading Re DV luminance ranges, NTSC "setup", and an explanation that North American DVD players (generally) include a builtin setup provision. I concluded that I should select to output YCbCR.

    However, recently I tried some test with colorbars - NTSC with black levels at 16-16-16 and superblack at 7-7-7 , and DV colorbars with black at 7-7-7 and superblack at 0-0-0. I converted with TMPGenc checking to output as basic YCbCR and leaving unchecked. I didn't make multiple generational encodes - just one each. I am using the Sony DV codec.

    What I discovered is that by leaving the output as basic YCbCR UNchecked, the resulting Mpeg files were kept quite close to the original - all colors. However, when selecting this option, the levels were changed considerably. Now all levels at 16-16-16 and below have gone to 0-0-0. Additionally, white changed from 235-235-235 to 253-253-253 and the other colors moved up in luma approximately 5%.

    Lately I have been using filters, such as "levels" in VDub and extend 0-235 in Tmpgenc to try and get a full DV range and then leave the "output as basic YCbCr " unchecked.

    I am still working on it ....
    Quote Quote  
  6. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    Hello, Berru and Andie,

    Yes, like you, I thought that checking the box leaves the source as-is, and unchecking it shrinks the luminance levels to 16-235. If my source is already in 16-235, then should not shrink it again.

    False!

    Andie's observations seem to be consistent with mine. The bottom line is, the advice that TMPGEnc gives in the yellow tip box "Since DV format movie has already recorded as CCIR601, better results can be expected if this option is enabled" is misleading!

    I suspect the problem originates in the expansion of the luminance levels from 16-235 to 0-255 by the MPEG or DV decoder, which TMPGEnc has to re-shrink again to 16-235. The real cause is that TMPGEnc only accepts input from RGB source, which is, I think, one of its biggest limitations.

    The real experiment should be carried on by testing MPEG compression from true RGB sources (as uncompressed AVI, for example) with 16-16-16 and 7-7-7 levels. If there is color alteration when encoding in YUV, then we should get MPEG/DV decoders which do not alter the source, and TMPGEnc is not lying - it's just mislead by the decoder that it uses. But if there is color alteration when encoding from true RGB into YCbBr, then it's definitely a bug in TMPGEnc.

    I will probably try this myself, but I won't do it soon, because I am busy right now. I will post the results here as soon as I have them. But if you guys want to try it out, I would be glad to hear about your results.

    Best regards
    Cosmin
    Quote Quote  
  7. I think cosmin's comment Re TMPGenc being misleading is valid. It seems to me that by selecting to have TMPGenc output YUV as YCbCr, TMPGenc attempts to ensure that there is a full range of luma - 0 to 255, and uses it's own code to expand out the values. Of course, the effectiveness of this depends to a large extent on the source and the luma already present.

    As I mentioned in a previous post, I have read that most North American DVD standalone players have built-in "setup", that kicks up the luma floor, and both of my players appear to support this. So if I encode my video with black at 16-16-16, the DVD player is going to kick it up so blacks get muddy. So what I am now trying, is to use various filters in other programs, to get my luma values, particularly blacks, down in the 7-7-7 to 0-0-0 range before encoding and then leave the YUV box UNchecked in TMPGenc.

    As cosmin suggested the need to use RGB sources, the colorbars I encoded began in Photoshop as RGB ("pure" RGB colors, and when the "outout YUV..." box was selected, TMPGenc did some not very nice things to the values, but when left UNchecked, the luma and the chroma,were left mostly untouched.
    Quote Quote  
  8. Member adam's Avatar
    Join Date
    Sep 2000
    Location
    United States
    Search Comp PM
    cosmin some DV codecs expand the 16-235 source to 0-255 and some don't. You have to know what your codec does and make decisions regarding this setting in TMPGenc accordingly. There is a good sticky thread in the DV forum on doom9.net which lists which range the various DV codecs operate in. From my experience best results are achieved with 16-235 sources when processed with the "Output YUV ..." CHECKED. And vice versa with 0-255 sources (ex: Canopus DV codec which expands to 0-255.)

    I've solved this issue once and for all by switching to Procoder for DV footage. Hands down the best encoder for DV footage in my opinion.
    Quote Quote  
  9. I also liked Procoder very much, when I tried it,; however, I couldn't find a way to frameserve to Procoder, and gave up on it. As I recall, it does have filters to expand and restrict luma ranges, but the default seemed to work pretty well.

    With the "output YUV data as Basic YCbCr not CCIR601" selected in TMPGenc, it certainly appears that TMPGenc is attempting to expand the luma to 0-255. I want to summarize my observations and try to be clear, because it can be confusing.

    I have taken pure RGB colorbars - 16-16-16 to 235-235-235 and values of 0-0-0 to 235-235-235. in Photoshop and saved in png format. I have then created avi's as uncompressed RGB and DV, with the Sony DV codec. I have taken these avi's into VDub and saved a still and taken this still back to Photoshop to check it's condition-the uncompressed is still right on, as is the DV, for the most part - very slight variance. I have then taken these avi's into TMPGenc and produced Mpegs - encoding each avi with the YUV selected and unselected. I then take each Mpeg into VDub and capture a frame and back into Photoshop and check RGB values.

    When YUV is left unchecked, the luma values are left virtually intact (as Adam mentioned, there are going to be slight differences - but they are very slight), but when YUV is selected, the luma values are blown up - both from the uncompressed avi and the DV avi - values of 24-24-24 and lower are all converted to 0-0-0 , there are no grays. And white (235-235-235) is now close to 255-255-255. All the other ranges are similarly distorted.

    I did very brief tests of the Mainconcept encoder, with similar observations, but my tests were brief.

    So the bottom line, for me, is to get my source video the way I want it, with filters, VDub, Avisynth, etc. and then leave the "output YUV ... box unchecked. Don't let TMPGenc mess with the luma values. And keep in mind, when adjusting the values, that the DVD player NTSC North America) is going to increase the black level- from 0 IRE to 7.5 IRE, if I understand this correctly, and 7.5 IRE is approximately 16-16-16 on the RGB scale - right???
    Quote Quote  
  10. A new wrinkle. I decided to see what the Mainconcept encoder did to the luma values. I examined the values as per my explanation above.

    As expected, when selecting the option in advanced video settings, "input video is RGB 16-235", the encoder did as TMPGenc with the "output YUV.." setting - expand values out to 0-255, and in so doing making all levels 24-24-24 and below 0-0-0, and significantly altering all ranges and colors.

    What really surprised me, was when encoding normally, without the RGB box selected, the colors were significantly altered. The upper and lower levels were maintained - 24-24-24 was still the same as was 7-7-7 and 235-235-235. However, the ranges inbetween were altered and thus, the colors. For example:

    Original RGB Red 180-16-16
    TMPGenc Red 180-16-16
    MC Red 194-32-15

    Original RGB Magenta 180-16-180
    TMPGenc Magenta 180-16-180
    MC Magenta 191-42-185

    Original RGB Green 16-180-16
    TMPGenc Green 16-180-16
    MC Green 4-154-10

    etc., etc.

    I was, for the most part, using the defaults with Mainconcept, so perhaps there is a setting that needs to be changed, but I don't know what it would be. Any ideas?
    Quote Quote  
  11. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    After I read about your experiments, there is one more thing that bothers me: what if the MPEG decoder, installed in the system, expands the 16-235 luma into 0-255, for computer viewers?

    If I am correct, then it's not TMPGEnc's fault, after all; it's just that we are mislead by the MPEG decoder, and think that it's TMPGEnc who does the expansion -- when, in fact, it's the decoder.

    I know that VirtualDub uses its own built-in MPEG decoder, and I had a quick look at the VD source code, to see if it does this expansion or not, but I couldn't draw any conclusion. I did notice, though, that the VD capture does the 0-255 --> 16-235 shrinking.

    BTW thanks, Adam, for the tip about the discussion on doom9.org. So it seems that all DV decoders mentioned in there do perform the expansion. And I wonder, what if it's the same with the MPEG decoders? Including VirtualDub?
    Cosmin
    Quote Quote  
  12. If I am correct, then it's not TMPGEnc's fault, after all; it's just that we are mislead by the MPEG decoder, and think that it's TMPGEnc who does the expansion -- when, in fact, it's the decoder.
    Interesting thought, but I don't think it is a decoder issue. If the luma levels are encoded by TMPGenc (no output YUV...) at 16-235, and then loaded in VDub and then a frame extracted, the luma levels are still 16-235. It is only when the "output YUV ..." option is selected that the luma gets blown up. Also, as indicated above, Mainconcept seems to have a problem even without trying to expand out to 0-255.
    Quote Quote  
  13. Member
    Join Date
    Dec 2003
    Location
    United States
    Search Comp PM
    Back to the TMPGenc issue of having to convert YUV to RGB, I have a related question: When I use ULEAD to create an uncompressed AVI from an analog source, I have the option of saving it as YUY1 or RGB. The default is YUY1 and from the sites I've checked, selecting RGB does little for quality and only increases file size.

    If I'm using TMPGenc to convert this to MPG, should I save the AVI file in ULEAD as RGB to avoid any conversion, thus quality loss? Thanks.
    Quote Quote  
  14. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    dwiesel,

    The bad news is that you cannot avoid the conversion. The capture is done in YUV space, and TMPGEnc accepts video only in the RGB space. If the tool says it captures "directly in RGB", it means that it converts YUV to RGB right in the moment of capture. On the other hand, if you don't do the conversion to RGB in Ulead, then TMPGEnc will use the YUY2 codec installed in your system, to do the conversion to RGB.

    I would advise you to do the capture in YUY2, because it takes less space and it's faster => the chance of dropping frames is lower. You don't win anything if you capture in RGB.

    The ideal solution is to capture in YUV (e.g. YUY2), and to have an MPEG encoder that recognizes YUV directly. CinemaCraft Encoder does that, but, unfortunately, I don't have it. I'm still waiting to see that implemented in TMPGEnc.

    By the way, I have just downloaded the TMPGEnc DVD Source Creator. It says it accepts DV directly, and uncompressed AVI, but I don't know if it considers YUY2, I420 or YV12, as "uncompressed AVI"...

    Best regards
    Cosmin
    Quote Quote  
  15. Member
    Join Date
    Dec 2003
    Location
    United States
    Search Comp PM
    Many thanks Cosmin!
    Quote Quote  
  16. Member
    Join Date
    Dec 2003
    Location
    United States
    Search Comp PM
    Seems I have come full circle. I'm currently trying to understand the YUV2/YUY2/RGB24 conversion issues when editing one AVI and saving to another AVI (which I need to find the right forum for)…but it does make me revisit this thread:

    Tell me if I've got this right: the nature of capturing --regardless of the card or software-- is always done in the YUV realm. If this is true, is there ANY benefit to capturing in RGB24? And, is this choice independent of what software you choose to render your MPG?

    I now use Pinnacle and I have been capturing in YUV2 (it has no RGB24 option). From what you're saying, is this the most desirable method (in the uncompressed realm) since there's no colorspace conversion going on?

    I have no issues with available hard drive space/usage, and just want to know the best method that will yield the highest quality capture and final MPG (which I assume involves colorspace choices). I currently capture in YUV2 and create my MPG in TMPGenc (and we'll assume my TMPGenc settings are optimal). Yes, I've tested and done comparisons, but I'm bug-eyed and just want to know the facts as reassurance.

    MUCH appreciation to anyone who can enlighten me.

    Thanks,
    Dave
    Quote Quote  
  17. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    Originally Posted by dwiesel
    Tell me if I've got this right: the nature of capturing --regardless of the card or software-- is always done in the YUV realm.
    True.

    Originally Posted by dwiesel
    If this is true, is there ANY benefit to capturing in RGB24?
    There is no difference in quality between capturing directly in RGB24, and capturing in YUV and post-converting in RGB - but...

    Originally Posted by dwiesel
    And, is this choice independent of what software you choose to render your MPG?
    There are many ways to convert between RGB and YUV, and different codecs use different ways: integer arithmetic (8-bit precision, 16-bit precision), floating-point arithmetic, binary-coded-decimal... The output is not bit by bit identical, but it is visually identical. Mathematically, there is a difference between these methods, but it is so tiny compared to the MPEG quantization, that it is like an extra water drop in a bucket full of water.

    Furthermore, some MPEG encoders accept directly input in YUV form; I know that Cinemacraft does it, I know that TMPGEnc does NOT do it; I don't know about the others. You should find this info in the encoder's manual.

    Originally Posted by dwiesel
    I now use Pinnacle and I have been capturing in YUV2 (it has no RGB24 option). From what you're saying, is this the most desirable method (in the uncompressed realm) since there's no colorspace conversion going on?
    It's the most desirable if the MPEG encoder accepts YUY2 (I think you meant YUY2, not YUV2). It's still desirable if the encoder accepts RGB only, because it's preferable to do the YUV->RGB conversion after capturing (you don't have to do it explicitly; the codec installed in the system will take care of that). It lowers the chances to drop frames during capture.

    Best regards
    Cosmin
    Quote Quote  
  18. Member
    Join Date
    Nov 2002
    Location
    Toronto
    Search Comp PM
    Originally Posted by cosmin
    Originally Posted by dwiesel
    Tell me if I've got this right: the nature of capturing --regardless of the card or software-- is always done in the YUV realm.
    True.
    I think it worths mentioning that the TV signal is in YUV. Some fancy capture card may convert automatically YUV to RGB (I don't know if such card exists...) but I wouldn't want to use that, because I prefer to do the conversion in the digital domain, rather than relying on analog electronic circuits.
    Cosmin
    Quote Quote  
  19. Member
    Join Date
    Dec 2003
    Location
    United States
    Search Comp PM
    Many thanks Cosmin...and yup, it is YUV2 (this is what Pinnacle calls it anyway).
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!