MJPEG to MPG2 color issue

[boriskey]

Guys, I am going to kill myself. Please help me!!

I have been trying for 3 days to figure out why my original file (MJPEG) is bright and has crisp colors while after conversion it loses colors and not so appealing as the original one.

When I had my old digital camera (which shot MPG) I did not have such issues. But now I have Canon G9 which shots video to MJPEG and the first clip I tried to render came out much worse than original video

I uploaded 2 clips. The first one basically original video and second one is compressed to MPG2

http://www.devcomzone.com/temp/AsFirstVideoClip.avi
http://www.devcomzone.com/temp/MPEG2.mpg

I played with thousands options including GAP structure (6 I-frames), different tools and codecs, but result is always the same. Just look at original colors of grass and yellow car and colors after compression

I would appreciate any help if you can guide me how to get MPG2 file after converstion with original crisp and bright colors

[JohnnyMalaria]

There are two reasons.

Both formats store the information as luminance (Y) and chroma (Cb and Cr) rather than red, green and blue. Each value is an 8-bit number that can range from 0 to 255. MPEG2 (and other formats like DV) require that the Y, Cb and Cr values are clamped between ~16 and ~235 (the range is slightly different for the Y and Cb/Cr values). For MJPEG, that restriction doesn't apply - each frame is nothing more than a JPEG image.

This means that the MJPEG image has a wider range of values to use. A wider range means greater contrast and whiter whites/black blacks. Hence, the image looks much better than the constrained MPEG2 one.

If you have a program like Paint Shop Pro, you can grab an image from each video and look at the histogram of values (that's what I did with your example) and it clearly shows the cut-off at ~16 and ~235 for the MPEG2.
If you "stretch" the histogram, the MPEG2 image looks much more like the MJPEG one but not as sharp - which leads to the second reason...

MPEG2 throws away of lot of color information - for every 4 pixels of black and white, you only get 1 pixel of color (a crude approximation).

MJPEG can store the image without throwing away any color information, so the image can look more vibrant.

In short, you are seeing the limits of the MPEG2 standard.

[boriskey]

In short, you are seeing the limits of the MPEG2 standard.

Hi John,

thank you very much for the explanation, it does make sense. Now since my camera shots in MJPEG only and I want to do some post-processing and burn a DVD, there is no way I can get original colors even if I use better/expensive codecs? So to say, I should get over it and live with it, right?

Is there any other formats for DVD (I doubt so), that can help with my case?

[manono]

Hi-

That's strange, as I found the opposite of JohnnyMalaria. I found the MJPEG AVI to be clamped to 16-235 and the MPEG-2 to use the full range of 0-255. That could still explain the problem, although I suspect other possible causes as well for the slight desaturation of the MPG as compared to the AVI.

MPEG2 (and other formats like DV) require that the Y, Cb and Cr values are clamped between ~16 and ~235 (the range is slightly different for the Y and Cb/Cr values).

At the risk of sounding too blunt, that statement is simply untrue, for MPEG-2 anyway. Every DVD I've ever seen - all the properly mastered ones, anyway - use the full 0-255. There is no such requirement.

[jagabo]

MJPEG uses YUV 4:1:1, 4:2:2 or 4:4:4 sampling. The luma range can be from 0 to 255. This can be converted to MPEG with the same luma range. But you risk black and white crush on playback.

AVISource("AsFirstVideoClip.avi")
VideoScope()
Info()



Use an MJPEG decoder that supports YUY2 or YV12 output. Don't convert to RGB.

PicVideo MJPEG config dialog:



Note the "1/1/1" means "4:4:4".

The OP's conversion process has compressed the luma range from 0-255 to 16-235:



Converted to MPEG with luminance range intact:

intact.mpv

[boriskey]

Converted to MPEG with luminance range intact:
intact.mpv

Jagabo,
Wow, It seems I am saved I love the clip made by you.

As I understand I should

1) install PicVideo codec and change setting like on your screenshot
2) convert from MJPEG to MPG2 using any conversion tool and I can use pretty much default setting of MPG2 encoder

So to say, all I need just to use right MJPEG codec with advanced colorspace options, am I reading this right?

And I am confused, why these options located on compressor tab of the codec's settings, not decompressor's. How it can be that compressor settings (encode) of the source video (MJPEG) affects final video (MPG)?

[jagabo]

PicVIdeo uses the same dialog for compression and decompression settings. I recommend you use the Force YUY2 Output option even though I didn't show it checked in my screeen cap.

I believe ffdshow's MJPEG encoder can also output YUY2 or YV12. You may have to force it in the output options.

You didn't say which MPEG encoder you are using. Some will clamp Y values below 16 and above 235 (everything below 16 becomes 16, everything above 235 becomes 235) losing detail in dark and light areas. Others will compress the range so that 0-255 becomes 16-235 (ie, 0 becomes 16, 1 becomes ~17, 2 becomes ~18, etc.). There will often be settings that let you specify these options.

At least one MPEG encoder I know, TMPGEnc, will convert any incoming YUV source to RGB before converting back to YUV for MPEG compression. In the process it will lose all details in the blacker-than-black (luma < 16) and whiter-than-white areas (luma > 235). To maintain the luma range with TMPGEnc you will have to use AviSynth to convert the YUY2 video to RGB without performing the usual luma expansion (ConvertToRGB(matrix="PC.601")) and then tell TMPEnc to leave the luma alone (select the option "Output YUV data as Basic YCbCr not CCIR601").

The other question is whether or not you really want to leave the luma level intact. As JohnnyMalaria points out, MPEG is not supposed to have much below luma 16 and above 235. You may find that your DVD player and/or TV shows everything below 16 as the same shade of black -- all detail in those dark areas will be lost. Whites are usually a little more forgiving (bright areas over 235 may still show some detail).

Notice how the shadowed hedge along the left of the image has lost lots of its detail in the full range samples. You can still see detail in the compressed range sample. I recommend you shoot a bunch of samples under different lighting conditions then encode both ways and compare the results on a TV, not the computer.

[boriskey]

I think I get it now, thank you very much for your time and explanaition. As always it is all about trade-offs

To answer your question what MPG encoder I use - I do not have too much choice here. I prefer to use Ulead Video Studio with its Ulead MPG encoder or Sony Vegas with MainConcepts codec.

It seems that these applications cannot use other codecs. And then I tried your advice, I got the same clip on Ulead, but I got better and vivid colors on Vegas (close to your clip, but still less in size - 4,2Mb versus your 4,8 even on the best quality settings). But I see now that white areas got worse, so I probably listen to John and will lose luma instead of making white points worse

[manono]

Hi-

Use an MJPEG decoder that supports YUY2 or YV12 output. Don't convert to RGB.

I don't have an MJPEG codec installed, and was forced to use DirectShowSource to open it. I had added the Info line and it showed the AVI to be RGB32. So you're saying I went about it wrong, opening the AVI that way? And to use ColorYUY to get detailed information on the luma values I was forced to ConvertToYUY2:

DirectShowSource("I:\Test\Test\AsFirstVideoClip.av i")
Info()
ConvertToYUY2()
ColorYUV(Analyze=True)

I was playing around last night turning the two commands on and off and seeing what I got, and it was always 16-235 with RGB32 (unless I put the Info command after the colorspace conversion). And just now I was investigating why I got different luma values from you and JohnnyMalaria, and stumbled across the reason - the ConvertToYUY2() command. It clamps it to 16-235 by default. You have to define a Matrix if you want it 0-255. I notice now that you said something about doing that when feeding a script into TMPGEnc.

Anyway, I did say it was strange (to me) that my results were so different. And the MPEG2.mpeg isn't clamped to 16-235, but has a normal amount of data below 16 and above 235.

As JohnnyMalaria points out, MPEG is not supposed to have much below luma 16 and above 235.

But that's not what he said, and that was my beef with his remarks. He said there's a requirement for it to be clamped between 16-235, and that's not true. Ideally, yes, there's little below 16 and above 235, especially above 235. It's usually OK for there to be a lot of blacker-than black information, assuming you have both a player and a calibrated display that can pass blacker-than-black.

The other question is whether or not you really want to leave the luma level intact.

I think I agree that the original MPG, the MPEG2.mpeg is better looking than either the source AsFirstVideoClip.avi or your faithful reencode to MPG, the intact.mpeg. Both have crushed blacks and badly blown out whites. The white car following the yellow car is a flat 2-dimensional blob, with no defined features in the 0-255 ones. And the MPEG2.mpg has a more typical amount of pixels below 16 and above 235. In my opinion the video was improved in the MPEG2.mpg encode.

[JohnnyMalaria]

I'll qualify my statement in that to be within the legal range of ITU-R 601, the components should be clamped. Yes, you can go beyond the legal range but then you run the risk of the playback equipment forcibly clamping the output.

As with many things, you can encode outside of the specifications but it isn't wise.

[boriskey]

now I like mpeg2.mpg more too I did not notice that sky and building and white car look better after conversion to MPG.

The only question I have though to you guys, why intact.mpv better in quality and size (just look at the grass and flowers) than my mpeg2.mpg. I played with bitrate and max quality settings, but I could not get the same quality as on intact.mpv

I also tried to add more I-frames, but it did not help

Is it because I use ulead codec and mainconcept?

[manono]

As with many things, you can encode outside of the specifications but it isn't wise.

It's important for black to be 16 and white to be 235. It's also important for any below 16 and any above 235 data be retained. Clamping to 16-235 isn't a good idea. Every (decent) DVD ever made has data outside of 16-235. Not some, not most, but all. For example, here's a frame - a typical frame - from the R1 Star Wars: The Empire Strikes Back:



Limiter(Show="Luma")
ColorYUV(Analyze=True)

The Limiter filter clamps it to 16-235. I put it on because it has a Show="Luma" parameter which shows the original pixels outside of 16-235. The red ones were originally below 16 and the green ones above 235. Parts of this movie are pretty dark (some of the space scenes are almost all below 16), and this data is supposed to be there. These are, after all, the THX folks who should know what they're doing. No DVD production house clamps to 16-235 just because viewers may be using inferior or uncalibrated equipment.

[jagabo]

I don't have an MJPEG codec installed, and was forced to use DirectShowSource to open it. I had added the Info line and it showed the AVI to be RGB32. So you're saying I went about it wrong, opening the AVI that way? And to use ColorYUY to get detailed information on the luma values I was forced to ConvertToYUY2:

DirectShowSource("I:\Test\Test\AsFirstVideoClip.av i")
Info()
ConvertToYUY2()
ColorYUV(Analyze=True)

I was playing around last night turning the two commands on and off and seeing what I got, and it was always 16-235 with RGB32 (unless I put the Info command after the colorspace conversion). And just now I was investigating why I got different luma values from you and JohnnyMalaria, and stumbled across the reason - the ConvertToYUY2() command. It clamps it to 16-235 by default. You have to define a Matrix if you want it 0-255.

No, your real problem was that your MJPEG decoder expanded the luma range when it converted to RGB. Luma 16-235 became RGB 0-255. All luma below 16 and above 235 were lost before you even got the data. This is the "normal" method of converting video YUV to RGB. Every program I've used of does it by default (Sony Vegas may not, but I've never used it).

If you want to avoid the luma expansion you need to perform the YUV to RGB conversion yourself with ConvertToRGB(matrix="PC.601"). The reverse would be ConvertToYUY2(matrix="PC.601"). Of course, this requires that you get ahold of the YUV data before it's been converted to RGB.

ConvertToRGB() and ConvertToYUY2() use luma expansion and compression. Any luma below 16 becomes 16 and anything above 235 becomes 235 after the pair of operations. The stuff in the middle remains intact (except for rounding errors). Note the implied matrix in these operations is rec.601. Here's a YUY2 source with luma from 0-255 on the left, and after ConvertToRGB() and ConvertToYUY2():



ConvertToRGB(matrix="PC.601") and ConvertToYUY2(matrix="PC.601") do not use luma expansion and compression. After the pair of operations all luma levels are retained (except for rounding errors). I was going to post a sample of this too but the before and after images are the same so there's no point!

Anyway, I did say it was strange (to me) that my results were so different. And the MPEG2.mpeg isn't clamped to 16-235, but has a normal amount of data below 16 and above 235.

MPEG2.mpeg has nothing below 16 and nothing above 235 if you look at the YUV data before conversion to RGB and the accompanying luma expansion.

[jagabo]

now I like mpeg2.mpg more too :) I did not notice that sky and building and white car look better after conversion to MPG.

Keep in mind that what you see on an RGB computer display has gone through the usual luma expansion during YUV to RGB conversion to put the data on the screen. If you watch the MPEG 2 data on a TV you may still see the details in the dark and light areas. This will depend on your DVD player and your TV and its settings (brightness, contrast).

The only question I have though to you guys, why intact.mpv better in quality and size (just look at the grass and flowers) than my mpeg2.mpg. I played with bitrate and max quality settings, but I could not get the same quality as on intact.mpv

The differences I see are the addtional contrast and a slightly sharper image in intact.mpv. The contrast difference is due to the luma compression issue. The sharpness is probably the resizing algorithm (640x480 -> 720x480). I used LanczosResize() in AviSynth (very sharp). You probably used a bilinear resize (not so sharp). Note that sharper is not always better.

Regarding the file size: that's simply a matter of the bitrates chosen. I used CCE with a single pass constant quality encode with min=1000, max=9000. I think I used a quality level of 30. Also note that my file had no audio. Yours had LPCM audio.

[JohnnyMalaria]

As with many things, you can encode outside of the specifications but it isn't wise.

It's important for black to be 16 and white to be 235. It's also important for any below 16 and any above 235 data be retained. Clamping to 16-235 isn't a good idea. Every (decent) DVD ever made has data outside of 16-235. Not some, not most, but all. For example, here's a frame - a typical frame - from the R1 Star Wars: The Empire Strikes Back:

<IMAGE REMOVED>

Limiter(Show="Luma")
ColorYUV(Analyze=True)

The Limiter filter clamps it to 16-235. I put it on because it has a Show="Luma" parameter which shows the original pixels outside of 16-235. The red ones were originally below 16 and the green ones above 235. Parts of this movie are pretty dark (some of the space scenes are almost all below 16), and this data is supposed to be there. These are, after all, the THX folks who should know what they're doing. No DVD production house clamps to 16-235 just because viewers may be using inferior or uncalibrated equipment.

Whether the THX folks know what they are doing or not, your example is likely out of specification. The limits on Y (16 - 235) and Cr/Cb (16 - 240) exist for one major reason: to ensure that, statistically, the corresponding RGB values will not exceed the 8-bit 0 - 255 range. Just because the Y, Cr and Cb channels aren't clipping, it doesn't mean the RGB won't. The question with the Star Wars frame is whether or not the RGB values associated with the <16 luma are valid. For EBU, the RGB range is required to be 16 - 235. If the Star Wars DVD falls out of the range, it is illegal - irrespective of who encoded it. The RGB domain is the important one - we view RGB displays. The luma/chroma variants are convenient means of reducing the bandwidth required to transmit the data and, more recently, to compress the data.

ITU-R 601 video content should be encoded with the prescribed Y, Cr and Cb limits. Existing content that is being processed further should maintain the footroom and headroom data. Conversion between YCrCb and RGB should use the Studio RGB colorspace.