Color change after MPEG2 encoding with CCE

[Z-C]

Hi everyone.

It seems that after encoding to MPEG2 the video changes it's colors to a darker/more reddish color. It's not too significant, but when comparing the original to the output you can surely see that the colors have changed.

Is this a normal reduction in quality when encoding to mpeg2? To filters were applied at all.

[guns1inger]

What settings did you use ?

CCE does have settings for the color range : 16-235 or 0-255

It is more likely sometghing to do with playback settings. How are you testing ?

[Z-C]

I used 16-235, yet I'm not sure if this is right for me. The movie is a capture to DV format. then opened for fitlering in vdub so it turned to RGB. The filtered footage was saved to lossless RGB(no colorspace conversion), and that's what being fed to CCE.

Also using 3 pass vbr with an avg. bitrate of 4350. A max/min of 8500/2000.
In the advance tab I changed N/M=4. Unchecked output top field first since it's BFF, and changed the offset line to 0.

The rest of the options are basically the default ones. No filters used at all.

[guns1inger]

It could be the colorspace change. Try encoding some of the raw DV in CCE. A few minutes worth should let you know if that is the cause.

[manono]

Hi-

You're just dropping an interlaced RGB AVI into CCE? Bad move that. You should be feeding it an AviSynth script file with the colorspace conversion already done:

ConvertToYUY2(Interlaced=True)

CCE does a lousy job with the RGB->YUY2 conversion, especially if the source is interlaced.

However, that probably doesn't explain the darkened colors. It probably results from the colorimetry of the source AVI. If the source is Rec.601 (as is likely the case), that would explain it, as CCE encodes to Rec.709. AviSynth's ColorMatrix filter will fix that:

In case you captured something or you have a XviD/DivX (both are encoded Rec.601 coefficients), and you want to encode it to mpeg-2 using CCE (which assumes Rec.709 coefficients), you should use the following script (progressive material)

ColorMatrix(clip, mode="Rec.601->Rec.709")

That's from the ColorMatrix readme.

http://forum.doom9.org/showthread.php?t=82217&highlight=colormatrix

[Z-C]

Thanks guys

guns1inger: I will try that and report the results!

manono: That filter is only for progressive footage, as mentioned in your quote. :/
Are you sure that if I converttoyuy2 using avisynth, CCE won't do any other color conversion on top of it? or mess something else up?

[guns1inger]

CCE is YUY2 native, I believe. FitCD adds a ConvertToYUY2 if you choose CCE output.

[jagabo]

I believe there is a bug in ffdshow's VFW DV decompressor where it causes a color shift when using VirtualDub. If you open the same video in VirtualDubMod normally, vs using the "Use AVISynth Template -> DirectShowSource" you'll see the difference:



In both cases ffdshow is performing the YUV to RGB conversion.

[manono]

Hi-

That filter is only for progressive footage, as mentioned in your quote.

The setting listed was for progressive sources. It handles interlaced sources equally well, but differently. From the same readme:

interlaced (default false)

For interlaced material use

Mpeg2source("F:\TestStreams\avs\AguileraGrammies.d 2v", info=3) # for dgdecode v1.20 or newer
ColorMatrix(hints=true, interlaced=true)
or if you don't want to use hints
Mpeg2source("F:\TestStreams\avs\AguileraGrammies.d 2v")
ColorMatrix(interlaced=true)

If you get the filter from the first post in the thread to which I linked, you can read all about it. Or you can work your way through that big thread.

Are you sure that if I converttoyuy2 using avisynth, CCE won't do any other color conversion on top of it? or mess something else up?

Yes, CCE accepts YUY2. If you don't feed it YUY2, it'll convert it itself, and not always well. It outputs YV12, of course.

[edDV]

This article is an interesting reference for fine tuning HDTV color space.
http://www.hdtvmagazine.com/forum/viewtopic.php?t=5317

[Z-C]

manono: Thank you
so RGB-YUY2-YV12 would be better than just RGB-YV12? I will give it a shot on a sample footage.
Will also check that filter!

I tried directly encoding the original DV file in CCE. In this case I didn't really get darker colors, however I did get the colors to be more smudged. Here is en example(I lack a real image editing software at the moment so sorry for the way it looks), the top is the original, the bottom is after CCE. No filters applied.
In this case, the file didn't go through Vdub, but I guess there still is an RGB conversion in the middle since ffdshow/panasonic dv codec both decode to RGB?

jagabo: I was using ffdshow the last time I tried comparing the output in CCE and the original. And ffdshow's VFW decoders are disabled. Except the HuffYUV one.

[jagabo]

In this case, the file didn't go through Vdub, but I guess there still is an RGB conversion in the middle since ffdshow/panasonic dv codec both decode to RGB?

Ffdshow will output the best matching format that both it and the encoder support. If you have YUY2 enabled in ffdshow's output dialog, it will output YUY2 to CCE (since CCE will accept YUY2). If you don't have YUY2 enabled, ffdshow will output RGB (if so configured) to CCE.

jagabo: I was using ffdshow the last time I tried comparing the output in CCE and the original. And ffdshow's VFW decoders are disabled. Except the HuffYUV one.

I've been looking into this ffdshow bug. For some reason VirtualDub won't accept YV12 or YUY2 video from ffdshow's VFW decoders. It only accepts RGB. When ffdshow converts YUV to RGB it does it incorrectly. Again, this only happens with the VFW decoder, not the Directshow decoder. I don't know if the problem is a CCIR 601 vs 709 issue or something else.

[Z-C]

My mistake jagabo... I meant I wasn't using ffdshow when I first encountered the problem. and, the VFW DV decoder is disabled - so that's not the issue in this case I guess. :/

EDIT: I tried making the colorpsace conversion in AviSynth(ConvertToYUY2(interlaced=true)). I'm afraid it didn't help and it looks exactly the same as when directly feeding CCE.

Is this color data loss(either the fact the colors get somewhat smudged, or occasionally darkened), a typical quality loss I'm going to have to deal with(it's not that bad, but I'd rather get almost 1:1 with the source)?

EDIT 2: Maybe it's the fact I'm using 16-235 lumiance level? should I use 0-255?
The video's history is - Analog VHS-PAL DV-Vdub(rgb)-saved as lossless RGB(Lagarith)-and now in CCE for last encoding.

[manono]

Hi-

I tried making the colorpsace conversion in AviSynth(ConvertToYUY2(interlaced=true)). I'm afraid it didn't help and it looks exactly the same as when directly feeding CCE.

I've seen examples of what can happen when just dropping RGB into CCE, and sometimes it's not pretty. If you have any red against black in your video, I've seen really nasty looking blocks sometimes with those 2 colors side-by-side. You can add Info() to the bottom of the script for opening in VDubMod to get some information about your video at that stage.

No filters applied.

CCE has a Low-pass filter on by default (which I keep disabled).

Since you're using Lagarith, you can set the Mode to YUY2 and get the AVI it produces already in YUY2 for CCE.

[jagabo]

I just ran some experiments with CCE and an RGB source. I used the Belle Nuit color chart that has been posted here in the past. My experiments were in NTSC, not PAL.

Encoding an RGB source with CCE resulted in a visible color shift. For example the RGB color 180,180,16 (R,G,B, the yellowish patch near the top left of the chart) became 180,170,9 after compression with CCE and decompression back to RGB (with VirtualDubMod). Other colors in the Belle Nuit chart were similarly effected. TMPGEnc is more accurate: the same patch returned 180,178,15.

Using AVISynth to convert the RGB source to YUY2 then converting it back to RGB (ie, simply ConvertToYUY2() followed by ConvertToRGB()) turned the 180,180,16 patch into 179,181,17 (the expected +/- 1 or 2 error for each component).

Using AVISynth to convert the RGB source to YUY2, then compressing with CCE, and decompressing with VirtualDubMod, turned the 180,180,16 patch into 182,171,10. Not much different than CCE's handling of the RGB source.

Belle Nuit RGB:

[manono]

Right, and that's what the ColorMatrix filter is designed to correct. If your RGB AVI source is BT.601 or similar, that's the kind of change you can expect. TMPGEnc outputs BT.601, so it's not a problem with it. It will change BT.701 sources, I think (which is most DVDs). There's also that VDub filter, BT.709->601, or something like that.

You can find the colorimetry of the AVISource in GSpot (I think). If it says nothing, it's BT.709. Otherwise one of the choices should look different from the rest. Unfortunately, I don't have any BT.601 AVI sources available to confirm at the moment, although it works with MPEG-2 sources. The colorimetry of MPEG-2 sources can be checked using DGIndex as well. Just run the Preview. An MPEG-2 created using TMPGEnc and one encoded using CCE should be different.

Using AVISynth to convert the RGB source to YUY2, then compressing with CCE, and decompressing with VirtualDubMod, turned the 180,180,16 patch into 182,171,10. Not much different than CCE's handling of the RGB source.

The colorspace conversion is what ConvertToYUY2(Interlaced=True) is designed to improve. There's no reason to expect the color change won't happen, whether fed the AVI directly, or frameserved with an AviSynth script. Using AviSynth's ColorMatrix filter correctly should keep the colors from changing.

Edit Later: TMPGEnc outputs FCC, which is very similar to BT.601.

[Z-C]

manono: What's this BT.709, 601 you keep talking about?
About the low pass filter - do you refer to the filter automatically checked in Quality section? I disabled any filters there...

Using AVISynth to convert the RGB source to YUY2 then converting it back to RGB (ie, simply ConvertToYUY2() followed by ConvertToRGB()) turned the 180,180,16 patch into 179,181,17 (the expected +/- 1 or 2 error for each component).

jagabo: Very well put! However, I didn't really understand why you converted to YUY2 and then right after that converted back to RGB? Or do you mean you converted once to YUY2, encoded to mpeg2, and then frameserved the encoded file to Vdub using Avisynth(to make the conversion in Avisynth before vdub tries to do the conversion by itself).

In addition, are you sure that the example you've presented here, applies to every footage? The frame you attached is quite a simple one. I wonder - can you conclude such wide conclusions from this?

With Regards,
Guy.

[manono]

Hi-

manono: What's this BT.709, 601 you keep talking about?

If you'd read the thread, or get the filter and read the doc, you'd know already. From the included doc:

Background information

There are several ways to convert a YUV stream to RGB. The most well known one, uses Rec.601 coefficients. It is for example used in the color conversion routines of AviSynth, VirtualDub and XviD/DivX. When playing back a XviD/DivX the stream is converted to RGB using Rec.601 coefficients. The main issue is that sometimes other coefficients are used for the YUV to RGB conversion (the other two are Rec.709 coefficients and FCC coefficients). A problem arises if a stream is encoded using one set of coefficients (Rec.709 coefficients for many dvd streams for example), and somewhere in the reencoding-processing-playback chain a different set of coefficients is assumed (Rec.601 coefficients for the XviD/DivX decoder or FCC coefficients for TMPGEnc/QuEnc or Rec.709 coefficients for CCE). You will get a slight color distortion, which looks like a change in brightness (it's not really a change in brightness, the colors are just slightly off).

About the low pass filter - do you refer to the filter automatically checked in Quality section? I disabled any filters there...

Good. If the green part of that black/green rectangle in the Adanced screen is no longer sloped, then you have, indeed, disabled it. Most people don't know about that.

[jagabo]

Using AviSynth's ColorMatrix filter correctly should keep the colors from changing.

Thanks for the info regarding CCE's rec.709 requirement. I've only just started using CCE so I'm not that familiar with it yet.

After changing my AVS script to:

AVISource("belle.avi")
ConvertToYUY2()
ColorMatrix(mode="Rec.601->Rec.709")

The colors were accurate after conversion with CCE. For example, 180,180,16 (R,G,B) became 180,181,17. All other colors were within +/- 1 or 2 for each component.

<edit>

I thought I remembered that AVISynth's ConvertToXXX() functions having 601/709 options:

AVISource("belle.avi")
ConvertToYUY2(matrix="rec709")

Works the same as the script given above for CCE.

</edit>

[jagabo]

I didn't really understand why you converted to YUY2 and then right after that converted back to RGB?

I was testing AVISynth's conversion of RGB to YUY2 and back to RGB to verify that AVISynth was working consistently. I wanted to be sure that AVISynth wasn't messing up the conversions.

are you sure that the example you've presented here, applies to every footage? The frame you attached is quite a simple one. I wonder - can you conclude such wide conclusions from this?

The best way to know exactly what is going on with colorspace is to use simple patterns with known colors. The RGB values of each patch in the Belle Nuit chart is known so it's easy to determine exactly how they changed during conversion. The chart has many features that make it very useful for examining colorspace and color subsampling issues.

[Z-C]

manono: Thanks for the explanation .

jagabo: If Avisynth is consistent in the colorspace conversion(as shown in the RGB-YUY2-RGB conversion), why did you get 'bad' results when you first converted to YUY2 with avisynth and then encoded with CCE? Could it be due to bad decompression of Vdub?

[jagabo]

jagabo: If Avisynth is consistent in the colorspace conversion(as shown in the RGB-YUY2-RGB conversion), why did you get 'bad' results when you first converted to YUY2 with avisynth and then encoded with CCE?

Because CCE wants rec.709 not the rec.601 that AVISynth defaults to. After changing the ConvertToYUY2(matrix="rec709") the colors are correct when encoding with CCE.