Converting rec.709 to rec.601?

[killerteengohan]

I have a blu-ray I want to scale down to 480p, and SD resolution is rec.601 colors, while the blu-ray is rec.709.

When I try to convert it, artifacts or color splotches seem to appear. Here is a screenshot comparison of what I am talking about. Look at the brown hair closely.

http://screenshotcomparison.com/comparison/125704



I am trying to convert it like this in my script and it's the only way I know how to. This is making the artifacts appear.

Colormatrix(mode="rec.709->rec.601", clamp=0, threads=0)

Is there a better way to do this without introducing those artifacts on the colors?

[pandy]

Dithering? It looks like "random" noise.

[killerteengohan]

I don't believe colormatrix uses dithering.

They appear to me like they are already there, just ALOT less visible until I try the conversion.

[sneaker]

Why do both screenshots have the same colors if you ran them through ColorMatrix()?

[jagabo]

ColorMatrix() doesn't dither. I think noise in the original is being accentuated by the conversion because noise in the three channels is crossing channels.

[themaster1]

what kind of source filter do you use; lsmash or else ?

[killerteengohan]

ColorMatrix() doesn't dither. I think noise in the original is being accentuated by the conversion because noise in the three channels is crossing channels.

Why doesn't it do this to the video, when I convert it to RGB? it looks perfect when I am viewing it with ConvertToRGB. I don't want it to end up RGB and would prefer to keep YV12, but it seems to change it better.

[killerteengohan]

what kind of source filter do you use; lsmash or else ?

AVCSource for this one

[jagabo]

ColorMatrix() doesn't dither. I think noise in the original is being accentuated by the conversion because noise in the three channels is crossing channels.

Why doesn't it do this to the video, when I convert it to RGB? it looks perfect when I am viewing it with ConvertToRGB. I don't want it to end up RGB and would prefer to keep YV12, but it seems to change it better.

Because limited range YUV has only about 1/6 as many valid colors as RGB. So errors in YUV to YUV conversions result in more visible errors than in YUV to RGB conversions.

[poisondeathray]

Also, ConvertToRGB is just one 8bit operation . Colormatrix and ConvertToRGB (either encoding, or for when it's displayed) adds another lossy 8bit step , so you'd expect more rounding errors too


You can try a higher bit depth conversion for colormatrix. It should be slightly better when using higher precision. But you probably have to play with the down conversion and dithering algorithm, and some of the settings used. Perhaps with avsresize, avisynthshader , or with vapoursynth

Also, if your desired end format was 10bit instead of 8bit, it would have fewer errors, more valid converted RGB colors . At decent bitrates compression efficiency is better too

Can you post a sample of that source?

[KarMa]

Why doesn't it do this to the video, when I convert it to RGB? it looks perfect when I am viewing it with ConvertToRGB. I don't want it to end up RGB and would prefer to keep YV12, but it seems to change it better.

Because limited range YUV has only about 1/6 as many valid colors as RGB. So errors in YUV to YUV conversions result in more visible errors than in YUV to RGB conversions.

We also view YUV in RGB anyway, as it has to be decoded to RGB to simply view it on any consumer level lcd monitor/gpu.

[jagabo]

Why doesn't it do this to the video, when I convert it to RGB? it looks perfect when I am viewing it with ConvertToRGB. I don't want it to end up RGB and would prefer to keep YV12, but it seems to change it better.

Because limited range YUV has only about 1/6 as many valid colors as RGB. So errors in YUV to YUV conversions result in more visible errors than in YUV to RGB conversions.

We also view YUV in RGB anyway, as it has to be decoded to RGB to simply view it on any consumer level lcd monitor/gpu.

Yes, so when using ColorMatrix() you have two colorspace conversions (YUV -> YUV -> RGB), not just one (YUV -> RGB).

[pandy]

What with conversion from YCbCr601 to RGB to YCbCr709 - do you see same result?

[jagabo]

I don't have the OP's video but using his before image in an AviSynth script -- yes, I got the same result.

I also verified that ColorMatrix() doesn't add grain using some test images with shallow gradients and Histogram(mode="luma").

[pandy]

I don't have the OP's video but using his before image in an AviSynth script -- yes, I got the same result.

I also verified that ColorMatrix() doesn't add grain using some test images with shallow gradients and Histogram(mode="luma").

Weird... same artifacts? have no source to verify but perhaps is able to average area (remove noise - selective gaussian blur?).
Perhaps doing this conversion in more than 8 bit depth can help. Can't see anything on samples except those noise...

[poisondeathray]

You can try a higher bit depth conversion for colormatrix. It should be slightly better when using higher precision. But you probably have to play with the down conversion and dithering algorithm, and some of the settings used. Perhaps with avsresize, avisynthshader , or with vapoursynth

Definitely better, but you can probably tweak the dither algorithm slightly better with fmtc in vapoursynth . It's the dither algorithm now that is adding a very fine pattern noise (mode 0 is ordered) . -1 disables the down conversion dither, but the rounding becomes off and you get color shift

Because I started with the PNG , converted it back to YUV to simulate the starting point, and back to RGB for the screenshot


1) "New" Original

ImageSource("1544429049_4120357968.png")
ConvertToYV12(matrix="rec709")
ConvertToRGB(matrix="rec709")

[Attachment 47451 - Click to enlarge]

2) "usual" 8bit colormatrix, and back to RGB for the screenshot

ImageSource("1544429049_4120357968.png")
ConvertToYV12(matrix="rec709")
Colormatrix(mode="Rec.709->Rec.601", clamp=0)
ConvertToRGB24(matrix="Rec601")

[Attachment 47452 - Click to enlarge]

3) 16bit colormatrix, back to 8bit , and to RGB for the screenshot

ImageSource("1544429049_4120357968.png")
ConvertToYV12(matrix="rec709")
ConvertBits(16) 
ResizeShader(matrixin="Rec709", matrixout="Rec601", kernel="colormatrix")
ConvertBits(8, dither=0)
ConvertToRGB24(matrix="Rec601")

[Attachment 47453 - Click to enlarge]

[jagabo]

I don't have the OP's video but using his before image in an AviSynth script -- yes, I got the same result.

I also verified that ColorMatrix() doesn't add grain using some test images with shallow gradients and Histogram(mode="luma").

Weird... same artifacts?

Not exactly because I was starting with his RGB image and converting it back to rec.709 YV12 as my starting point. But very similar.

[killerteengohan]

I was gonna start a new thread, but figured Id just continue this one since its the same subject pretty much.

When I last left off with this, I ended up with just dealing with the colormatrix artifacts.

I do not wish to deal with these anymore and I loved the result of #3 in poisondeathray's suggestion. All that garbage was pretty much nonexistant when the bits were increased for the colormatrix conversion. The problem is, I could not use that suggestion because I guess the avisynth 2.6.0 I was using does not support 3 of those functions so I just accepted the artifacts.

ConvertBits(16) 
ResizeShader(matrixin="Rec709", matrixout="Rec601", kernel="colormatrix")
ConvertBits(8, dither=0)

Can I install avisynth+ along side my current avisynth installation without having to remove it? I like the familiarity of what I already have, but I cannot use this feature unless I have avisynth+ apparently.

[poisondeathray]

Can I install avisynth+ along side my current avisynth installation without having to remove it? I like the familiarity of what I already have, but I cannot use this feature unless I have avisynth+ apparently.

3.6.1 broke compatibility with many plugins. They need to be recompiled/updated for 3.6.1, and many are being slowly updated

But switching back and forth between avisynth versions is just swapping a .dll (avisynth.dll). For x64 in the Windows/System32 directory; for x86 in the Windows/SysWOW64 directory .

If you look closely or zoom in , the main reason this is working is because of the dither applied in the 16bit to 8bit down conversion. ConvertBits dither=0 is adding ordered dither pattern. Some people don't like that pattern - there are other patterns you can use with other filters eg. error diffusion (floyd-steinberg) , sierra lite etc... ConvertBits dither=1 uses error diffusion

You can get similar results with other methods that work in 3.6.1 currently right now, eg. avsresize (zimg)

ConvertBits(16) 
z_ConvertFormat(colorspace_op="709:709:709:f=>170m:709:709:f", dither_type="none")
ConvertBits(8, dither=0)

[killerteengohan]

So, should I install an older Avisynth+ version? such as a 3.5? or 3.6.0 to keep plugins from being broken? I still want to continue using the same filters I have and a 32-bit version avisynth. I mostly just want the ability to use convertbits and resizeshader so I can get those results in #3 while mostly everything else stays the same.

So if I am understanding you right, basically I can keep my old one, install the + version along side it, and I can switch between which is being used by my encoders, by simply replacing the dll file in the correct folder as I choose to switch between regular and + versions?



While I typically do not like dithering, or adding in grain like patterns, that still looks much better than just accepting the result of colormatrix on its own. Its actually fairly acceptable when it looks that much better overall.

What if I were going to be doing a 480p to 720p scale and doing the opposite which is rec.601 to rec.709? Would I just change the information to the reverse color matrix like below, or does it have a better way of being done?

ConvertBits(16) 
ResizeShader(matrixin="Rec601", matrixout="Rec709", kernel="colormatrix")
ConvertBits(8, dither=0)

[poisondeathray]

So, should I install an older Avisynth+ version? such as a 3.5? or 3.6.0 to keep plugins from being broken? I still want to continue using the same filters I have and a 32-bit version avisynth. I mostly just want the ability to use convertbits and resizeshader so I can get those results in #3 while mostly everything else stays the same.

So if I am understanding you right, basically I can keep my old one, install the + version along side it, and I can switch between which is being used by my encoders, by simply replacing the dll file in the correct folder as I choose to switch between regular and + versions?

3.5.x are the versions that currently works with almost everything. 3.6.x versions break many plugins

You can only concurrently install one x86 , and one x64 version of avisynth

It's not really "along side it". You keep folders with different versions of the .dll (inactive, just as storage) , and you can swap between those versions by changing a .dll in the SysWOW64 directory for x86 (that's the currently active version). Just copy/paste

x64 host is faster, both for avisynth and most encoders, even in the absence of a bottleneck


(You can even mix/match x86/x64 plugins in a script by using mp_pipeline)

While I typically do not like dithering, or adding in grain like patterns, that still looks much better than just accepting the result of colormatrix on its own. Its actually fairly acceptable when it looks that much better overall.

What if I were going to be doing a 480p to 720p scale and doing the opposite which is rec.601 to rec.709? Would I just change the information to the reverse color matrix like below, or does it have a better way of being done?

ConvertBits(16) 
ResizeShader(matrixin="Rec601", matrixout="Rec709", kernel="colormatrix")
ConvertBits(8, dither=0)

But I wouldn't necessarily ordered dithering on everything. Check your source. There might be better methods

You can multithread ResizeShader and get it faster maybe 3-4x on a quadcore; but avsresize (zlib) is about 4-5x faster than that when multithreading (20x faster total)

[killerteengohan]

I guess I will give 3.5 a try then, and see how it works out. Should I do a complete uninstall of avisynth 2.6 before installing +?


This is what I am currently trying to scale. In the original post I was downscaling 1080p to 480p. (rec709 to rec601) This time I'm scaling 480p to 720p. (rec601 into rec709)

https://slow.pics/c/yOyRac6h

You can see how colormatrix is making it appear as if it has more visible halo's, and filthy artifacts appearing in the brown hair kind of like in the original post. Not wanting that mess is whats driven me to want to try the convertbits method to see if it does any better. I feel like the usual colormatrix method is ruining the look.

[killerteengohan]

I will even give a sample video this time. last time I did not have one, but I can provide one this time if you want to bother.

https://mega.nz/file/IhgUVCBA#0yNwgPDknTawrgfCnUxGLTdaoYi7-WeI4zcSECHBWnQ