FFMpeg + xBR filter color conversion

[chummy02]

I'm using xBR for anti-aliasing purpose but this is causing colors to come wrong. When encoding without xBR filter colors are fine. Source is YUV limited range BT.709.

-vf "xbr=2,scale=iw/2:-1:sws_flags=spline:sws_dither=a_dither,colormatrix=bt601:bt709"

[sneaker]

If your source is BT.709 why do you add "colormatrix=bt601:bt709"?

[chummy02]

I tried without but was looking weird too so i tried that one to find out if colors were wrong. The filter blur make bright colors look a bit different and was making me think if that was wrong color conversion.

[sneaker]

Show ffmpeg log, upload sample file and screenshots before/after.

[pandy]

https://github.com/FFmpeg/FFmpeg/blob/ed93ed5ee320db299dc8c65d59c4f25e2eb0acdc/libavfilter/vf_xbr.c

Seem that this filter works only in RGB color space - IMHO you should force RGB before explicitly.

[chummy02]

This is another file, bt601 source. The change is in contrast or brightness because of unsharp. Xbr2 comes out darker. Is there someway to apply small amount of sharp after xbr process inside ffmpeg like in avisynth?

Original: https://forum.videohelp.com/images/imgfiles/NHBYkpX.png

xbr2: https://forum.videohelp.com/images/imgfiles/GdeRJqw.png

[poisondeathray]

This is another file, bt601 source. The change is in contrast or brightness because of unsharp. Xbr2 comes out darker. Is there someway to apply small amount of sharp after xbr process inside ffmpeg like in avisynth?

Original: https://forum.videohelp.com/images/imgfiles/NHBYkpX.png

xbr2: https://forum.videohelp.com/images/imgfiles/GdeRJqw.png

That's not xbr2 alone. There is something else going on. Probably some colorspace conversion issues, or decoding issues, or display issues. For example if you take your "original" RGB png, and run it through ffmpeg's xbr2 filter with your settings (without colormatrix) and output another RGB png , it looks almost the same in terms of colors and "brightness" . You should take control of the colorspace conversions before/after and to your final format

Also, if something is "darker" shouldn't you "brighten" it instead of sharpening it ? "brightness" isn't quite the same thing as "sharpness" , but there is some overlap in terms of local contrast

[chummy02]

I use supaa+ffmpeg encoding for comparison, take a look the difference against xbr2.

Supaa: https://forum.videohelp.com/images/imgfiles/K7dimSh.png

Supaa command:

ffmpeg32 -i "%%a" -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m -c:v libx264 -preset veryfast -crf 19 -x264opts

xbr2 command:

ffmpeg32 -i "%%a" -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m -c:v libx264 -preset veryfast -crf 19 -x264opts -vf "xbr=2,scale=iw/2:-1:sws_flags=spline:sws_dither=none" -pix_fmt yuv420p

I had no idea why this going on.

[pandy]

check if this make any difference:

sws_flags=spline+accurate_rnd+full_chroma_int+full_chroma_inp:sws_dither=0:in_range=1:out_range=1

[chummy02]

check if this make any difference:

sws_flags=spline+accurate_rnd+full_chroma_int+full_chroma_inp:sws_dither=0:in_range=1:out_range=1

Same result.

[poisondeathray]

post a small sample of your source

what is your final format goal ? x264 YUV 4:2:0 ? with xbr2 applied ?

You probably have to control the colorspace conversions in between since xbr2 is working in RGB

[chummy02]

post a small sample of your source

what is your final format goal ? x264 YUV 4:2:0 ? with xbr2 applied ?

You probably have to control the colorspace conversions in between since xbr2 is working in RGB

I tried with many sources samples, all of them are YUV 4:2:0 limited. Some bt709 others bt601, but result with xbr2 applied is same with slight "darker" video result.
Yes my goal is keeping same settings just with xbr2 applied.

Here a sample from the screenshot posted.

[pandy]

Isn't perceived luminance is outcome of antialiasing (as overall sharpness is reduced also overall amount of energy is lower) - did you verified levels for flat areas - they are changed or not?

[poisondeathray]

Local contrast can change , and that can change "perceived brightness" but not to the extent that he is showing



You can test the avisynth version and you don't get shifting

ffmpeg RGB to RGB test shows no shifting . It's only YUV source with ffmpeg and you can see this on test gradients. Chroma is also shifted so there are other issues . ffmpeg YUV to RGB alone doesn't show these issues using swscale, so there is an implementation problem somewhere

Also the "quality" of the xbr filter is different in ffmpeg, it seems lower than avs version

I'll try to look in more detail to see what is going on and for a workaround later, but for now use the avisynth version if you "need" results. There are other options too like mild downsize/upsize using different algorithms, other AA filters, motion blur, temporal smoothing etc.. (I'm assuming you want it "smoother" for playback)

[pandy]

[Attachment 41942 - Click to enlarge]

[Attachment 41943 - Click to enlarge]

IMHO content is different (soften/blurred). And there is difference between full quantization range and limited quantization range.

[poisondeathray]

It's blurred on purpose. It's not full vs. limited. It's definitively ffmpeg's YUV to RGB conversion, regardless of input or flags or testing other sources. RGB to YUV within ffmpeg is ok. Other (non ffmpeg) YUV to RGB as input is ok (so filter doesn't cause the brightness difference)

Unfortunately no other way in ffmpeg currently for RGB<=>YUV except through swscale. All other filters like zscale work in same color model.

[pandy]

It's blurred on purpose.

I know - i use same method for anialiasing and struggle with same issue (but it was never critical for my need).

It's not full vs. limited. It's definitively ffmpeg's YUV to RGB conversion, regardless of input or flags or testing other sources. RGB to YUV within ffmpeg is ok. Other (non ffmpeg) YUV to RGB as input is ok (so filter doesn't cause the brightness difference)

Unfortunately no other way in ffmpeg currently for RGB<=>YUV except through swscale. All other filters like zscale work in same color model.

Seem avs as source for ffmpeg may workaround this issue...

[chummy02]

I tried avs xbrz and hqx but both of them cause excessive chroma loss for the purple text. Some avs AA scripts dont mess with specific color but i like the pixel art scaler because it dont smooth small text too much like avs AA's does.

[poisondeathray]

I tried avs xbrz and hqx but both of them cause excessive chroma loss for the purple text. Some avs AA scripts dont mess with specific color but i like the pixel art scaler because it dont smooth small text too much like avs AA's does.

So feed it avs script with RGB conversion in avs. FFMpeg can still do the xbr2 filtering if you prefer

Your "original" screenshot "NHBYkpX.png" used the wrong colors for the RGB conversion BTW, 601