Problems converting a clip to 10-bit using ffmpeg

[rgr]

I want to convert a file (YUV420, range 16-255) to RGB 10-bit or YUV444 10-bit, but I fail:

a)

ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264rgb -crf 0 -pix_fmt gbrp10le "2019_11_09 12_32_42(rgb).mp4"

Incompatible pixel format 'gbrp10le' for codec 'libx264rgb', auto-selecting format 'bgr24'


b)

ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264 -crf 0 -pix_fmt yuv444p10le "2019_11_09 12_32_42(yuv444p10).mp4"

It works, but the output file is 10x smaller than expected and has poor quality. It even looks like unsharp is not applied.
(ffmpeg version 2022-07-28-git-6f7d3bde11-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers built with gcc 12.1.0 (Rev2, Built by MSYS2 project))

[poisondeathray]

I want to convert a file (YUV420, range 16-255) to RGB 10-bit or YUV444 10-bit, but I fail:

a)
ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264rgb -crf 0 -pix_fmt gbrp10le "2019_11_09 12_32_42(rgb).mp4"

Incompatible pixel format 'gbrp10le' for codec 'libx264rgb', auto-selecting format 'bgr24'

10bit RGB is not supported by libx264rgb
you can check supported pixel formats with ffmpeg -h encoder=libx264rgb

b)
ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264 -crf 0 -pix_fmt yuv444p10le "2019_11_09 12_32_42(yuv444p10).mp4"

It works, but the output file is 10x smaller than expected and has poor quality. It even looks like unsharp is not applied.
(ffmpeg version 2022-07-28-git-6f7d3bde11-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers built with gcc 12.1.0 (Rev2, Built by MSYS2 project))

(Note, -crf 0 is not lossless for 10bit YUV libx264 , you'd need to use -qp 0, in this example I just used crf 1 for demonstration)

ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=full:out_range=full,curves=master='0.0625/0 1/1',unsharp,format=yuv444p10le" -c:v libx264 -crf 1 output.mp4

[rgr]

I want to convert a file (YUV420, range 16-255) to RGB 10-bit or YUV444 10-bit, but I fail:

a)
ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264rgb -crf 0 -pix_fmt gbrp10le "2019_11_09 12_32_42(rgb).mp4"

Incompatible pixel format 'gbrp10le' for codec 'libx264rgb', auto-selecting format 'bgr24'

10bit RGB is not supported by libx264rgb
you can check supported pixel formats with ffmpeg -h encoder=libx264rgb

Only 8-bit RGB. Sad.

b)
ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264 -crf 0 -pix_fmt yuv444p10le "2019_11_09 12_32_42(yuv444p10).mp4"

It works, but the output file is 10x smaller than expected and has poor quality. It even looks like unsharp is not applied.
(ffmpeg version 2022-07-28-git-6f7d3bde11-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers built with gcc 12.1.0 (Rev2, Built by MSYS2 project))

(Note, -crf 0 is not lossless for 10bit YUV libx264 , you'd need to use -qp 0, in this example I just used crf 1 for demonstration)

ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=full:out_range=full,curves=master='0.0625/0 1/1',unsharp,format=yuv444p10le" -c:v libx264 -crf 1 output.mp4

Indeed -- now it works as it should. Thanks. But strange -- why is qp used here?

[poisondeathray]

But strange -- why is qp used here?

It just works out that way, the x264 dev's coded it like that. CRF 0 is only lossless for 8bit YUV

[rgr]

I hurried a little with joy. The conversion to yuv444p10 works, but the unsharp filter is not applied at all. The output video is noticeably less sharp.

[poisondeathray]

I've never used unsharp before, but it also supports yuv444P10le

https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_unsharp.c

Maybe you have to adjust the settings?
https://www.ffmpeg.org/ffmpeg-filters.html#unsharp-1

[rgr]

I've never used unsharp before, but it also supports yuv444P10le

https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_unsharp.c

Maybe you have to adjust the settings?
https://www.ffmpeg.org/ffmpeg-filters.html#unsharp-1

I already know. My editing program just can't handle 10 bit files and for some reason they are out of focus in the output file.
Even MPC-HC can't handle it -- it distorts the colors.

[poisondeathray]

unsharp doesn't appear to do much even at higher strengths in the example

Apply strong luma sharpen effect:

unsharp=luma_msize_x=7:luma_msize_y=7:luma_amount= 2.5

CAS instead works ok, as a different type of sharpener . So there is no problem with the filter chain
https://www.ffmpeg.org/ffmpeg-filters.html#cas

[poisondeathray]

For the colors, it looks like a 709/601 mismatch when YUV is converted to RGB for display by your display chain

You can set the libx264 encoding metadata to 709 , it might help guide your player to display properly

 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

Not sure about the sharpness difference, there might be different scaling for the unsharp filter in 8bit vs. 10bit (e.g. a 1 pixel difference is larger in 8bit in 0-255 vs. 0-1023 in 10bit ) (Not sure when the order of filter pixel format is applied before or after sharpen, you can check with -report) . If I have time I'll look into it later


Most modern NLE's should be able to handle 10bit files by now (not necessarily 10bit444), and 10 bit video is becoming fairly common in consumer video (10bit 4:2:0, that is) . AVC-Intra was AVC and 10bit422 and supported many years by NLE's

[rgr]

unsharp doesn't appear to do much even at higher strengths in the example

Apply strong luma sharpen effect:

unsharp=luma_msize_x=7:luma_msize_y=7:luma_amount= 2.5

CAS instead works ok, as a different type of sharpener . So there is no problem with the filter chain
https://www.ffmpeg.org/ffmpeg-filters.html#cas

In both examples the unsharp filter worked (original video is less sharp). In general, it is enough for me, but I will check your advice.

[rgr]

For the colors, it looks like a 709/601 mismatch when YUV is converted to RGB for display by your display chain

You can set the libx264 encoding metadata to 709 , it might help guide your player to display properly

 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

MPC-HC interprets the color space rather correctly (attachment). I added the above but no changes.

Not sure about the sharpness difference, there might be different scaling for the unsharp filter in 8bit vs. 10bit (e.g. a 1 pixel difference is larger in 8bit in 0-255 vs. 0-1023 in 10bit ) (Not sure when the order of filter pixel format is applied before or after sharpen, you can check with -report)

The unsharp filter works fine. When I loaded this 10-bit file into my editor and rendered it, the sharpness disappeared. I do not know how it's possible But based on that, I assumed the unsharp filter didn't work for yuv444p10.
After checking in MPC-HC, I noticed that unsharp worked fine -- but I discovered bad color reproduction.

[poisondeathray]

MPCHC works for me, colors are accurate . Did you check colorbars on yuv444p10 ?

[poisondeathray]

Here is a yuv444p10 colorbars video .

eg. the "red" bar should be converted to display as RGB 191,0,0

Be careful about screenshots from mpchc - the pngs have a gama tag written in them, so they might look different depending on the application used to view them

[rgr]

Here is a yuv444p10 colorbars video .

eg. the "red" bar should be converted to display as RGB 191,0,0

Be careful about screenshots from mpchc - the pngs have a gama tag written in them, so they might look different depending on the application used to view them

And it's 191,0,0. Regardless of whether I take a screenshot from MPC-HC and watch it in Irfaview or make PrtScr and paste it in Irfanview.
Interesting...

[rgr]

I even recompressed:

ffmpeg -i colorbars_YUV444P10_libx264_crf1_10bit444_flagged. mp4 -c libx264 -pix_fmt yuv444p10le -crf 0 -qp 0 yuv444p10-colorbar.mp4

and I still get 191,0,0...

But I can see the difference in the output section (ffmpeg):
Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv444p10le(tv, bt709, progressive)

This bt709 I don't have when I convert to yuv444p10.

[poisondeathray]

The bt709 is just from the flagging, the -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709 switches

If you check with mediainfo this shows up as
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709


=> The other issue with -vf curves filter is it works in RGB . If I take my test file and use curves=master='0/0 1/1' , it should be a no-op . But colors are wrong. So you need to control the YUV<=>RGB conversions explicitly

The following below results in correct colors, because the conversions are specified as 709 to/from and 16bit planar RGB is used for the curves (slower but better than 8bit in terms of precision)

-vf "zscale=min=709,format=gbrp16le,curves=master='0/0 1/1',zscale=m=709,format=yuv444p10le"

So you can adjust the values for your case

[rgr]

I just figured it out by trial and error:

ffmpeg -i "2019_11_09 14_21_27.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp,colormatrix=bt601:bt709" -c:v libx264 -qp 0 -pix_fmt yuv444p10le "2019_11_09 14_21_27(yuv444p10).mp4"

This solved the color problem.

Which way will be better?

It's a pity that the libx264rgb filter (or utvideo) does not save in RGB 10-bit.

[poisondeathray]

The 16bit intermediate method with zscale will be slightly more accurate for colors and curves manipulations, but slower to process. The colormatrix filter will introduce more rounding errors but faster.

utvideo VFW supports 10bit RGB as UQRG . ffmpeg libavcodec cannot encode it, only VFW can

Another option for lossless 10bit RGB compression would be ffv1

[rgr]

Long night...

Ultimately:

ffmpeg -i "2019_11_09 12_32_42.mts" -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264rgb -qp 0 -pix_fmt gbrp10le "2019_11_09 12_32_42.mp4"

It distorts the colors. I don't know what is causing it (I'm guessing the curve filter assumes bt601 as input). Too bad ffmpeg doesn't display any information. But generally, that's not how it should work. That's what I took as a starting point, but it turned out that my template was just wrong.

This time I transformed the video in Vegas (Brightness/Contrast filter -> contrast=0.62 <-- this is how the 16-255 to 0-255 conversion works) and that was my template.

This:

ffmpeg -i "2019_11_09 14_21_27.mts" -c:a copy -vf "zscale=min=709:rin=full,format=gbrp16le,curves=ma ster='0.0625/0 1/1',zscale=m=709,format=yuv444p10le,unsharp" -c:v libx264 -qp 0 -pix_fmt yuv444p10le "2019_11_09 14_21_27(yuv444p10).mp4"

works properly and best of all. The histogram is the most accurate. Visually identical to Vegas.

The best part is that I found the correct solution along the way, but rejected it as not following the pattern.

ffmpeg -i "2019_11_09 14_21_27.mts" -colorspace bt709 -c:a copy -vf "scale=in_range=fullut_range=full,curves=master='0.0625/0 1/1',unsharp" -c:v libx264 -qp 0 -pix_fmt yuv444p10le "2019_11_09 14_21_27(yuv444p10).mp4"

The graph is more jagged (probably less accurate), but also visually identical to the pattern.

Thanks for your help, the night would be longer without it
1 - vegas (but without sharpening)
2 - your template
3 - colorspace

[poisondeathray]

-colorspace bt709 just tags the file, the actual Y,Cb,Cr pixel values will be wrong . You can test the 3 scenarios on the colorbars and using -vf "curves=master='0/0 1/1'" . For the -colorspace bt709 case, the colors will end up correct if bt601 is used for the RGB conversion for display. Therefore, it will be actual bt601 (due to the RGB auto conversion from the curves) , but tagged as bt709

[rgr]

-colorspace bt709 just tags the file, the actual Y,Cb,Cr pixel values will be wrong . You can test the 3 scenarios on the colorbars and using -vf "curves=master='0/0 1/1'" . For the -colorspace bt709 case, the colors will end up correct if bt601 is used for the RGB conversion for display. Therefore, it will be actual bt601 (due to the RGB auto conversion from the curves) , but tagged as bt709

I don't quite understand...

My mts file has no color information at all. Maybe just tagging it works?
Because why does it work?

Wideo
ID                                      : 4113 (0x1011)
ID menu                                 : 1 (0x1)
Format                                  : AVC
Format/Informacja                       : Advanced Video Codec
Profil formatu                          : High@L4.2
Ustawienia formatu                      : CABAC / 2 Ref Frames
Ustawienia formatu, CABAC               : Tak
Ustawienia formatu, poklatkowy          : 2 klatki
Format settings, GOP                    : M=1, N=15
ID kodeka                               : 27
Czas trwania                            : 10s 10 ms
Rodzaj przepływności                    : Zmienna
Przepływność bitów                      : 24,8 Mb/s
Przepływność maksymalna                 : 26,0 Mb/s
Szerokość                               : 1.920 pikseli
Wysokość                                : 1.080 pikseli
Proporcje obrazu                        : 16:9
Szybkość wyświetlania klatek            : 59,940 (60000/1001) kl/s
Color space                             : YUV
Chroma subsampling                      : 4:2:0
Bit depth                               : 8 bitów
Rodzaj skanu                            : Progresywny
bity/(piksele*ramki)                    : 0.200
Rozmiar strumienia                      : 29,6 MiB (95%)
IrisFNumber                             : 5.000000

[poisondeathray]

"HD" uses 709 by convention, that's the underlying assumption. "SD" uses 601

If you convert YUV=>RGB=>YUV using the same matrix, you get basically the same thing with some rounding errors (will look simliar)

It's also useful to test known colors through a workflow, such as colorbars. If you take control of each step, each conversion, you can find out where the problems are.

You're examining the RGB values with a RGB histogram , not the actual YCbCr values - this means video has been converted to RGB already somewhere . If you convert to RGB using 601 matrix you will get one thing , 709, something different. If you're "viewing" something, it has been converted somewhere to RGB

Tagging does nothing, it just a "label." The actual underlying YCbCr pixel values are unchanged. If you only tag a file and it looks similar as the proper 709 conversion, then something has converted it to RGB using (the wrong) 601 matrix. 2 wrongs make it a basically "right" (with slightly more rounding errors, each conversion causes a few more rounding errors). YUV=>RGB using 601 (for the curves) ,then RGB to YUV using 601 (for the 10bit file) then YUV to RGB for display using 601 . If you use proper 709 for the last display conversion (it should be for HD, by convention), the colors will look wrong on that file, because the actual values had used 601

[rgr]

I tested:

ffmpeg -i colorbars_YUV444P10_libx264_crf1_10bit444_flagged. mp4 -vf "curves=master='0/0 1/1'" out.mp4
ffmpeg -i colorbars_YUV444P10_libx264_crf1_10bit444_flagged. mp4 -colorspace bt709 -vf "curves=master='0/0 1/1'" out.mp4

In both cases I got RGB 208,18,0.

Good, bad?

[poisondeathray]

I tested:

ffmpeg -i colorbars_YUV444P10_libx264_crf1_10bit444_flagged. mp4 -vf "curves=master='0/0 1/1'" out.mp4
ffmpeg -i colorbars_YUV444P10_libx264_crf1_10bit444_flagged. mp4 -colorspace bt709 -vf "curves=master='0/0 1/1'" out.mp4

In both cases I got RGB 208,18,0.

Good, bad?

Bad and the expected wrong result, because curves works in RGB, and ffmpeg uses 601 by default for YUV/RGB conversions. It auto converts behind the scenes

[rgr]

OK I understand.
So earlier conversions worked fine for me by accident.
And the one that was supposed to work properly didn't work right.
Ah, that ffmpeg...why does it use 601 for HD if it should interpret it as Rec709?

[poisondeathray]

Ah, that ffmpeg...why does it use 601 for HD if it should interpret it as Rec709?

That gets asked all the time on the ffmpeg boards - it just does and the developers aren't going to change the behaviour

Sometimes there are cases where it reads flags and some filters work accordingly. Sometimes not. Sometimes some builds react differently, or some commit breaks something. It's constantly changing

It's always best to control everything, and test your workflow end to end