ffmpeg creating prores in MOV -- color matrix flagging

[jagabo]

I need to create a 1080p video with Prores in MOV using ffmpeg. I can do that but I can't figure out how to specify the color matrix.

For example, I have a source video for which MediaInfo reports:

Video
ID                                       : 9
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 422 HQ
Codec ID                                 : apch
Duration                                 : 3 min 23 s
Bit rate mode                            : Variable
Bit rate                                 : 218 Mb/s
Width                                    : 1 920 pixels
Clean aperture width                     : 1 888 pixels
Height                                   : 1 080 pixels
Clean aperture height                    : 1 062 pixels
Display aspect ratio                     : 16:9
Clean aperture display aspect ratio      : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 3.509
Stream size                              : 5.17 GiB (96%)
Writing library                          : aja1
Language                                 : English
Encoded date                             : UTC 2019-01-07 01:43:55
Tagged date                              : UTC 2019-01-07 01:43:55
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
matrix_coefficients_Original             : BT.709

But after I encode (input is an AviSynth script) with:

ffmpeg -i %1 -vcodec prores -profile:v 3 -quant_mat -1 -pix_fmt yuv422p10le "%~dpn1.mov"

MediaInfo reports:

Video
ID                                       : 1
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 422 HQ
Codec ID                                 : apch
Duration                                 : 3 min 23 s
Bit rate mode                            : Variable
Bit rate                                 : 330 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 5.310
Stream size                              : 7.82 GiB (99%)
Writing library                          : fmpg
Language                                 : English
Matrix coefficients                      : BT.601

Does anybody know how to force the Color Primaries, Transfer Characteristics, and Matrix Coefficients to BT.709? I guess I'm looking for something like "-x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709" but for Prores, not x264. I don't see anything like this in the private Prores options:

https://ffmpeg.org/ffmpeg-codecs.html#Private-Options-for-prores_002dks

[poisondeathray]

-vf zscale=matrixin=709:matrix=709:transferin=709:transfer=709:primariesin=709:primaries=709

[pandy]

Doubt if there any private codec option, you can try to flag this trough general commands:

  -color_primaries   <int>        ED.V..... color primaries (from 1 to INT_MAX) (default unknown)
     bt709                        ED.V..... BT.709
     unknown                      ED.V..... Unspecified
     bt470m                       ED.V..... BT.470 M
     bt470bg                      ED.V..... BT.470 BG
     smpte170m                    ED.V..... SMPTE 170 M
     smpte240m                    ED.V..... SMPTE 240 M
     film                         ED.V..... Film
     bt2020                       ED.V..... BT.2020
     smpte428                     ED.V..... SMPTE 428-1
     smpte428_1                   ED.V..... SMPTE 428-1
     smpte431                     ED.V..... SMPTE 431-2
     smpte432                     ED.V..... SMPTE 422-1
     jedec-p22                    ED.V..... JEDEC P22
     unspecified                  ED.V..... Unspecified
  -color_trc         <int>        ED.V..... color transfer characteristics (from 1 to INT_MAX) (default unknown)
     bt709                        ED.V..... BT.709
     unknown                      ED.V..... Unspecified
     gamma22                      ED.V..... BT.470 M
     gamma28                      ED.V..... BT.470 BG
     smpte170m                    ED.V..... SMPTE 170 M
     smpte240m                    ED.V..... SMPTE 240 M
     linear                       ED.V..... Linear
     log100                       ED.V..... Log
     log316                       ED.V..... Log square root
     iec61966-2-4                 ED.V..... IEC 61966-2-4
     bt1361e                      ED.V..... BT.1361
     iec61966-2-1                 ED.V..... IEC 61966-2-1
     bt2020-10                    ED.V..... BT.2020 - 10 bit
     bt2020-12                    ED.V..... BT.2020 - 12 bit
     smpte2084                    ED.V..... SMPTE 2084
     smpte428                     ED.V..... SMPTE 428-1
     arib-std-b67                 ED.V..... ARIB STD-B67
     unspecified                  ED.V..... Unspecified
     log                          ED.V..... Log
     log_sqrt                     ED.V..... Log square root
     iec61966_2_4                 ED.V..... IEC 61966-2-4
     bt1361                       ED.V..... BT.1361
     iec61966_2_1                 ED.V..... IEC 61966-2-1
     bt2020_10bit                 ED.V..... BT.2020 - 10 bit
     bt2020_12bit                 ED.V..... BT.2020 - 12 bit
     smpte428_1                   ED.V..... SMPTE 428-1
  -colorspace        <int>        ED.V..... color space (from 0 to INT_MAX) (default unknown)
     rgb                          ED.V..... RGB
     bt709                        ED.V..... BT.709
     unknown                      ED.V..... Unspecified
     fcc                          ED.V..... FCC
     bt470bg                      ED.V..... BT.470 BG
     smpte170m                    ED.V..... SMPTE 170 M
     smpte240m                    ED.V..... SMPTE 240 M
     ycgco                        ED.V..... YCGCO
     bt2020nc                     ED.V..... BT.2020 NCL
     bt2020c                      ED.V..... BT.2020 CL
     smpte2085                    ED.V..... SMPTE 2085
     unspecified                  ED.V..... Unspecified
     ycocg                        ED.V..... YCGCO
     bt2020_ncl                   ED.V..... BT.2020 NCL
     bt2020_cl                    ED.V..... BT.2020 CL
  -color_range       <int>        ED.V..... color range (from 0 to INT_MAX) (default unknown)
     unknown                      ED.V..... Unspecified
     tv                           ED.V..... MPEG (219*2^(n-8))
     pc                           ED.V..... JPEG (2^n-1)
     unspecified                  ED.V..... Unspecified
     mpeg                         ED.V..... MPEG (219*2^(n-8))
     jpeg                         ED.V..... JPEG (2^n-1)

Beware - seem there are two different prores implementations, plain prores is quite limited:

Encoder prores [Apple ProRes]:
    General capabilities: threads intraonly 
    Threading capabilities: frame
    Supported pixel formats: yuv422p10le yuv444p10le

but prores_ks is:

Encoder prores_ks [Apple ProRes (iCodec Pro)]:
    General capabilities: threads intraonly 
    Threading capabilities: frame and slice
    Supported pixel formats: yuv422p10le yuv444p10le yuva444p10le
ProRes encoder AVOptions:
  -mbs_per_slice     <int>        E..V..... macroblocks per slice (from 1 to 8) (default 8)
  -profile           <int>        E..V..... (from -1 to 5) (default auto)
     auto                         E..V.....
     proxy                        E..V.....
     lt                           E..V.....
     standard                     E..V.....
     hq                           E..V.....
     4444                         E..V.....
     4444xq                       E..V.....
  -vendor            <string>     E..V..... vendor ID (default "Lavc")
  -bits_per_mb       <int>        E..V..... desired bits per macroblock (from 0 to 8192) (default 0)
  -quant_mat         <int>        E..V..... quantiser matrix (from -1 to 6) (default auto)
     auto                         E..V.....
     proxy                        E..V.....
     lt                           E..V.....
     standard                     E..V.....
     hq                           E..V.....
     default                      E..V.....
  -alpha_bits        <int>        E..V..... bits for alpha plane (from 0 to 16) (default 16)

[jagabo]

-vf zscale=matrixin=709:matrix=709:transferin=709:transfer=709:primariesin=709:primaries=709

Hooray, that works! Thanks so much. A short test video:

Video
ID                                       : 1
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 422 HQ
Codec ID                                 : apch
Duration                                 : 368 ms
Bit rate mode                            : Constant
Bit rate                                 : 233 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 3.753
Stream size                              : 10.2 MiB (4%)
Writing library                          : Lavc
Language                                 : English
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Beware - seem there are two different prores implementations

Thanks for pointing that out, I had missed it before. That probably explains why changing parameters like -quant_mat didn't make any difference before.

Thanks again guys.

[poisondeathray]

prores_ks produces higher quality , higher bitrates (than even Apple certified prores)

If you want to "spoof" the vendor writing library tag , add -vendor apl0 , and it will say "Writing library : Apple" , instead of Lavc, or fmpg . It increases compatibility in some programs

[pandy]

if this is not required i would avoid video filtering section and try regular flag approach, not sure about zscale performance, probably ordinary scale can do same faster but once again i would try to avoid video filter.

[poisondeathray]

if this is not required i would avoid video filtering section and try regular flag approach, not sure about zscale performance, probably ordinary scale can do same faster but once again i would try to avoid video filter.

If you can find a "regular flag approach" that actually works, you should mention it. "regular flag approach" doesn't work here.

If argument input = output, the zscale filter isn't actually applied or do anything to the video bitstream (it's a no-op) . If you check the differences with and without zscale, there are no differences. ie. It's all metadata

But for actual filtering, zscale is about 2x faster than swscale for typical operations, at least with Intel CPU's

Performance

"z" is optimized for Intel(R) Architecture and features faster processing times than industry standard swscale software.

Time (ms) to resize FHD image to UHD with Lanczos filter.
	z.lib 2.8 	swscale 4.0.2*
Intel(R) Core(TM) i7-8565U 	7.7 	15.2
Intel(R) Xeon(R) Platinum 8176 	10.8 	22.2

Time (ms) to convert FHD BT.709 (YUV) to FHD BT.2020.
	z.lib 2.8 	swscale 4.0.2**
Intel(R) Core(TM) i7-8565U 	8.3 	17.5
Intel(R) Xeon(R) Platinum 8176 	11.5 	25.6

* scale=3840:2160:sws_flags=lanczos+accurate_rnd:sws_dither=none

** colorspace=all=bt2020:iall=bt709:format=yuv420p10

[jagabo]

I tried the regular flags, -color_primaries bt709 -color_trc bt709 -colorspace bt709, but nothing shows up in MediaInfo. I also tried numerically, eg 1 instead of bt709, also nothing.

-vf zscale... didn't slow the encoding.

[pandy]

I tried the regular flags, -color_primaries bt709 -color_trc bt709 -colorspace bt709, but nothing shows up in MediaInfo. I also tried numerically, eg 1 instead of bt709, also nothing.

-vf zscale... didn't slow the encoding.

This is weird then... i made few tries with various configurations and indeed this doesn't work - it must be some serious regression in ffmpeg as there is no other way to control metadata without re-encoding... IMHO using vf (video filter) path is only workaround that may work if you decode and encode video but it will not work for plain copy (i know, not your case).

And remark about zscale - i was not aware that it is faster internal swscale and at the same time offer higher quality and new functionality not present in swscale.

[poisondeathray]

You can alter the metadata for prores afterwards with a bitstream filter. This will work with stream copy (not inplace edit)

https://ffmpeg.org/ffmpeg-bitstream-filters.html#prores_005fmetadata

ffmpeg -i input.mov -c copy -bsf:v prores_metadata=color_primaries=bt709:color_trc=bt709:colorspace=bt709 output.mov

But -bsf:v prores_metadata works for encoding too, this is probably a better way to do it

[pandy]

You can alter the metadata for prores afterwards with a bitstream filter. This will work with stream copy (not inplace edit)

https://ffmpeg.org/ffmpeg-bitstream-filters.html#prores_005fmetadata

ffmpeg -i input.mov -c copy -bsf:v prores_metadata=color_primaries=bt709:color_trc=bt709:colorspace=bt709 output.mov

But -bsf:v prores_metadata works for encoding too, this is probably a better way to do it

Seem this is relatively new filter - at least not present on filter list in ffmpeg binary i use
Thx poisondeathray

[Selur]

iirc prores_metadata was added end of October 2018, so depending on the age of your binary it's not surprising you build doesn't know the filter.

[pandy]

iirc prores_metadata was added end of October 2018, so depending on the age of your binary it's not surprising you build doesn't know the filter.

I use build from 8 November 2018 so seem it should be there... anyway it is quite strange that prores need to use separate bitstream filter where other codecs follow general commands... this is most annoying thing in ffmpeg - lack of systematic approach - everything depends on developer personal preferences not on standardization guidelines... ffmpeg could be really nice software if simple guidelines will be in use.

[Selur]

yeah, that's how ffmpeg is. :P
Another nice example is that with hevc_nvenc there are options named '-spatial_aq' and '-temporal_aq' where as for 'h264_nvenc' they are named '-spatial-aq' and '-temporal-aq' and those are just some of tons of examples like this all across ffmpeg and it's plugins, somewhere along the general evolution of the project codec standards got kind of twisted and not really understandable for non-ffmpeg-developers.
Assuming anything works similar to X in Y, even when X and Y seem similar usually goes wrong.

Pro side to this is that not too many developers get scared away by to strict coding standards and ffmpeg is still under active development unlike mplayer&mencoder.

Cu Selur