How to add HDR metadata?

[Boxcart]

Hello,

I am capturing from a DeckLink Mini Recorder 4K card via FFmpeg, which works mostly flawlessly. The only problem is that FFmpeg in this particular configuration (DeckLink via dshow) apparently is incapable of adding HDR metadata to the output MKV file. This results in playback where the colors are washed out and skew reddish and bluish at times. The video itself is an HDR recording, it just lacks the metadata that allows the software/display to show the colors properly. If I could add the HDR metadata to the output MKV file, I assume it would then playback properly? I know what the parameters are, I just need to add them. I heard MKVToolNix can do this at the container level but I was unsuccessful figuring out what integers to put in all the header fields. Does anyone know how to achieve that?

Thanks.

[Selur]

Here is an example using mkvmerge:

mkvmerge --ui-language en -o "G:\Output\2023-03-13@11_22_36_5110__04.mkv" -d 0 --default-track 0:yes --colour-matrix-coefficients 0:9 --colour-range 0:1 --colour-transfer-characteristics 0:16 --colour-primaries 0:9 --max-frame-light 0:300 --max-luminance 0:1000.0 --min-luminance 0:0.0001 --chromaticity-coordinates 0:0.68,0.32,0.27,0.69,0.15,0.06 --white-colour-coordinates 0:0.3127,0.3290 --colour-bits-per-channel 0:10 --track-name 0:"Dummy" --default-duration "0:25/1fps" --aspect-ratio-factor 0:1/1 --no-chapters --compression -1:none --forced-track 0:yes --field-order 0:0 --no-audio --no-subtitles "J:\tmp\2023-03-13@11_22_36_5110_02.265"

You could also use mkvpropedit:

mkvpropedit --edit track:1 -s colour-primaries=9 -s colour-transfer-characteristics=16 -s colour-matrix-coefficients=9 output.mkv

-s colour-matrix-coefficients=9 -s colour-range=1 -s colour-transfer-characteristics=16 -s colour-primaries=9 -s max-frame-light=300 -s max-luminance=1000.0 -s min-luminance=0.0001 -s chromaticity-coordinates=0.68,0.32,0.27,0.69,0.15,0.06 -s white-colour-coordinates=0.3127,0.3290
to edit exiting mkvs.

See https://mkvtoolnix.download/doc/mkvmerge.html#mkvmerge.description under 2.8 to see what inter matches to what value.
Alternatively see: https://manpages.ubuntu.com/manpages/impish/man1/mkvmerge.1.html
Whether setting those values on the container level is enough, depends on the player you use.

Cu Selur

[Boxcart]

Here is an example using mkvmerge:

mkvmerge --ui-language en -o "G:\Output\2023-03-13@11_22_36_5110__04.mkv" -d 0 --default-track 0:yes --colour-matrix-coefficients 0:9 --colour-range 0:1 --colour-transfer-characteristics 0:16 --colour-primaries 0:9 --max-frame-light 0:300 --max-luminance 0:1000.0 --min-luminance 0:0.0001 --chromaticity-coordinates 0:0.68,0.32,0.27,0.69,0.15,0.06 --white-colour-coordinates 0:0.3127,0.3290 --colour-bits-per-channel 0:10 --track-name 0:"Dummy" --default-duration "0:25/1fps" --aspect-ratio-factor 0:1/1 --no-chapters --compression -1:none --forced-track 0:yes --field-order 0:0 --no-audio --no-subtitles "J:\tmp\2023-03-13@11_22_36_5110_02.265"

You could also use mkvpropedit:

mkvpropedit --edit track:1 -s colour-primaries=9 -s colour-transfer-characteristics=16 -s colour-matrix-coefficients=9 output.mkv

-s colour-matrix-coefficients=9 -s colour-range=1 -s colour-transfer-characteristics=16 -s colour-primaries=9 -s max-frame-light=300 -s max-luminance=1000.0 -s min-luminance=0.0001 -s chromaticity-coordinates=0.68,0.32,0.27,0.69,0.15,0.06 -s white-colour-coordinates=0.3127,0.3290
to edit exiting mkvs.

See https://mkvtoolnix.download/doc/mkvmerge.html#mkvmerge.description under 2.8 to see what inter matches to what value.
Alternatively see: https://manpages.ubuntu.com/manpages/impish/man1/mkvmerge.1.html
Whether setting those values on the container level is enough, depends on the player you use.

Cu Selur

Thanks for this. I'm trying to use the mkvmerge method that you provided but I'm having trouble with the syntax. Where is the input and output specified? Right now I'm using:

mkvmerge --ui-language en -o "C:\Users\Me\InputVideoWithHDRcolorProblem.mkv" -d 0 --default-track 0:yes --colour-matrix-coefficients 0:9 --colour-range 0:1 --colour-transfer-characteristics 0:16 --colour-primaries 0:9 --max-frame-light 0:300 --max-luminance 0:1000.0 --min-luminance 0:0.0001 --chromaticity-coordinates 0:0.68,0.32,0.27,0.69,0.15,0.06 --white-colour-coordinates 0:0.3127,0.3290 --colour-bits-per-channel 0:10 --track-name 0:"Dummy" --default-duration "0:25/1fps" --aspect-ratio-factor 0:1/1 --no-chapters --compression -1:none --forced-track 0:yes --field-order 0:0 --no-audio --no-subtitles "C:\Users\Me\Desktop\OutputMKVMergeVideoWithGoodHDR.mkv"

and getting the error "Error: The file '--colour-matrix-coefficients' could not be opened for reading: open file error."

For reference my FFmpeg command is:

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -c:v v210 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range tv -chroma_sample_location left -color_primaries bt2020 -colorspace bt2020nc -color_trc smpte2084 -c:a ac3 -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -maxrate:v 100M -minrate:v 100M -bufsize:v 100M output.mkv

[Boxcart]

Update: I just manually entered the values you provided using MKVToolNix and MediaInfo now detects it as HDR10 but when playing in VLC the colors are still messed up (too cyan and magenta) just the same.

[Selur]

too cyan and magenta

your source uses HDR10 not DV, right?

getting the error "Error: The file '--colour-matrix-coefficients' could not be opened for reading: open file error."

Depending on you mkvmerge version you need to use: '--colour-matrix' instead of '--colour-matrix-coefficients'. (this was changed recently)

For reference my FFmpeg command is:

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -c:v v210 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range tv -chroma_sample_location left -color_primaries bt2020 -colorspace bt2020nc -color_trc smpte2084 -c:a ac3 -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -maxrate:v 100M -minrate:v 100M -bufsize:v 100M output.mkv

Okay, depending on your systems cpu power you could use x265 and set the metadata in the x265_params.

I just manually entered the values you provided using MKVToolNix and MediaInfo now detects it as HDR10 but when playing in VLC the colors are still messed up (too cyan and magenta) just the same.

Assuming you are using an up-to-date VLC this either means:
a. VLC does not look at the metadata in the container headers for HEVC or it does not support HDR handling. (you need at least VLC 3.+ for HDR handling iirc)
b. the color information in the hevc stream are messed up somehow so that VLC can't properly apply the metadata.

=> What does ffprobe and mediainfo report about your 'patched' file?
Since you use dshow your are probably on Windows, so try how the colors look if you use mpc-hc + mdvr (https://www.youtube.com/watch?v=4LDPrJhsmwA).


Cu Selur

[Boxcart]

too cyan and magenta

your source uses HDR10 not DV, right?

getting the error "Error: The file '--colour-matrix-coefficients' could not be opened for reading: open file error."

Depending on you mkvmerge version you need to use: '--colour-matrix' instead of '--colour-matrix-coefficients'. (this was changed recently)

For reference my FFmpeg command is:

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -c:v v210 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range tv -chroma_sample_location left -color_primaries bt2020 -colorspace bt2020nc -color_trc smpte2084 -c:a ac3 -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -maxrate:v 100M -minrate:v 100M -bufsize:v 100M output.mkv

Okay, depending on your systems cpu power you could use x265 and set the metadata in the x265_params.

I just manually entered the values you provided using MKVToolNix and MediaInfo now detects it as HDR10 but when playing in VLC the colors are still messed up (too cyan and magenta) just the same.

Assuming you are using an up-to-date VLC this either means:
a. VLC does not look at the metadata in the container headers for HEVC or it does not support HDR handling. (you need at least VLC 3.+ for HDR handling iirc)
b. the color information in the hevc stream are messed up somehow so that VLC can't properly apply the metadata.

=> What does ffprobe and mediainfo report about your 'patched' file?
Since you use dshow your are probably on Windows, so try how the colors look if you use mpc-hc + mdvr (https://www.youtube.com/watch?v=4LDPrJhsmwA).


Cu Selur

Thanks. Yes I'm 100% sure it is HDR not DV. I had a >3.0 version of VLC but updated to a slightly newer version anyways and got the same result. Plex also displays the colors the same way. I also tried with MPCHD + MadVR and got the same washed out colors. Here is what MediaInfo reports once the file is patched with the parameters you provided:

General
Unique ID : 339250711850741098710078177243341531829 (0xFF394F7A5BF2B501BE1D3F64707C2AB5)
Complete name : C:\Users\Me\testfile.mkv
Format : Matroska
Format version : Version 4
File size : 648 MiB
Duration : 1 min 4 s
Overall bit rate : 84.3 Mb/s
Writing application : Lavf60.4.100
Writing library : Lavf60.4.100
ErrorDetectionType : Per level 1

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 min 3 s
Bit rate : 82.1 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.413
Stream size : 626 MiB (97%)
Writing library : Lavc60.6.100 hevc_nvenc
Default : No
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : R: x=0.680000 y=0.320000, G: x=0.270000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Frame-Average Light Level : 300

Audio
ID : 2
Format : AC-3
Format/Info : Audio Coding 3
Commercial name : Dolby Digital
Codec ID : A_AC3
Duration : 1 min 4 s
Bit rate mode : Constant
Bit rate : 448 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Bit depth : 32 bits
Compression mode : Lossy
Stream size : 3.44 MiB (1%)
Writing library : Lavc60.6.100 ac3
Service kind : Complete Main
Default : No
Forced : No

Here is what ffprobe reports:

Metadata:
ENCODER : Lavf60.4.100
Duration: 00:01:04.47, start: -0.005000, bitrate: 84256 kb/s
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn
Metadata:
ENCODER : Lavc60.6.100 hevc_nvenc
DURATION : 00:01:03.939000000
Side data:
Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2700,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.000100, max_luminance=1000.000000
Stream #0:1: Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
Metadata:
ENCODER : Lavc60.6.100 ac3
DURATION : 00:01:04.472000000

[Selur]

Regarding Plex, make sure your Plex setup supports hdr to sdr tonemapping (https://support.plex.tv/articles/hdr-to-sdr-tone-mapping/)
Metadata looks fine to me. (the correct metadata for your content might be different, my values were just examples)

If you can share, a small sample of untagged video (can be video only and just few seconds of content) and the correct hdr data of your source other users could look at the content.

Cu Selur

[Boxcart]

Regarding Plex, make sure your Plex setup supports hdr to sdr tonemapping (https://support.plex.tv/articles/hdr-to-sdr-tone-mapping/)
Metadata looks fine to me. (the correct metadata for your content might be different, my values were just examples)

If you can share, a small sample of untagged video (can be video only and just few seconds of content) and the correct hdr data of your source other users could look at the content.

Cu Selur

Ok I'll clip a segment to share so people can see the color problem, thank you. In the meantime, I've found some posts online where people talk about dshow being a problem/too simple to handle complex formats. Could that be the issue? Alternatively could NVENC be the problem? I have tried x265 instead of NVENC but within about 25 seconds the PC freezes and I have to hit the reset button on the case. I have an 8 core Intel 11700k but even so it appears not to be able to handle this level of recording. NVENC on my RTX 3080 Ti meanwhile works flawlessly. I previously tried recording through OBS with FFMpeg, but it would have significant frame loss & crash when using a 10-bit color space, which is why I went to FFmpeg directly, which works much better except for this final problem with colors & HDR. I've heard there's a Decklink API enabled version of FFmpeg but I can't figure out the command.

[Boxcart]

Here is a link to an untagged, no audio clip that shows the color problem (too cyan and magenta): https://drive.google.com/file/d/1s6vaCRCpqCHOlQAoFiS7scfXEANiatAK/view?usp=sharing

[Selur]

According to mediainfo the content you shared is tagged:

Video
Count                                    : 381
Count of stream of this kind             : 1
Kind of stream                           : Video
Kind of stream                           : Video
Stream identifier                        : 0
StreamOrder                              : 0
ID                                       : 1
ID                                       : 1
Unique ID                                : 15735890026496679617
Format                                   : HEVC
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format/Url                               : http://www.itu.int
Commercial name                          : HEVC
Format profile                           : Main 10@L5@High
HDR format                               : SMPTE ST 2086
HDR format                               : SMPTE ST 2086, HDR10 compatible
HDR_Format_Commercial                    : HDR10
HDR_Format_Compatibility                 : HDR10
Internet media type                      : video/H265
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 11804.000000
Duration                                 : 11 s 804 ms
Duration                                 : 11 s 804 ms
Duration                                 : 11 s 804 ms
Duration                                 : 00:00:11.804
Duration                                 : 00:00:11;19
Duration                                 : 00:00:11.804 (00:00:11;19)
Bit rate                                 : 84186944
Bit rate                                 : 84.2 Mb/s
Width                                    : 3840
Width                                    : 3 840 pixels
Height                                   : 2160
Height                                   : 2 160 pixels
Stored_Height                            : 2176
Sampled_Width                            : 3840
Sampled_Height                           : 2160
Pixel aspect ratio                       : 1.000
Display aspect ratio                     : 1.778
Display aspect ratio                     : 16:9
Frame rate mode                          : CFR
Frame rate mode                          : Constant
Frame rate                               : 23.976
Frame rate                               : 23.976 (23976/1000) FPS
FrameRate_Num                            : 23976
FrameRate_Den                            : 1000
Frame count                              : 283
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Chroma subsampling                       : 4:2:0
Bit depth                                : 10
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.423
Delay                                    : 0
Delay                                    : 00:00:00.000
Delay                                    : 00:00:00;00
Delay                                    : 00:00:00.000 (00:00:00;00)
Delay, origin                            : Container
Delay, origin                            : Container
Stream size                              : 124217836
Stream size                              : 118 MiB (100%)
Stream size                              : 118 MiB
Stream size                              : 118 MiB
Stream size                              : 118 MiB
Stream size                              : 118.5 MiB
Stream size                              : 118 MiB (100%)
Proportion of this stream                : 0.99993
Default                                  : No
Default                                  : No
Forced                                   : No
Forced                                   : No
colour_description_present               : Yes
colour_description_present_Source        : Container / Stream
Color range                              : Limited
colour_range_Source                      : Container / Stream
Color primaries                          : BT.2020
colour_primaries_Source                  : Container / Stream
Transfer characteristics                 : PQ
transfer_characteristics_Source          : Container / Stream
Matrix coefficients                      : BT.2020 non-constant
matrix_coefficients_Source               : Container / Stream
Mastering display color primaries        : R: x=0.680000 y=0.320000, G: x=0.270000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
MasteringDisplay_ColorPrimaries_Source   : Container
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
MasteringDisplay_Luminance_Source        : Container
Maximum Frame-Average Light Level        : 300
MaxFALL_Source                           : Container

My guess is that the issue is the master display color info.


assuming you used mkvprobedit, try:

-s colour-matrix-coefficients=9 -s colour-range=1 -s colour-transfer-characteristics=14 -s colour-primaries=9 -s max-content-light=300 -s max-luminance=1000 -s min-luminance=0.0001 -s colour-bits-per-channel=10

leaving out the 'chromaticity-coordinates' and 'white-colour-coordinates' and chaging the colour-transfer-characteristics to 14 instead of 16.

Cu Selur

[Boxcart]

According to mediainfo the content you shared is tagged:

Video
Count                                    : 381
Count of stream of this kind             : 1
Kind of stream                           : Video
Kind of stream                           : Video
Stream identifier                        : 0
StreamOrder                              : 0
ID                                       : 1
ID                                       : 1
Unique ID                                : 15735890026496679617
Format                                   : HEVC
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format/Url                               : http://www.itu.int
Commercial name                          : HEVC
Format profile                           : Main 10@L5@High
HDR format                               : SMPTE ST 2086
HDR format                               : SMPTE ST 2086, HDR10 compatible
HDR_Format_Commercial                    : HDR10
HDR_Format_Compatibility                 : HDR10
Internet media type                      : video/H265
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 11804.000000
Duration                                 : 11 s 804 ms
Duration                                 : 11 s 804 ms
Duration                                 : 11 s 804 ms
Duration                                 : 00:00:11.804
Duration                                 : 00:00:11;19
Duration                                 : 00:00:11.804 (00:00:11;19)
Bit rate                                 : 84186944
Bit rate                                 : 84.2 Mb/s
Width                                    : 3840
Width                                    : 3 840 pixels
Height                                   : 2160
Height                                   : 2 160 pixels
Stored_Height                            : 2176
Sampled_Width                            : 3840
Sampled_Height                           : 2160
Pixel aspect ratio                       : 1.000
Display aspect ratio                     : 1.778
Display aspect ratio                     : 16:9
Frame rate mode                          : CFR
Frame rate mode                          : Constant
Frame rate                               : 23.976
Frame rate                               : 23.976 (23976/1000) FPS
FrameRate_Num                            : 23976
FrameRate_Den                            : 1000
Frame count                              : 283
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Chroma subsampling                       : 4:2:0
Bit depth                                : 10
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.423
Delay                                    : 0
Delay                                    : 00:00:00.000
Delay                                    : 00:00:00;00
Delay                                    : 00:00:00.000 (00:00:00;00)
Delay, origin                            : Container
Delay, origin                            : Container
Stream size                              : 124217836
Stream size                              : 118 MiB (100%)
Stream size                              : 118 MiB
Stream size                              : 118 MiB
Stream size                              : 118 MiB
Stream size                              : 118.5 MiB
Stream size                              : 118 MiB (100%)
Proportion of this stream                : 0.99993
Default                                  : No
Default                                  : No
Forced                                   : No
Forced                                   : No
colour_description_present               : Yes
colour_description_present_Source        : Container / Stream
Color range                              : Limited
colour_range_Source                      : Container / Stream
Color primaries                          : BT.2020
colour_primaries_Source                  : Container / Stream
Transfer characteristics                 : PQ
transfer_characteristics_Source          : Container / Stream
Matrix coefficients                      : BT.2020 non-constant
matrix_coefficients_Source               : Container / Stream
Mastering display color primaries        : R: x=0.680000 y=0.320000, G: x=0.270000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
MasteringDisplay_ColorPrimaries_Source   : Container
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
MasteringDisplay_Luminance_Source        : Container
Maximum Frame-Average Light Level        : 300
MaxFALL_Source                           : Container

My guess is that the issue is the master display color info.


assuming you used mkvprobedit, try:

-s colour-matrix-coefficients=9 -s colour-range=1 -s colour-transfer-characteristics=14 -s colour-primaries=9 -s max-content-light=300 -s max-luminance=1000 -s min-luminance=0.0001 -s colour-bits-per-channel=10

leaving out the 'chromaticity-coordinates' and 'white-colour-coordinates' and chaging the colour-transfer-characteristics to 14 instead of 16.

Cu Selur

Thank you. Making this adjustment seems to have fixed it in VLC and MPC, but Plex is still showing the washed out colors.

[Boxcart]

Regarding Plex, make sure your Plex setup supports hdr to sdr tonemapping (https://support.plex.tv/articles/hdr-to-sdr-tone-mapping/)
Metadata looks fine to me. (the correct metadata for your content might be different, my values were just examples)

If you can share, a small sample of untagged video (can be video only and just few seconds of content) and the correct hdr data of your source other users could look at the content.

Cu Selur

Just following up regarding Plex after my last post now that we have it working. I'm using Plex with an HDR display, so it isn't quite HDR>SDR tonemapping, but trying to figure out why Plex is still reading the colors wrong even though with the latest adjustments it looks proper in VLC and MPC.

I tried Plex on a PC connected to an HDR monitor from the video card using the Plex HTPC program, as well as on an NVIDIA Shield TV connected to a 4K HDR projector, and the Plex app on an iPhone pro max, all of which showed the messed up colors. On the Plex app on an Xbox Series X there was no video at all, just audio. It only shows the colors properly now since this last fix using VLC or MPC on PC.

But it's almost there, just need to figure out this last hurdle. Thanks again for your help! Any ideas why this is?

I'm far from an expert but if I had to take a guess, it seems like Plex is disregarding the container level metadata and this metadata needs to be added at the bitstream level. I was taking a look at some of the commands here: https://ffmpeg.org/ffmpeg-bitstream-filters.html

but I'm not entirely sure what I'd need to add to accomplish that, if that's even the issue.

[Selur]

The messed up colors happen if something is wrong with the HDR info. If the HDR info is ignored, the colors are usually simply dull.
Can't really say what's the issue since I don't use Plex&co (got no tv only monitors fed through pcs ).
-> hopefully someone using Plex can shed some light on this.

Cu Selur

[Boxcart]

The messed up colors happen if something is wrong with the HDR info. If the HDR info is ignored, the colors are usually simply dull.
Can't really say what's the issue since I don't use Plex&co (got no tv only monitors fed through pcs ).
-> hopefully someone using Plex can shed some light on this.

Cu Selur

Just for reference I opened a MakeMKV UHD BluRay rip (all of which Plex plays properly with proper colors) in MKVToolNix and saw that there is no container level HDR metadata at all, as it relates to what we added that got it to work in VLC and MPC.

So it would seem like it needs to be at the bitstream/video level. Can FFmpeg do that? It seems like the instructions in my command just add that metadata to the container but if I can figure out how to tell FFmpeg to record the HDR input with that information in the video as opposed to adding it to the MKV (which Plex seems to not see) then it should work.

[Selur]

Can FFmpeg do that?

No. Afaik. there is no tool which can change the stream (without reencoding) and add hdr10 metadata.

[Boxcart]

Can FFmpeg do that?

No. Afaik. there is no tool which can change the stream (without reencoding) and add hdr10 metadata.

I can record it again I'm just wondering if I can add something to the ffmpeg command that would put it in the stream on the next recording attempt?

If ffmpeg can't do that at all just in terms of a function, perhaps I could run it through something like Shutter Encoder on lossless settings but with the HDR specs added to the re-encode.

[Selur]

You could capture to a lossless format and then reencode it with x265 or nvencc to hevc. Both of them can add hdr metadata.

[autodidact]

This patcher supposedly adds HDR10 metadata to a hevc_nvenc encoded stream, since that encoder can't pass that info to the output like libx265 can with the x265-params switch.

https://github.com/SK-Hardwired/nv_hevc_hdr_patcher

However, it seems like you may have lost HDR somewhere in your capture chain. Viewing your sample on my non-HDR display with no tone-mapping, the picture appears "correct". Normal HDR10 would look washed-out and flat in that case. When I enable tone-mapping via madvr the colors are skewed as you describe, as if tone-mapping something that doesn't require it. This is somewhat speculative on my part.

You might consider removing the pre-input video codec from your capture command to see if that is affecting HDR output:

-c:v v210

Also, maybe setting the same colorimetry pre-input as you have set in the ouptut.

[Boxcart]

This patcher supposedly adds HDR10 metadata to a hevc_nvenc encoded stream, since that encoder can't pass that info to the output like libx265 can with the x265-params switch.

https://github.com/SK-Hardwired/nv_hevc_hdr_patcher

However, it seems like you may have lost HDR somewhere in your capture chain. Viewing your sample on my non-HDR display with no tone-mapping, the picture appears "correct". Normal HDR10 would look washed-out and flat in that case. When I enable tone-mapping via madvr the colors are skewed as you describe, as if tone-mapping something that doesn't require it. This is somewhat speculative on my part.

You might consider removing the pre-input video codec from your capture command to see if that is affecting HDR output:

-c:v v210

Also, maybe setting the same colorimetry pre-input as you have set in the ouptut.

Thanks for looking into it. I tried recording without:

-c:v v210

and got the same outcome. I tried that patcher you linked to with the following code and made no adjustments to container level metadata (fresh from ffmpeg) and got messed up colors still, in VLC and MPC, not even just in Plex:

"C:\Users\Me\Desktop\nvhsp with runtimes\nvhsp.exe" "C:\Users\Me\Desktop\nvhsp with runtimes\infile.h265" -colorprim bt2020 -transfer bt2020-10 -colormatrix bt2020nc -chromaloc 0 -maxcll 300 outfile.h265"

I then tried putting the colorimetry info pre-input in the ffmpeg command, which is what I think you were describing:

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -c:v v210 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -color_trc smpte2084 -color_primaries bt2020 -colorspace bt2020nc -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range 1 -chroma_sample_location left -color_trc smpte2084 -color_primaries bt2020 -colorspace bt2020nc -c:a ac3 -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -maxrate:v 100M -minrate:v 100M -bufsize:v 100M outputprepost.mkv

Which also resulted in the same outcome. To your point about HDR "getting lost," I notice that one of the things that "fixed" the color problem was changing the transfer characteristics from 16 to 14 as Selur suggested above. Even though software like MediaInfo then still says the file is HDR10, it does look flat as you are describing. So I don't know if that solves the problem actually I think it just fixes the colors but once again breaks HDR which does seem to be getting "lost."

[Selur]

It was not just changing the color matrix,...

but also not setting ' -s chromaticity-coordinates=0.68,0.32,0.27,0.69,0.15,0.06 -s white-colour-coordinates=0.3127,0.3290'

nv_hevc_hdr_patcher only patches the first header in the stream not all, so depending on the play it might or might not help.

Cu Selur

[Boxcart]

It was not just changing the color matrix,...

but also not setting ' -s chromaticity-coordinates=0.68,0.32,0.27,0.69,0.15,0.06 -s white-colour-coordinates=0.3127,0.3290'

nv_hevc_hdr_patcher only patches the first header in the stream not all, so depending on the play it might or might not help.

Cu Selur

I tried to mux the output h.265 file from nv_hevc_hdr_patcher into an MKV so I could add those values to the header (it does not seem like the tool can add those to the stream itself) and MKVToolNix failed to open the h265 file with error 0.

I'm beginning to wonder as per the other poster's comment if the recording coming out of FFmpeg is actually HDR at all. It's interesting that after applying the metadata, the colors are off, as if an unnecessary/improper adjustment is being made to the footage. Then, as per one of your suggestions, changing the transfer characteristics from 16 to 14 "fixes" the color problem, but only by dulling the colors it seems, in other words while the colors are no longer too magenta and cyan, and the metadata in MediaInfo and Plex identifies HDR, the colors are dull and do not look like HDR is actually present.

The evidence against that is when the colors are having that "magenta and cyan" problem at transfer characteristics 16, the contrast and lighting looks correct, a sign HDR is in fact working, which after changing transfer characteristics to 14 goes away, along with the improper colors, resulting in a dull, non-HDR looking video that is still read as having HDR.

At this point it would seem like hevc_nvenc is the issue. In OBS' implementation of FFmpeg, it would drop significant frames and crash using hevc_nvenc with a 10 bit color space (P010) but worked with an 8-bit colors space (NV12) albeit with the wrong colors in the output file.

I can't find anything conclusive on this online. All I can conclude is that for some reason FFmpeg, hevc_nvenc, and HDR do not work together, perhaps because hevc_nvenc does not support stream level HDR metadata it seems to appear.

In this case I guess I need to try libx265, but now need to make a command that would work for that to give it a try. The "x265 params" seem to allow more precise HDR parameters that should work. While I have a powerful PC and a recent high power Intel CPU, my bit of experience with non-NVENC/hardware accelerated encoders has not been promising, and I wonder if FFmpeg can do it without crashing.

Other than that I'm not sure what to do.

[Boxcart]

I just tried recording with libx265 with this command:

 ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range 1 -chroma_sample_location left -color_trc smpte2084 -color_primaries bt2020 -colorspace bt2020nc -c:a ac3 -c:v libx265 -profile:v main10 -preset: slower -b:v 80M -maxrate:v 100M -minrate:v 100M -bufsize:v 100M outputx2656.mkv

and the outcome is exactly the same. When I set the HDR parameters in the header it's exactly the same as what happens when doing so with the hevc_nvenc file. The only difference is ffmpeg crashes after a few seconds with libx265 as the buffer gets full and frames are dropped. But it seems to handle HDR exactly the same way.

That means I might as well keep using hevc_nvenc since it does not crash. I'm also unsure if changing transfer characteristic from 16 to 14 actually fixes it (in VLC and MPC, since Plex is still showing wrong colors as it seems to disregard container level metadata) and that the colors are actually too dull/HDR is not working or if it is working and I'm just biased from having just looked at the washed out footage, or if HDR is actually not working at all at transfer characteristics 14 but since the colors are no longer washed out it seems :fixed."

[Boxcart]

Update: I got libx265 working with this command:

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -x265-params "range=limited:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:chromaloc=2:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1):hdr10=1:max-cll=1000,300" -color_range 1 -chroma_sample_location left -color_trc smpte2084 -color_primaries bt2020 -colorspace bt2020nc -c:a ac3 -c:v libx265 -profile:v main10 -preset: ultrafast -b:v 80M -maxrate:v 100M -minrate:v 70M -bufsize:v 100M outputx265.mkv

it's exactly the same situation as with hevc_nvenc in terms of the color problem. I've confirmed that libx265 is putting the HDR metadata in the stream (container header empty) and even so the colors are wrong. I'm really scratching my head at what could be the issue. I think it's narrowed down to the fact transfer characteristics 16 has proper HDR contrast/brightness but is too magenta and cyan, whereas transfer characteristics 14 has accurate colors but is flat and the HDR does not seem to be effective or possibly even enabled.

Just so everyone can see what I mean:

Here is the raw output of the hevc_nvenc command:

https://drive.google.com/file/d/1ioVx2YLHB5-dJJwL7DMJfSm_P7gY6AZ-/view?usp=sharing

Here is that same file with the header transfer characteristics changed from 16 to 14:

https://drive.google.com/file/d/1WkmpxROdUNI1yXq__1oyrQZoyQTLb78X/view?usp=sharing

You can see how 16 has clear HDR but the colors are too cyan and magenta. 14 has accurate colors but looks flat as if HDR is not even enabled. For reference I played these back in MPC-HD and VLC.

For reference the hevc_nvenc command is below, and I am going back to using this since libx265 is having exactly the same problem and NVENC is more efficient.

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -c:v v210 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range 1 -chroma_sample_location left -color_trc smpte2084 -color_primaries bt2020 -colorspace bt2020nc -c:a ac3 -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -maxrate:v 100M -minrate:v 70M -bufsize:v 100M output.mkv

[autodidact]

Can you run this and see what pixel formats are available?

ffmpeg -list_options true -f dshow -i video="Decklink Video Capture"

From what I gather the device will use its default unless one is specified. This is true whether using dshow or a decklink enabled ffmpeg. And the decklink device uses an 8-bit format by default. To set a pixel format you can't specify a video codec or if specified it must be set as rawvideo.

Example:

-c:v rawvideo -pixel_format yuv422p10

http://ffmpeg.org/ffmpeg-devices.html#dshow

Of course, disregard if you have already explored these options.

[Boxcart]

I think I finally solved the problem. It was the simplest oversight all along. The device was not actually outputting HDR but outputting SDR. I randomly got the idea to check and enabled developer options on the device which put an overlay on the screen of the actual output and lo and behold it was SDR even though the device settings were HDR. A complete reset fixed the glitch and got it to output HDR (confirmed in overlay) and now the output files are finally proper HDR. Some more testing will confirm it's finally all working with no problem. Lol my bad everyone, I wish I looked at this sooner and questioned what it was saying. I'll report back if there are still problems. The only problem I notice is MediaInfo does not show the output files as HDR10 (Plex does) unless I add the previously mentioned metadata to the header (Plex ignores the header apparently).

Here is my command:

ffmpeg -f dshow -video_pin_name 1 -audio_pin_name 1 -video_size 3840x2160 -rtbufsize 2147480000 -framerate 24000/1001 -channels 6 -sample_rate 48k -sample_size 16 -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range 1 -chroma_sample_location left -color_trc smpte2084 -color_primaries bt2020 -colorspace bt2020nc -c:a ac3 -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -maxrate:v 100M -minrate:v 70M -bufsize:v 100M -t output.mkv

Thanks for all the help in any event getting it working!