Modify Color Space Flag

[chris319]

On some mp4 videos there is no data in the following metadata fields:

Color primaries
Transfer characteristics
Matrix coefficients

Is there a way to add this data after encoding?

Thank you.

[sneaker]

For H.264/AVC:
https://forum.doom9.org/showthread.php?t=152419

But the mod is not developed anymore, difficult to use and buggy. You may need to demux first and remux after.

[chris319]

I'll pass on that program.

Here is an ffmpeg command line I have used to add color flags to a video which didn't have them before. It adds the flags:

Color primaries: BT.709
Transfer characteristics: BT.709
Matrix coefficients: BT.709

bin\ffmpeg -y -i test_pattern709.mp4 -color_primaries bt709 -color_trc bt709 -colorspace bt709 colorsflagged709.mp4

[sneaker]

That will not only add those flags, it will completely re-encode the file.

[chris319]

That will not only add those flags, it will completely re-encode the file.

As long as it doesn't change any of the video encoding parameters, that shouldn't be a problem.

The user must make sure the color flags he specifies match the color space used to encode the video.

[sneaker]

As long as it doesn't change any of the video encoding parameters, that shouldn't be a problem.

If loss of quality, time and energy aren't problems for you...

[chris319]

As long as it doesn't change any of the video encoding parameters, that shouldn't be a problem.

If loss of quality, time and energy aren't problems for you...

Time and energy aside, if you reencode the original material there shouldn't be a problem, should there? That is, encode first-generation material as opposed to second-generation material.

Do you know of a way to do this without completely reencoding?

[sneaker]

Time and energy aside, if you reencode the original material there shouldn't be a problem, should there? That is, encode first-generation material as opposed to second-generation material.

You lose quality with every iteration (ignoring lossless mode) - that includes encodes from "first-generation" material. Are you willing to accept that loss? Your decision.

Do you know of a way to do this without completely reencoding?

See post #2.

[chris319]

By first-generation material I mean the original thing to be encoded. In my case it is a .bmp image.

I should ask this question in the ffmpeg forum but their email verification system is apparently not working and I am not receiving their verification email and thus cannot post.

[sneaker]

If you need to re-encode anyways (e.g. BMP to H.264) it doesn't matter, of course. Then quality of adding flags when encoding is identical to adding flags after encoding (the post #2 variant). From your first post I got the impression you wanted to only add the flags.

[poisondeathray]

If you were encoding, or re-encoding (not "after encoding" as per the original topic, or maybe it was poorly worded because I thought the same as sneaker), I would recommend using the actual libx264 flags with -x264opts . The assumption is you're using libx264 and AVC with ffmpeg (-c:v libx264). These are actual flags that don't affect the conversion or manipulation of the actual video data to be displayed

-x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

A potential problem is -color_primaries, -color_trc, and -colorspace aren't "flags" - they are ffmpeg switches that actually do conversions . So depending on what ffmpeg "thinks" the input is, you can get unexpected results. Although the behaviour has improved, in the past it had been very very inconsistent - in other ffmpeg builds, they don't even work at all. IMO, it's better to explicitly control the actual conversions going on by specifying input and output by using swscale with -vf scale and /or zscale/zimg , and specifying the actual flags



For actual VUI parameter manipulation of AVC streams (without re-encoding) , besides the mentioned roozhou's modified ffmpeg build, there is a mp4box patch that can adjust AVC/h264 VUI parameters. Mainstream vanilla GPAC builds won't have the patch, but Komisar hosted some binaries with it, unfortunately they are very old binaries. But it works with most generic AVC streams. You would have to build it yourself i you wanted a newer GPAC/mp4box with it

https://github.com/golgol7777/gpac/commit/f0a3c4bbc7904c870ae2952d6b1a1c218c07c6e8
http://komisar.gin.by/tools/

[chris319]

There is a -c copy command in ffmpeg but it doesn't seem to alter the color flags.

[poisondeathray]

There is a -c copy command in ffmpeg but it doesn't seem to alter the color flags.

It does with roozhou's ffmpeg build for AVC streams

But it's a very old binary, you use -vcodec copy and -acodec copy. ie. the old ffmpeg syntax . Not -c:v copy , or -c:a copy, or -c copy

[chris319]

Let me make this more clear.

If I'm re-encoding the original, unencoded material, such as an original bmp image, there will be no loss of quality and I can re-encode.

If I'm dealing with a file from out in the wild, say YouTube, and there are no color-space flags and I want to add these flags without re-encoding, what's the way to do that (adding only the flags without re-encoding)? Or is it even possible? Preferably without using a very old binary of ffmpeg.

[poisondeathray]

Both cases were addressed

If I'm re-encoding the original, unencoded material, such as an original bmp image, there will be no loss of quality and I can re-encode.

There will be loss of quality from the colorspace conversion, and encode if you're using lossy encoding - but no additional quality loss from flagging the bitstream during encoding. The quality loss incurred can be thought of as a "sunk cost", because you were going to encode the "original" anyways. You're not flagging it after encoding as implied by the first post. This is a different situation. You're flagging it during.

If I'm dealing with a file from out in the wild, say YouTube, and there are no color-space flags and I want to add these flags without re-encoding, what's the way to do that (adding only the flags without re-encoding)? Or is it even possible? Preferably without using a very old binary of ffmpeg.

Here you are flagging it after encoding, which is what both sneaker and I thought you wanted

Yes, either the old ffmpeg roozhou build mentioned in post #2, or the old golgol patch for mp4box (old binary available). Yes both are old . You can build it yourself from the posted patches if you really wanted a newer ffmpeg or mp4box build. But I'm assuming the patches are even compatible (sometimes new API "breaks" old things) .

My opinion is who cares if it's "old" ... if it works. In my experience they work 99% of the time on generic AVC streams. Better than the alternative of incurring additional quality loss, or wasting time (re-wrapping is much faster than re-encoding. re-encoding involves decoding and decompressing the stream, then recompressing the stream). When you see all those reported issues of it not working in the roozhou ffmpeg thread, it's partially due to statistical effect - you're not seeing all the successful cases. Nobody reports those. Only problems are reported

[chris319]

-x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

OK, I'll try this.

Let's forget about some guy's old ffmpeg build. If it works 99% of the time, that means it fails 1% of the time

[poisondeathray]

-x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

OK, I'll try this.

Let's forget about some guy's old ffmpeg build.

That's encoding (slower, quality loss) , not re-wrapping . You wouldn't want to do that on some random youtube video, for example. Or someone else's produced video that forgot to flag. You would only do that if you were producing the video. Otherwise "old guy" is much, much more preferable way to go

Also, just a reminder that those flags do nothing to the actual video . They really are flags, not ffmpeg switches. You stilll have to do the proper conversions upstream (e.g. if you were converting RGB to YUV , proper matrix etc...) with whatever method -vf scale, etc...

[poisondeathray]

For the encoding case - if you 're certain that your input file is correctly interpreted, there is nothing wrong with the way you suggested with -color_primaries bt709 -color_trc bt709 -colorspace bt709 if it gives you the correct result. It might actually be preferred in some cases because it's easier, fewer characters to type. I'm just saying be aware that it can lead to unexpected results in some cases, depending on how ffmpeg interprets the file, or which ffmpeg build is being used. For control freaks, I think it's better to use the actual libx264 flags and specify the actual conversions being done instead of letting ffmpeg "guess" or "auto".....and hope it's right.

[poisondeathray]

If it works 99% of the time, that means it fails 1% of the time

haha , true .

But it's actually understating the success. It will work on 99.999% of youtube AVC streams, because they are generic. Let's say I haven't seen one case of AVC on YT where it won't work. That 1% is fudge factor so I don't have to recant the statement.

For other streams, other sources that don't seem to work, sneaker already mentioned but it actually works better on elementary streams. And if that still doesn't work, then the mp4box golgol patch will tend to work. If neither works, I suspect there was a problem with the bitstream in the first place , and you would have to re-encode.

Last resort is having to re-encode. Because of quality loss, and slow. If you encoded say a feature film in 1-2 hours (maybe much longer depending on the quality and settings used), it might take 2-3 minutes to flag by re-wrapping instead. It should be even faster if in-place editing was allowed (neither ffmpeg or mp4box are capable, you're limited basically by your storage speed in re-wrapping) . Let's say you had a batch of unflagged videos (oops maybe you forgot) . The time difference adds up.

[chris319]

You raise an interesting point.

Take a random video from YouTube. MediaInfo gives no information on the color space, so color space = unknown. So you don't know how to reflag it. Might as well leave it alone.

ffmpeg doesn't fill in those fields unless you encode it with:

-color_primaries btX
-color_trc btX
-colorspace btX

Speaking of YouTube, I don't know of a browser that reads the color information from a file and adjusts its parameters accordingly. Firefox, SeaMonkey, Chrome, Chromium and Opera default to bt.601. Microsoft Edge and IE default to bt.709, the coefficients called for by sRGB, the putative standard for computer video.

I don't know what VLC does.

[poisondeathray]

Take a random video from YouTube. MediaInfo gives no information on the color space, so color space = unknown. So you don't know how to reflag it. Might as well leave it alone.

Some videos are flagged now on youtube. I'm talking about the downloaded AVC version . What you see in browser might be different unless you control that

But you cannot control whether or not YT flags on their end when it re-encodes (that's the version the public sees, not the uploaded one)

If you wanted to flag a downloaded YT AVC version, no problem - ffmpeg roozhou modified build, or the mp4box golgol build

ffmpeg doesn't fill in those fields unless you encode it with:

-color_primaries btX
-color_trc btX
-colorspace btX

But at the risk of unwanted conversions . IMO, the "proper" way is to use the actual libx264 flags explicitly, and controlling the behinds the scenes conversions, explicitly.

Speaking of YouTube, I don't know of a browser that reads the color information from a file and adjusts its parameters accordingly. Firefox, SeaMonkey, Chrome, Chromium and Opera default to bt.601. Microsoft Edge and IE default to bt.709, the coefficients called for by sRGB, the putative standard for computer video.

I don't know what VLC does.

It also depends if you're referring to HTML5 or flash player, or if you have GPU acceleration on or off. Yes, HTML5 defaults to 601 for most browsers (which is a bad mistake IMO, it should have been set to 709 given the prevalence of HD in today's HTML5 world. Even phones, devices are "HD"). In browser flash actually gets it correct with HW accel off , it obeys flags including full range/ limited range. Undefined with HW off it uses 709. Undefined with HW on it uses 601. But flash is going bye-bye like the dodo bird. Officially retired in 2020 (?) or something like that.

VLC depends on how you have it configured, which renderer, GPU on/off, driver settings, same with other video players. Multiple points of possible failure for end clients. The best you can do if producing video is get it correct on your end, and flag it correctly

[chris319]

bt.601 was never intended for computer video.

sRGB (709 coefficients) is supposed to be the computer video standard so they should stick with it.

That's what happens when computer people and mathematicians get their hands on video.

Adobe Crash Player needs to die a merciful death.

[poisondeathray]

bt.601 was never intended for computer video.

sRGB (709 coefficients) is supposed to be the computer video standard so they should stick with it.

That's what happens when computer people and mathematicians get their hands on video.

But the "powers that be" should have figured this out by now don't you think

HTML5 was supposed to be better terms of color consistency across platforms ...but it's actually not... same problems.

Adobe Crash Player needs to die a merciful death.

I agree, terrible problems.

But it's the only player that got everything correct in terms of color and flags when HW acceleration was off. There were even builds that supported 10bit video.

AVC is in decline too, especially for web. A big reason is the licensing structure. So those tools above will be less useful . VP9 and it's successors (VP10) will be the ones to look at, but right now there aren't as many tools around for manipulating things .

[poisondeathray]

So some progress ... ; VP9 709 is fixed in firefox now (HTML5) , in full 4:4:4 too if you want (chrome had it long time ago) - so all those issues remaining are mostly AVC in HTML5 (flagged or non flagged, doesn't matter)

alpha channel is also supported in VP9 now (chrome, firefox are ok); flash/AVC never supported it, you had to use VP6a - but only as yuva420p, not 4444

But as of right now, there are fewer devices, HW support for VPx , but it's starting to catch on. AVC is almost standard for HW acceleration on everything , all devices.

I don' t know of VPx specific flags like libx264 has for VUI parameters . mediainfo never shows any on VP9 streams, and neither ffmpeg or vpxenc show options for it in the full help. You can set --color-space= of the input file, but it doesn't show up in flags (at least not detected by mediainfo)

[chris319]

But the "powers that be" should have figured this out by now don't you think

That's what happens when computer people and mathematicians get their hands on video.

In Windows, these programmers don't seem to be dealing with DXVA2, the Linux equivalent of which is VDPAU.

[chris319]

I am in touch with some Mozilla people concerning AVC. They know they've got issues.

I use this to check colors:

https://www.youtube.com/watch?v=oA3-TFJv1xI

I should re-encode it for bt.2020.

[poisondeathray]

Hmm Vp9 seems fixed now. Not just your own websites, but even previous YT videos that were "wrong", now streamed as VP9 look ok now in the default YT HTML5 player.

Can you check again on linux for chrome and firefox on YT ? Force a VP9 stream. I think it's fixed for VP9 across the board for HTML5 (problem still with AVC, but hey maybe it's a conspiracy to phase AVC out )


DXVA2 isn't exclusively used in Windows, sometimes CUVID or QS are used for HW . So part of the blame has to be allocated to HW manufacturers and their driver writers . Eitherway you'd think those folks would have go together and fixed things by now

[chris319]

I don't rely on "looks OK". I take measurements.

I'll have to encode the test pattern to VP9.

What's a good "eyedropper" program for Linux (color reader)?

[poisondeathray]

Not sure about color pickers for linux, I don't use linux much. But "Looks ok" is usually +/-2 . "Looks clearly wrong" as in wrong matrix - you shouldn't have to measure, it should be obviously wrong.

Actually don't bother, all the bug reports are reported as fixed for VP9 . I saw your comments in the mozilla AVC bug report, that one is still open. And if the conspiracy theory holds, it will stay that way

[poisondeathray]

And you shouldn't have to encode to VP9 unless you were testing directly (ie. not in YT), because YT should encode a VP9 version. There's that word... should.

Another thing I don't get is why some YT videos don't get VP9 treatment, but others do . For example, your video didn't - it's not even an available option. I've given this thought for a year now - at first I thought it was age of upload, but that theory is gone because they started to convert older videos too to VP9. Then maybe resolution, nope, that theory didn't pan out. Then maybe "active" users, like users that have many views - nope, that didn't pan out either. Or monetized vs. non (nope didn't explain it) . Or upload version (e.g. upload a vp9 vs. hevc, vs. h264 version) , nope. Or audio vs. no audio - nope. It almost seems random, I cannot explain it. Sometimes it takes a day or so for the VP9 version to show up, but your link has been up since Mar 2017