Inaccurate YUV -> RGB Conversion.

[jagabo]

That test pattern is limited range rec.709, not full range. That's why your second conversion equation gives the colors you expect (it's for limited range YUV). The first equation is for full range YUV.

With limited range YUV full black (RGB 0,0,0) is defined as Y=16, full white RGB 255,255,255) as Y=235. A video file may contain Y values less than 16 or greater than 235 but they lead to illegal RGB values.

[chris319]

That test pattern is limited range rec.709, not full range.

Please explain. The test pattern as modified today contains video from #000000 through #FFFFFF. Does that not make it full range? I'm getting accurate reproduction of the #000000 and #FFFFFF patches.

I could make a colored patch, say yellow, which is #FFFFFF - #FFFFFF - #000000 but I kind of doubt the reproduction will be inaccurate. It will have to wait until later.

The #000000 is toward the lower right.

[poisondeathray]

Look at a Y waveform monitor - it's a limited range test video . Unless YT screwed something up ? (e.g. if you upload a full range video, flagged full range, YT will clamp it.)

For example the "0" patch is YUV 16,128,128 . The patch is YUV 237,127,127

If you make a graphic/design in RGB, you have to convert it to a YUV video using a full range matrix (ie. mapping RGB 0-255 to YUV 0-255)

[chris319]

There could be something else going on here. What screwed me up previously was that I was reading pixels off the screen using an eyedropper program after the samples had been decoded, run through DirectX, etc. It is convenient but turned out to be inaccurate as we learned. The accurate way is to read YUV samples directly from the .mp4 file. Forget YouTube; I have the local files and can make them available as I have done in the past.

What I could do is make a .bmp file which is a 1280x720 raster of a single color and encode it to .mp4 using ffmpeg. I could then have my program read the pixel at, say, x=640 and y = 360, which would be the center of the raster. I could decode it using the full-range code posted earlier and see how the RGB values look.

I have to look up the "full range" code for ffmpeg.

Thank you, PDR, for chiming in. You always make me think.

[jagabo]

That test pattern is limited range rec.709, not full range.

Please explain. The test pattern as modified today contains video from #000000 through #FFFFFF.

But those became Y values from 16 to 234 -- because a limited range matrix was used. Upon converting back to RGB with a limited range matrix the Y=16-235 range is stretched back to RGB=0-255.

You can see that your RGB 255,255,255 patch has mapped to Y=234:

[Attachment 47091 - Click to enlarge]

And your RGB 0,0,0 patch has mapped to Y=16:

[Attachment 47092 - Click to enlarge]

Read up on video waveform monitors (sometimes inappropriately called histograms) if you don't understand what those images are showing. Those were made with AviSynth. The bottom of the graph (which is 256 lines high) is Y=0, the top Y=255. Between Y=0 and Y=15 are marked in brown. So is Y=236 to Y=255.

[poisondeathray]

I have to look up the "full range" code for ffmpeg.

e.g using ffmpeg zscale with bmp input, converted to 4:2:0 full range using 709 (with full range and 709 flag metadata)

ffmpeg -r 24 -i INPUT.bmp -vf zscale=matrix=709:rangein=full:range=full,format=yuvj420p -c:v libx264 -crf 16 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709:force-cfr -an -frames:v 24 ffmpeg_yuvj420p_fullrange_709.mp4

Full range for consumer video would be uncommon. Is your camcorder limited range, but just with over and undershoots; or is it truly full range ?

The most common for consumer video by far is actually limited range, but with overbrights only. ie. Y = 16-255

[chris319]

Thanks, PDR, I will use that ffmpeg code to encode my bitmaps, adding loop 1 and t 10.

Rarely do I see blacks below 16 from this camcorder but I definitely see whites above 235. Will mess with this some more when I'm back on my regular computer and will let y'all know later.

[chris319]

I made a green raster .bmp file. The input colors are R=16, G=180, B=16. I then encoded it with PDR's code:

ffmpeg -y  -loop 1 -t 10  -i raster.bmp    -r 24  -vf zscale=matrix=709:rangein=full:range=full,format=yuvj420p -c:v libx264 -crf 16 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709:force-cfr -an -frames:v 24 raster.mp4

and decoded it with this code:

rf = yf + 2*(vf-128)*(1-#Kr)

gf = yf - 2*(uf-128)*(1-#Kb)*#Kb/#Kg - 2*(vf-128)*(1-#Kr)*#Kr/#Kg 

bf = yf + 2*(uf-128)*(1-#Kb)

If rf > 255: rf = 255:EndIf
If gf > 255: gf = 255:EndIf
If bf > 255: bf = 255:EndIf

If rf < 0:rf = 0:EndIf
If gf < 0:gf = 0:EndIf
If bf < 0:bf = 0:EndIf

I read the Y, Cb, Cr values directly from the .mp4 file, bypassing the Windows graphics infrastructure. Results:

R = 28, G = 171, B = 28.

I then made the following change to PDR's ffmpeg command:

yuvj420p

was changed to

yuv420p

, omitting the "j".

Now I get:

R = 16, G = 179, B = 16.

Close enough.

[jagabo]

In you original post all those (255/219) and (255/112) were about scaling limited range luma and chroma to full scale. For example, full range luma has black at 0, white at 255. But limited range luma has black at 16, white at 235. 235 - 16 = 219. Hence limited range luma (after subtracting 16) needs to be multiplied by a factor of 255/219 to get the equivalent full scale range.

[poisondeathray]

I read the Y, Cb, Cr values directly from the .mp4 file, bypassing the Windows graphics infrastructure. Results:

R = 28, G = 171, B = 28.

I then made the following change to PDR's ffmpeg command:

yuvj420p

was changed to

yuv420p

, omitting the "j".

Now I get:

R = 16, G = 179, B = 16.

Close enough.

yuvj420p is more correct for full range data, because it's flagged as full range . The flag is supposed to indicate to the decoding hardware/software that it is full range data . Of course not all are setup to handle it correctly; some ignore, some obey, some do other things entirely different

yuvj420p

Color range                              : Full
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

yuv420p

Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

The conversion underneath is actually the same, it's about how the receiving application handles it . So if you are getting those RGB values, this implies the way you are decoding/reading the file and/or converting to RGB is non standard

The application is supposed to use one set of equations if it "sees" the flag, another if it doesn't . Full range vs. Limited range.

[chris319]

OK, got it now, thank you. I had to change the line in my program where ffmpeg loads the .mp4 file:

pipeIn$ = "-i " + filename$ + " -f image2pipe  -s 1280x720  -pix_fmt yuvj420p  -vcodec rawvideo -"

Now VLC does not play the video but SMPlayer does.

[chris319]

The program submission specs for the big-shot broadcasters I've seen ask for 10-bit 4:2:2 YUV. ffmpeg does not support the "j" version of this (YUVj422p10le) so it will have to be YUV422p10le.

[poisondeathray]

The program submission specs for the big-shot broadcasters I've seen ask for 10-bit 4:2:2 YUV. ffmpeg does not support the "j" version of this (YUVj422p10le) so it will have to be YUV422p10le.

For broadcast - 100% of them will not want full range in the first place . The legal range will be Y 64-940 CbCr 64-960 for Bt709 . But some places are using Bt2020 now

For 10bit422 - most submission formats will likely be prores preferentially , a few places might are ok with avc-intra , but each place will have very specific guidelines

[chris319]

The preferred limits are 5 and 246 (8 bit) when overshoots are taken into account per EBU r 103. White and black are still 16 and 235.

https://tech.ebu.ch/docs/r/r103.pdf

No ProRes, but they do accept H.264 now. If you know of a broadcaster/webcaster that accepts ProRes, please name them.

[poisondeathray]

It depends on what you are doing. If you are sending mezzanine material for features, they usually want prores only. For ENG stuff, shorts, commercial - they usually perferred XDCAM HD 8bit422 long time ago in the past, for both Europe and North America, but that has changed , AVC-I (that's h.264 by the way, but in I-frame format) is usually equally accepted for many years too. A lot of it had to do with the way the hardware infrastructure and edit features were set up. Many stations had significant $ investments in doing things a certain way and with certain brands and cameras (e.g. Panasonic 's AVC-Intra was popular in the USA for ENG, but Sony's XDCAM was popular in Europe, especially UK) , but that has changed with cross platform solutions now - things tend to be more flexible with formats for ingest. Things have gotten much more relaxed, especially for web delivery platforms

There was a time when things were very very strict, specific "approved" acquistion cameras with certain specs had to be used for some stations, certain people got blacklisted for "cheating" etc... Some are still quite strict, e.g. BBC is very strict

Just give them what they want. If they don't like it, you'll get a sheet telling you where it got rejected spec wise and you have to resubmit . But 100% of them will not accept full range for submission (it's ok for acquisition)

[chris319]

I'm talking about programs which are submitted by producers to a network or major syndicator such as Warner Bros. for distribution to multiple stations. There are many sitcoms among the titles. There are also national commercial spots.

ENG is generally not distributed to individual stations but is usually run by the station or network that shot/edited it. Most of the material remains in-house.

PBS is very strict. If your show doesn't meet their technical standards, they reject it regardless of content or how much was spent to produce it.

[poisondeathray]

I'm talking about programs which are submitted by producers to a network or major syndicator such as Warner Bros. for distribution to multiple stations. There are many sitcoms among the titles. There are also national commercial spots.

Distribution format to stations is later downstream, not what an original content creator / producer / or production studio usually submits as a "master"

The original program submission is usually a higher quality "master" format , usually without breaks , commercials. Afterwards, there are often different edited down versions for commercials, etc..for different markets . Some are standards converted (e.g. 59.94Hz areas like North America vs. 50Hz areas like many parts of Europe ) - those sent out versions are appropriately lower bandwidth, lower quality. Prores would be inappropriate at that stage because of the bandwidth. But that part is usually handled internally, so they should know what they want. But various productions, independents, or external submissions could involve many different parties around the world - that's why a spec sheet is available for content submissions. For example, the submitted video master for BBC Worldwide main programs is Prores HQ. Here in Canada for the CBC, it's either Prores HQ or XDCAMHD422

Many web CDN platforms , VOD , etc.. request prores too. Because they have multiple destination end formats , different resolutions , even different codecs required for clients - and prores is typically used as the high quality "master" from which everything else is derived.

[chris319]

BBC and CBC are ahead of the U.S. if they're accepting ProRes. Here in the U.S. you can either safely submit an mpeg2 .mxf file. Some are now accepting H.264 in an .mp4 file. Shows are distributed by satellite using either Pathfire or Pitchblue with built-in commercial breaks according to a format which is uniform for all stations running that show. The breaks have to be "scripted in" according to the program content.

[chris319]

Here is a brand new set of specs:

https://www.tvtechnology.com/news/new-dpp-spec-focuses-on-uhd-delivery

[poisondeathray]

In North America , the most common for "edited" broadcast formats is still 50Mbps 4:2:2 CBR 8bit MPEG2 (Basically XDCAMHD422 MXF) - that's the "common currency" . Actual broadcast is still MPEG2 only, and most of the infrastructure and equipment is setup to handle that only. In Europe, they are ahead in the sense they have been using h264 for actual end delivery for years (but they also use lower bandwidth, negating the quality "benefit" . But I guess from broadcaster point of view that's good.)

In quick turn around scenarios, XDCAMHD422 is still preferred in North America because most of the equipment is set up to smart render it - the edits are passed through quickly (basically stream copied, with only a few frames re-encoded). You could argue that accepting prores is not necessarily "ahead" . It takes ~3x more bandwidth so transmission is slower, costs more money, and needs to be re-encoded also, which takes time, also much slower. From the end user point of view, they can't tell the difference quality wise in the final distribution format (at least most of them) . From a content producer perspective, you typically want higher quality at every stage, but the reality is many tradeoffs

[pandy]

I read the Y, Cb, Cr values directly from the .mp4 file, bypassing the Windows graphics infrastructure. Results:

R = 28, G = 171, B = 28.

I then made the following change to PDR's ffmpeg command:

yuvj420p

was changed to

yuv420p

, omitting the "j".

Now I get:

R = 16, G = 179, B = 16.

Close enough.

You can force ffmpeg to full quantization range by preceding input file '-i' with option '-color_range 2' . ffmpeg doesn't provide absolute colour accuracy (it may silently use automatic color space conversion).

Will check avisynth equation for those erroneous results (as you reporting).

The preferred limits are 5 and 246 (8 bit) when overshoots are taken into account per EBU r 103. White and black are still 16 and 235.

Consumer equipment usually clamping (saturate) Y values between 16 and 235 and Cb,Cr between 16 and 240 anyway (so 'blacker than black' and 'whiter than white' will be lost).
There is very limited usage for values out of this range (some signals like 'Pluge', perhaps live layer mixing).
With modern electronics filters are also usually without under and overshoots.

[chris319]

With modern electronics filters are also usually without under and overshoots.

There is plenty of high-frequency ringing.

[pandy]

With modern electronics filters are also usually without under and overshoots.

There is plenty of high-frequency ringing.

Low quality filters (sinc based).

Tested bellow equations - looks fine - only Kr and Kb coefficients are used, error is around 1LSB (quantization) thus clamp is required anyway. It assume normalized YPbPr - digital YCbCr can be easily converted to YPbPr .
Translated this from spreadsheet - hope without errors, didn't simplified anything, brackets are excessive to split every part.

R=Y+Pr*2*(1-Kr)
G=Y-(2*Pb*(1-Kb)*(Kb/(1-Kr-Kb)))-(2*Pr*(1-Kr)*(Kr/(1-Kr-Kb))) 
B=Y+Pb*2*(1-Kb)

[chris319]

Tested bellow equations - looks fine - only Kr and Kb coefficients are used, error is around 1LSB (quantization) thus clamp is required anyway. It assume normalized YPbPr - digital YCbCr can be easily converted to YPbPr .
Translated this from spreadsheet - hope without errors, didn't simplified anything, brackets are excessive to split every part.

R=Y+Pr*2*(1-Kr)
G=Y-(2*Pb*(1-Kb)*(Kb/(1-Kr-Kb)))-(2*Pr*(1-Kr)*(Kr/(1-Kr-Kb))) 
B=Y+Pb*2*(1-Kb)

I don't know what you think you've accomplished there but it doesn't work. Green comes out as magenta.

[pandy]

I don't know what you think you've accomplished there but it doesn't work. Green comes out as magenta.

Tested with colorbars stimulus and i get proper values.

[chris319]

Please explain what you think you've accomplished.

I am reading Y, Cr and Cb values directly out of the file and it doesn't work. Is there a conversion to Pr and Pb that you neglected to post?

[pandy]

Please explain what you think you've accomplished.

I am reading Y, Cr and Cb values directly out of the file and it doesn't work. Is there a conversion to Pr and Pb that you neglected to post?

Yes, you need to normalize your signal to use directly those formulas - normalization is a process of "restoring analog" signal i.e. "removing quantization to made signal like before quantization" - you can do this by "inverting quantization" so your current sample must be divided by maximal digital value present in your system - e.g. if your system is 8 bit then analog values between 0 and 1 can be expressed by quantized levels from 0 to 255 and to normalize it back to normalized value you must divide each sample by 255. So if your digital input data are within 0 and 255 then you must divide them by 255, if they are within 16 and 235 then first you must subtract from sample value representing your 0 i.e. then divide your sample by 219 (235-16).
In your case perhaps it will be safe to assume that your YCbCr is full scale thus divide every sample by 255. Remember that normalized Cb and Cr are within -0.5 and 0.5 thus their '0' (zero) is shifted by offset equal to half of your maximum (127.5 in case of 255) and you need to subtract this from your Cb, Cr values first (similar rule to removing '0' i.e. offset from Y signal).

[chris319]

Please explain what you think you've accomplished.

I am reading Y, Cr and Cb values directly out of the file and it doesn't work. Is there a conversion to Pr and Pb that you neglected to post?

Yes, you need to normalize your signal to use directly those formulas - normalization is a process of "restoring analog" signal i.e. "removing quantization to made signal like before quantization" - you can do this by "inverting quantization" so your current sample must be divided by maximal digital value present in your system - e.g. if your system is 8 bit then analog values between 0 and 1 can be expressed by quantized levels from 0 to 255 and to normalize it back to normalized value you must divide each sample by 255. So if your digital input data are within 0 and 255 then you must divide them by 255, if they are within 16 and 235 then first you must subtract from sample value representing your 0 i.e. then divide your sample by 219 (235-16).
In your case perhaps it will be safe to assume that your YCbCr is full scale thus divide every sample by 255. Remember that normalized Cb and Cr are within -0.5 and 0.5 thus their '0' (zero) is shifted by offset equal to half of your maximum (127.5 in case of 255) and you need to subtract this from your Cb, Cr values first (similar rule to removing '0' i.e. offset from Y signal).

Now why would I want to do it this way as opposed to the way I already have it? Dividing each sample by 255 adds another computational step though your way of doing it saves a step, so what is the benefit of this new way? I don't see the benefit.

[pandy]

Now why would I want to do it this way as opposed to the way I already have it? Dividing each sample by 255 adds another computational step though your way of doing it saves a step, so what is the benefit of this new way? I don't see the benefit.

You doing this anyway - most if not all those formulas perform normalization - those nice scaling coefficients, numbers like 16, 235, 219, 240, 112, 128 associated with formula are used for normalization - my intention was to recover process of accurate conversion from YCbCr to RGB with only 2 standardized coefficients in use. Also provided formula was not optimized and without normalization to avoid ambiguity - call this "debug formula" - mostly for comparison and also to verify errors (as i wrote earlier - they should be within 1LSB).
You don't need to use this in your code but you can verify used code product against it (i saw earlier that you struggle with some weird coefficients or miss some parts of formula).

[chris319]

This code works:

rf.f = yf + 2*(vf-128)*(1-#Kr)
gf.f = yf - 2*(uf-128)*(1-#Kb)*#Kb/#Kg - 2*(vf-128)*(1-#Kr)*#Kr/#Kg 
bf.f = yf + 2*(uf-128)*(1-#Kb)

If rf.f > 255: rf = 255:EndIf
If gf.f > 255: gf = 255:EndIf
If bf.f > 255: bf = 255:EndIf

And this code doesn't, despite your claim of having tested it.

Pr = v / 255: Pb = u / 255: Y / 255

Rf.f=Yf+Pr*2*(1-#Kr)
Gf.f=Yf-(2*Pb*(1-#Kb)*(#Kb/(1-#Kr-#Kb)))-(2*Pr*(1-#Kr)*(#Kr/(1-#Kr-#Kb))) 
Bf.f=Yf+Pb*2*(1-#Kb)

rf * 255
gf * 255
bf * 255