Lossless Workflows using Premiere Pro, VirtualDub, ffmpeg and others

[jagabo]

I wasn't interested in a YUV -> RGB -> YUV round trip involving all possible YUV values because I was focused on how to transform the colorbars from known RGB values.

Either way, RGB-YUV-RGB or YUV->RGB-YUV, you'll find that the round trip isn't perfect for all valid colors.

[SameSelf]

Either way, RGB-YUV-RGB or YUV->RGB-YUV, you'll find that the round trip isn't perfect for all valid colors.

I am not sure what you are saying. If you are saying that there is a triad of RGB24 values that cannot be recovered from a floating point transformation to 10-bit YUV values, then provide an example. Just post back here with the values and I will push them through my calculator.

If however you are saying that the transformations that Avisynth or vdub use are lossy because of the integer math employed for speed and legacy issues, well, that is another topic that is somewhat imbedded in this discussion.

But please provide more detail or at least an example.

[jagabo]

Just post back here with the values and I will push them through my calculator.

Without the exact equations you're using I can't duplicate your results. But if you have a program that does the conversions just feed it all 16 million RGB colors and see how many don't survive the round trip intact.

[SameSelf]

Then stop making unsubstantiated conclusions. I provided a detailed example showing how 8-bit and 9-bit fail while 10-bit is sufficient. So the burden of proof is now on you.

[jagabo]

I provided a detailed example showing how 8-bit and 9-bit fail while 10-bit is sufficient.

With three colors out of 16 million. Go ahead and delude yourself.

[poisondeathray]

Asking which equations are being used is a pretty fair question. I know "proof" is being used in a generic sense , like "example", or "evidence"; but "Proof" has a very specific connotation scientific community. To me , "proof" is more like a mathematical proof. I would argue that because YUV/RGB are not completely overlapping color models, I think the burden of proof should be on showing 10bit YUV is sufficient to represent all 16.7M 8bit RGB colors

But this has been discussed before on several sites, including doom9 - and 10bit YUV is the commonly accepted bit depth required to represent all 8bit RGB colors - people have run it through matlab etc... so at least in theory 10bit is the "magic" bit depth. But I could give a rat's ass about theory. In practice, it' s more complex because different programs add another several other layers of inconsistency and problems. Recall how this thread began and progressed? YUY2 is supposed to be the same functionally as UYVY, 2Vuy, YV16, etc... right ? They are all 4:2:2 just stored differently. But different program handling resulted in different real world results. Well it just happens that 8bit RGB => 10bit YUV conversions can be handled slightly differently in the real world too

Recall I said earlier that bit depth conversions were trickier to test? To expand:

1) There are different equations used for 8bit RGB => 10bit YUV based on what model or set of assumptions you are making . For example, are you assuming standard or extended range ? Are you using ITU Rec709 or Rec1361 (wide gamut) definitions ?

2) Slight differences in how the conversion is actually done for 8 to 10bit (are values scaled linearly, or dithered, or is it just padded 8bit msb/lsb ?) Or do you convert 8bit RGB to 10bit RGB, then to 10bit YUV ? In reality, almost all bit depth conversions get dithered in professional programs, so converting back to a certain bit depth can yield different results more often than you would expect. Is dithering accounted for in the mathematical model ? There are at least a few commonly used dithering algorithms used eg. floyd-steinberg, sierra 4a, sierra lite, atkinson, stucki, etc...they will all give slightly different results

3) Discrepancy in output range. Most commercial programs don't use "extended range" 10bit YUV , especially on export. ie. You usually don't have "mappable" 0-1023 values in YUV , but you do have 10bit RGB in 0-1023 values. Valid ranges if we assume Rec709 are Y=64-940, U,V=64-960. (I'm using "U, V" , because I'm lazy, but it should really be Cb, Cr). Standard 10bit YUV "mapping" is Y 64-940 , UV 64-960 to RGB 0,0,0-1023,1023,1023. (This is even true for "professional" 10bit formats like prores - that is actually one reason why the "official" Apple implementation produces more consistent results - is it doesn't allow for extended range. That's also one of the reasons why ffmbc/ffmpeg sometimes yield more inconsistent results in some programs.) Anyways, some equations reflect these range differences, others do not. So it depends on what set of assumptions you use

4) Testing difficulties - While v210 (10bit 422 YUV) is very common in commercial production software, v410 (10bit 444 YUV) is not. v210, as nice as it is, is chroma subsampled. A RGB source is not subsampled at any bitdepth. A proper test should be using v410. But in reality, even 10bit RGB is more common (r10k, r210) than v410 . v410 is almost never used, or even available as an option. This makes actual testing difficult. But in reality, nobody really cares. They just use 8bit RGB to 8bit RGB. And there are plenty of truly lossless options for that

[SameSelf]

The equations used for the RGB -> YUV conversion are not the source of the lossiness. If you use floating point arithmetic throughout, you can always recover the original RGB values no matter what equations you use. This implies (from a mathematical proof concept) that the source of the lossiness is the 8-bit rounding applied in YUV space. Do I actually need a full 32/64 bits (even floating point arithmetic is not infinite precision)? Of course not. But a rigorous mathematical proof is not necessary here and neither is a Matlab demo. Most who closely study the example I provided will easily prove to themselves what is going on and not require a Matlab demo. Mathematical proofs are so mathematicians can talk to each other. Conceptual examples are much more useful for a forum.

Since the equations are of such interest I will post the link that I used.
http://www.equasys.de/colorconversion.html

But few programs do any matrix math as shown in the link (Avisynth may be the rare exception from what I have seen). Rather they employ LUTs which are far faster computationally. My guess is all the trickiness lies in the LUT used. For example, 10 bit LUTs that are not truly 10 bits.

I remain open to correction should anyone produce a set RGB values that cannot be recovered. But at the end of the day, this will just imply one needs to move up to 11 or 12 bits. TBH, I am not sure what all the controversy is.

[poisondeathray]

The equations used for the RGB -> YUV conversion are not the source of the lossiness. If you use floating point arithmetic throughout, you can always recover the original RGB values no matter what equations you use.

Yes, those equations are ok with enough precision. But other variations apply a clamp on the YUV => RGB conversion (representing out of gamut vaules). Anytime you clamp something the math gets dicey, and it isn't necessarily reversible. Think about it - you're asking someone to demonstrate a number that "fails" , but you didn't even provide the equation at the time. I suppose some math genius could world backwards from 1 example and deduce what equations you were using. Probably easier to post a link

But RGB => YUV is usually less of a problem, especially starting from 8bit RGB. It's YUV => RGB that is typically the problem . YUV is larger color model, recall in the color cube representation on the 1st page, that RGB fits entirely in YUV. A typical YUV source (e.g. from a camcorder) actually begins life at higher bitdepth RGB from the sensor and for in camera processing. For most consumer level cameras goes down to 8bit YUV for storage (e.g. AVCHD, HDV, AVC, HEVC , etc...) . That's why typically there are many more out of gamut values in camera sourced 8bit YUV. But if you start with 8bit sRGB , those values are already "culled" before going to any YUV conversion. You only have 0-255 values per channel to start with, none of the negative values or out of gamut values that should be present when converting from 8bit YUV

[SameSelf]

Yes, those equations are ok with enough precision. But other variations apply a clamp on the YUV => RGB conversion (representing out of gamut vaules). Anytime you clamp something the math gets dicey, and it isn't necessarily reversible. Think about it - you're asking someone to demonstrate a number that "fails" , but you didn't even provide the equation at the time. I suppose some math genius could world backwards from 1 example and deduce what equations you were using. Probably easier to post a link

But RGB => YUV is usually less of a problem, especially starting from 8bit RGB. It's YUV => RGB that is typically the problem . YUV is larger color model, recall in the color cube representation on the 1st page, that RGB fits entirely in YUV. A typical YUV source (e.g. from a camcorder) actually begins life at higher bitdepth RGB from the sensor and for in camera processing. For most consumer level cameras goes down to 8bit YUV for storage (e.g. AVCHD, HDV, AVC, HEVC , etc...) . That's why typically there are many more out of gamut values in camera sourced 8bit YUV. But if you start with 8bit sRGB , those values are already "culled" before going to any YUV conversion. You only have 0-255 values per channel to start with, none of the negative values or out of gamut values that should be present when converting from 8bit YUV

I hate to get into long diatribes about mathematics, but a few points.

First, I should have made this point sooner: let's stop talking about "equations." This is just a simple transformation. The only material information is the constants used. I could just as easily use transcendental constants:

[Pi^3
e^-Pi
-Pi/e]

in the matrix. The "equations" won't change, only the constants, and my example would likely still apply because I am working with integer RGB and YUV values versus real values.

Second, I have never talked about converting full gamut YUV sources to RGB. I am well aware of the out-of-gamut YUV colors in RGB space. That is a separate issue. This thread is about converting RGB color bars to YUV, not how YUV camcorder sources are clamped that must go through RGB filters. But that is a interesting discussion none-the-less because I am curious what the source of these out-of-gamut YUV colors is (see point 4 below).

Third, is it correct to say that YUV is a "larger" color space than RGB? To wit, they are both 8-bit, and each have 16.8 million colors mappable to a LUT. One could argue that 8-bit YUV is too small when using clamped ranges because now YUV only has 11.1 million mappable colors. Plus, I am not aware of any camcorders that record to YUV444. The fact that RGB "fits" entirely into the YUV space on page one is a consequence of the transformation. Engineers could have easily defined the transformation is such a manner where we would be complaining about out-of-gamut RGB values.

Fourth, if CMOS sensors actually start with RGB before "culling" for YUV, then how on earth do we ever get out-of-gamut YUV values? IOW, out-of-gamut YUV values would then only exist on paper, not in real life. Otherwise, there is a flaw somewhere in the process described.

[poisondeathray]

You originally were talking about 8bit RGB, to YUV at "x" bitdepth, back to recover 8bit RGB. In equations that clamp, is 10bits enough ? Certainly 10bits is enough for equations on that page, because no clamping is done

In 8bits, it is correct to say that YUV is a larger color space than RGB. Because all 8bit RGB colors can be represented in 8bit YUV, but not all 8bit YUV values can be represented in 8bit RGB. Sure, an engineer could have defined a "superduper sameself" colormodel, but we're talking about real standards here , real consequences and data loss. There are professional cameras that can export 10bit RGB or YUV (e.g. prores 4444 in YUV), even RAW

Upstream is processed at a higher bit depth. DSP conversions are typically done at 10-14 bits in consumer camcorders, even higher in professional cameras. You actually start with "RAW" sensor data this is eventually debayered to RGB. You can get RAW video recordings out of professional cameras with external recorders, and apply your own debayering algorithms, or use third party ones. Out of gamut values exist in real life in >90% of consumer cameras . There is a script by gavino that detects out of gamut values, run any random 8bit consumer video through it and you will be surprised at how much is getting lost

[poisondeathray]

Here is the link to gavino's thread "U and V ranges for valid RGB"
http://forum.doom9.org/showthread.php?t=154731

As for YUV444 recording, there are several Panasonic models that can record AVC Ultra 444 at 10 or 12bits, and several Sony models that can record a XAVC variant at 444 10 or 12 bits onboard to cards (no external recorder) . Of course no consumer level camera will record that right now, typically they are still 8bit, 4:2:0 . But those still contain non valid RGB values

One of the comments in that thread asked about xvYCC, which is present in some Sony consumer AVCHD cameras. xvYCC is the consumer name for ITU Rec 1361 - it's also known as "wide gamut RGB." Fewer values are lost because there are some negative RGB values that would have normally been thrown out are kept. But it still doesn't map all 8bit YUV values. In reality , it's more of a marketing gimmick, because you need the full chain to be xvYCC complaint, the player (eg. BD), the TV as well. Not very many setups have this. The newer one is ITU Rec2020 for UHD, it's even larger. Wide gamut RGB basically never caught on in mass, but Rec2020 will because it will eventually be the new standard like Rec709 was for HD

[SameSelf]

The clamping in YUV space is not the sole source of the lossiness otherwise 8 bits would be enough with clamping removed. But that is not the case. Moving to 10 bits is enough and my example uses clamping. I think we are on the same page in that regard, right?

By camcorders I meant consumer versus pro. Sorry if I wasn't more clear.

Thanks for the doom9 thread. That is an interesting read. I think I will start hunting for out-of-gamut YUV values in my own video.

[poisondeathray]

The clamping in YUV space is not the sole source of the lossiness otherwise 8 bits would be enough with clamping removed. But that is not the case. Moving to 10 bits is enough and my example uses clamping. I think we are on the same page in that regard, right?

The clamping occurs in RGB (it refers to clamping out of gamut RGB values, not full range vs. limited range YUV), but yes I think we're on the same page. You would expect the RGB=>YUV direction to work just from looking the color cube diagram . And intuitively, it wouldn't make sense, that when starting with RGB, you suddenly get YUV values that become invalid when going back to RGB, since the starting RGB values were valid to begin with since 8bit YUV can represent all 8bit RGB values. Unfortunately, it usually doesn't work perfectly in real life because of some of the reasons mentioned earlier.

Math isn't my strong point you can probably tell, but I think going from 8bit RGB to 10bit YUV back to 8bit RGB require different set of equations (or transform if you like), because the range in 10bit is different, the offsets are different. I mentioned this earlier, but the valid ranges for 10bit are Y 64-940 CbCr 64-960

The equations for 10bit to 10bit (assuming Rec 709) are as follows.

The RGB to YcbCr Matrix-Multiply And Offset Equations are as follows:
Y = (Y Range/RGB Range)(0.299R + 0.587G + 0.114B) + 64
Cb = (Cb Range/RGB Range)(-0.1687R - 0.3313G + 0.5B) + 512
Cr = (Cr Range/RGB Range)(0.5R - 0.4187G - 0.0813B) + 512
Since the Y Output Range is 876 (940-64), the Cb/Cr Output Range is 896 (960-64), and the RGB
Input Range is 1023 (1023-0), the equation for this application becomes:
Y = (0.256R + 0.50267G + 0.0976B) + 64
Cb = (-0.14779R + -0.2902G + 0.4379B) + 512
Cr = (0.4379R + -0.3667G + -0.0712B) + 512

The YcbCr to RGB Matrix-Multiply And Offset Equations are as follows:
Y’ = Y - 64
Cb’ = Cb - 512
Cr’ = Cr - 512
R = (RGB Range /Y Range)(1Y’) + (RGB Range /Cr Range)(1.402Cr’)
G = (RGB Range /Y Range)(1Y’) + (RGB Range /Cb Range)(-0.3441Cb’ - 0.714Cr’)
B = (RGB Range /Cr Range) (1Y’) + (RGB Range /Cb Range)(1.772Cb’)
Since the RGB Output Range is 1023 (1023-0), the Y Input Range is 876 (940-64), and the Cb/Cr
Input Range is 896 (960-64), the equation for this application becomes:
R = (1.1678Y’ + 0 + 1.6007Cr’)
G = (1.1678Y’ - 0.3929Cb’ - 0.81532Cr’)
B = (1.1678Y’+ 2.0232Cb’ + 0)

A ratio is used for the input/output ranges (you can see it described above) which modifies the constants. The Y,CbCr ranges should be the same at 10bit, but I'm not sure if just plugging in 8bit RGB (255-0=255) is valid for the RGB input/output ranges. If it do plug it in and modify it, there are values that cannot be recovered, or maybe it's just late and my spreadsheet is full of errors

[jagabo]

I had a chance to fully test this. Using the matrices from SameSelf's link in post #97 above, with only 3 digits to the right of the decimal place, does not preserve all 16 million RGB values on a round trip from 8 bit RGB to 10 bit YUV and back to 8 bit RGB. About 5000 RGB vales were changed. But using higher precision matrices I was able to make the round trip with no errors. Some details that SameSelf didn't spell out were how he converted floating point values to integers and vice versa. Many compilers and CPUs will truncate floats rather than round. So a value like 100.9 would become 100, not 101. I made sure all my conversions rounded rather than truncated. Another detail left out was what constitutes the range of 10 bit values. Most programs I've seen convert 8 bit integers to 10 bit integers by multiplying by 4. But that leaves the top three 10 bit values unused -- ie, 8 bit 255 becomes 10 bit 1020, leaving 1021, 1022, and 1023 "unused". Of course, the floating point intermediates may end up generating those values. To fully utilize the range of 10 bit YUV you would have to multiply by about 4.0118. I used 4.0 in the following code. These types of issues are why I asked for the exact equations used.

/************************************************************************/
//  Convert an 8 bit RGB value to a 10 bit YUV value. Rec.601.
/************************************************************************/

rgb_to_yuv(int R, int G, int B, int *Y, int *U, int *V)
{
	double fr, fg, fb, fy, fu, fv;

	fr = (double)R;
	fg = (double)G;
	fb = (double)B;

	// higher precision matrix from wikipedia
	fy = ( 0.256789*fr + 0.504129*fg + 0.097906*fb) +  16.0;
	fu = (-0.148223*fr - 0.290992*fg + 0.439215*fb) + 128.0;
	fv = ( 0.439215*fr - 0.367789*fg - 0.071426*fb) + 128.0;

	*Y = (int)(fy * 4.0 + 0.5); // + 0.5 to get rounding rather than truncation
	*U = (int)(fu * 4.0 + 0.5);
	*V = (int)(fv * 4.0 + 0.5);
}

/************************************************************************/
//  Convert a 10 bit YUV value to an 8 bit RGB value. Rec.601.
/************************************************************************/

yuv_to_rgb(int Y, int U, int V, int *R, int *G, int *B)
{
	double fr, fg, fb, fy, fu, fv;

	fy = (double)Y / 4.0 - 16.0;
	fu = (double)U / 4.0 - 128.0;
	fv = (double)V / 4.0 - 128.0;

	// high precision values from wikipedia
	fr = (1.16438 * fy                 + 1.59603 * fv);
	fg = (1.16438 * fy - 0.391762 * fu - 0.81297 * fv);
	fb = (1.16438 * fy + 2.017234 * fu               );

	*R = (int)(fr + 0.5);
	*G = (int)(fg + 0.5);
	*B = (int)(fb + 0.5);
}

/************************************************************************/
//  Test all 16 million 8 bit RGB values through a round trip
//  from 8 bit RGB to 10 bit YUV and back to 8 bit RGB.
/************************************************************************/

test_round_trip()
{
	int r, g, b, y, u, v, nr, ng, nb;
	int count, diff;

	printf("\nTesting round trip...\n");

	count = 0;
	diff = 0;

	for (r=0; r<=255; r++)
	{
		for (g=0; g<=255; g++)
		{
			for (b=0; b<=255; b++)
			{
				rgb_to_yuv(r, g, b, &y, &u, &v);
				yuv_to_rgb(y, u, v, &nr, &ng, &nb);
				if ((r!=nr) || (g!=ng) || (b!=nb))
				{
					//printf("%d, %d, %d -> %d, %d, %d -> %d, %d, %d\n", r, g, b, y, u, v, nr, ng, nb);
					diff++;
				}
				count++;
			}
		}
	}
	printf("%d colors tested\n", count);
	printf("%d colors differed\n", diff);
}

/************************************************************************/

Note that the above code is using 16 or 32 bit integers (depending on your compiler) to hold the 8 and 10 bit values. Other parts of my test program tested for bit overflow. For example, does the returned 10 bit value exceed 1023, or is the returned 8 bit value less than 0? Using the above equations, and the compiler I built with, there were no bit overflows or underflows. I tested both limited (shown) and full (not shown) range rec.601. My guess is rec.709 will work similarly.

That should be enough for anyone else who wants to test for themselves.

[shekh]

Btw, in official vdub rgb24 -> v210 -> rgb24 fails (has too much error).
Hope somebody cares

[SameSelf]

Well done, jagabo! I am impressed. You did what I was unwilling to do. I applaud your programming chops. You are clearly more than just a video buff.

I know there are problems with the matrix coefficients in the link posted above (q.v. the second row of the Rec.709 matrix does not add to zero), but at the end of the day that is not germane to my original point: 10 bits is sufficient for an RGB24 -> YUV -> RGB24 pathway in 32/64-bit floating point space. Should anyone find examples otherwise, it is for reasons unrelated to scaling up to 10 bits e.g. rounding vs truncating, using only 3 sig figs in the matrix coefficients, etc. It may sound pedantic, but anything else results in concluding incorrectly that 10 bits is not enough. So, I think we are on the same page now if we were to say, 11-bit is not needed.

Since most programs use LUTs instead of matrices, the coefficients are immaterial because I assume that problems can just be fixed in the LUT (although it is quite possible many don't and thus causing many problems, q.v. the vdub comment above). Therefore, the question is how large does the YUV cube need to be? 9-wide? 10-wide? A layman's way of thinking about the problem is asking what is the maximum number of RGB24 values that collapse to a single 8-bit YUV value? If the number is two, then expanding the YUV cube to 9x9x9 should be sufficient. It has been a while, but I recall that I couldn't find four RGB values collapsing to a single 8-bit YUV value. The most was three. So doubling to 9-bit is not quite enough. But quadrupling to 10-bit is actually a little too much. Since computers don't operate in threes, we are left needing a 10-bit LUT. So while it might sound like losing the values of 1021, 1022, and 1023, is a bit of a compromise that could cost us down the road, not really since quadrupling is actually a little overkill. And with a LUT, I could easily claim those numbers if I needed them.

Great discussion and thank you for taking the time to program this question up.

[shekh]

Since most programs use LUTs instead of matrices, the coefficients are immaterial because I assume that problems can just be fixed in the LUT (although it is quite possible many don't and thus causing many problems, q.v. the vdub comment above)

If you want to know, the vdub problem is simple error in math. There is nothing wrong with matrix way itself.

[jagabo]

With limited range rec.601 less than 1/6 of the 8 bit YUV cube maps to valid 8 bit RGB colors. So, on average, about 6 RGB colors map to the same YUV value. You can see it visually here:

https://software.intel.com/en-us/node/503873

See the image "RGB Colors Cube in the YCbCr Space":

[SameSelf]

With limited range rec.601 less than 1/6 of the 8 bit YUV cube maps to valid 8 bit RGB colors. So, on average, about 6 RGB colors map to the same YUV value. You can see it visually here:

https://software.intel.com/en-us/node/503873

See the image "RGB Colors Cube in the YCbCr Space":

Admittedly, I have focused on Rec.709 vs Rec.601, but I don't think my observations would differ that dramatically given that the only "real" difference between the two color spaces are the red and blue weights (which btw is the only real requirement, i.e. the weights sum to unity). Even so, if six nearby RGB24 triads return the same YUV triad in 8-bit space, then we would need higher than 10-bit to separate those six RGB24 triads into six distinct YUV triads. While it may not be obvious, the transformation is linear.

Can you provide an example of six RGB24 triads that collapse to a single YUV triad in 8-bit space?

[jagabo]

How else are you going to fit 16 million RGB colors into 2.6 M YUV colors? Here's an image with seven blocks near RGB=3,3,3 that all come back as 3:3:3 on a round trip in AviSynth (with both rec.601 and rec.709).

ImageSource("333.png") 
StackHorizontal(last, ConvertToYUY2().ConvertToRGB24())

And they all map to 19,128,128 YUV in AviSynth.

[shekh]

x6 in volume is less than x2 in single dimension

[SameSelf]

jagabo, thanks. I will be testing that once I get some time. Super busy right now.

[SameSelf]

jagabo, thank you for posting the png example. You are right. All seven of those black squares collapse to YUV [19,128,128]. And [4,4,4] collapses to [19,128,128] as well. So that is a total of eight RGB24 values that collapse to the same YUV value in 8-bit space. However, all of them transform to unique YUV values in 10-bit space, and I can recover all of the original RGB values.

So I decided to look into your code to see why you are unable to recover 5000 RGB values as stated above. I am not certain, but I think the problem may be due to the fact that you don't have enough precision in your inverse matrix. The zero elements aren't truly zero, and sparse matrices sometimes cause problems.

Try this and see if you can recover all RGB values:

// higher precision values
	fr = (1.1643829236258 * fy - 5.31270171792572E-06 * fu + 1.59602583236107     * fv);
	fg = (1.1643829236258 * fy - 0.391760947437725    * fu - 0.8129702070444      * fv);
	fb = (1.1643829236258 * fy + 2.01723509196775     * fu - 2.03859633058533E-07 * fv);

EDIT: I also did some rough hand calcs, and the RGB box in 8-bit YUV space is roughly 2.8–2.9 millions colors which is close to the 2.4 million you quoted above. In 10-bit space, the RGB box expands to ~180 million colors, more than 10x than what is needed. It is worth noting that in 9 bit space the RGB box is roughly 22 million colors which would imply that 9-bits is plenty. And indeed, the eight RGB values above, that collapse to a single 8-bit YUV value, map to unique 9-bit YUV values. However, recovering the original RGB values from the 9-bit YUV values is problematic.

[jagabo]

So I decided to look into your code to see why you are unable to recover 5000 RGB values as stated above. I am not certain, but I think the problem may be due to the fact that you don't have enough precision in your inverse matrix.

I already said that was the problem with the low precision matrices given at http://www.equasys.de/colorconversion.html. And as I indicated, with the higher precision code I posted all 16 million colors survived the round trip intact.

[SameSelf]

But I was trying to point out that I don't think the problem is due to low precision in the initial matrix. I recover all RGB values, using only the 3 sig figs from the link for the initial matrix coefficients. This is why I stated earlier that it doesn't matter what coefficients you use in the initial matrix. It all comes down to how much precision you retain in the intermediate calcs i.e. the inverse matrix. Maybe this wasn't obvious before. Also, it goes without saying that if you truncate versus round to 8-bit values, you will have problems. And as pdr stated above, the valid range of 10-bit YUV doesn't extend all the way to 1,023. So multiplying by some number >4 to fully utilize the 10-bit YUV range is moot. But I did check to truncate to inside the valid range if a YUV value went outside it. But apparently none did because I didn't have any problems.

[jagabo]

Once again, I used the low precision values from equasys:





Otherwise the code was the same as the code I posted. About 5500 colors did not survive the round trip. I just checked the values in the code, they all matched, there were no typos. I've tested with two different x86 compilers.

[SameSelf]

Great! We are finally on the same page. I was able to mimic your workflow in Rec.601 space, and similarly, I did not recover about 5000 RGB triads from 10-bit space. But, here is the problem. As I have been saying, the YCbCr to RGB (bottom) matrix is not precise enough and slightly sparse. Maybe that is OK in 8-bit space. But, if you invert the RGB to YCbCr (top) matrix by hand and retain enough precision, you will recover all RGB values from 10-bit space. Try using these coefficients (mine are double precision and go out a little farther). As you can see, the zero elements are not truly zero even, hence the problems.

1.164144354	-0.001788898	 1.595786205
1.164144354	-0.391442764	-0.813482069
1.164144354	 2.017825510	-0.001245839

In summary, there is nothing wrong with using only 3 sig figs in the top matrix (assuming you don't have out-of-gamut problems). The key is retaining enough precision in the intermediate calcs including the inversion matrix.

[SameSelf]

I did a little more research, and here is the deal. The RGB to YCbCr matrix should be built from defined Kr and Kb constants. These constants are specified to a max of 4 sig figs (I have never seen values with more precision). Referencing the matrix values is confusing. I initially built my matrix off of Kr and Kb but got twisted off from them thinking they weren't right. Time to get back on track.

It is only when you maintain floating point precision in the initial matrix that some inversion matrix elements collapse to zero. If you round the initial matrix elements to three or four sig figs, the inversion matrix elements no longer collapse to zero (as shown in my post above). But, that is not a problem necessarily because, at this point, the initial matrix is still only functioning as defined constants. Constants can contain any number of sig figs without affecting precision (accuracy may suffer but don't confuse this with precision). Therefore, whether you maintain full precision in the initial matrix or round the values, the inversion matrix should always be full precision, or you will lose precision and potentially not recover some RGB values even in 10-bit space. So, while the link I posted was useful, it is clearly for reference only and not meant for programmatic calcs.

In the future, all this discussion can be avoided by just referencing the Kr and Kb values. Those are fairly well known for Rec.601, 709 and even 2020.

[jagabo]

But if you use more precise values in the RGB to YUV conversion you get more accurate YUV values. So it's best to use more precise values for both. Or course it doesn't matter if all your going to do is convert RGB to YUV and back to RGB.

[SameSelf]

But what does "more accurate YUV values" even mean? Recall that the RGB block in 10-bit YUV space has a volume of ~180 million colors. Therefore, only about 1/10th of the possible values will be used. As long as I can generate unique YUV values for every RGB, then why should I care if I generate YUV[75,512,514] vs YUV[75,513,514] for a given RGB value? I am not convinced accuracy is necessary which was my point above. We are not expanding out the YUV space to 10-bit based on some ITU-R spec. Is there even an "official" mapping defined for RGB24 to 10-bit YCbCr?