Lossless Workflows using Premiere Pro, VirtualDub, ffmpeg and others

[SameSelf]

Here is a recap based on the things discussed in this thread. It began with how to generate clean YUV color bars. PP has a nifty RGB bars and tone generator, but exporting clean YUV bars without a bunch of chroma subsampling artifacts is not possible. So the following workaround, suggested by pdr, works wonders:

1. In PP generate bars and tone. These will be RGB. I like to use the SD version even though my timeline/colorspace is HD/Rec.709. It is easier to QC chroma subsampling issues with the SD bars.
2. In AE dynamically link to the PP timeline with the bars and tone. From here, export as RGB24. This effectively bypasses any RGB to YUV conversion which is not possible in PP.
3. Load the RGB bars in Avisynth and convert to YUV with point resizing for the chroma. This effectively avoids messy borders between the bars from other subsampling techniques. Mind you, point resizing is not recommended with normal video, but for color bars we want clean borders. You can change the matrix to Rec601 if you require that.

AVISource(v,pixel_type="RGB24")
ConvertToYV16(matrix="Rec709", chromaresample="point")

4. Now, from here the trick is to encode as a lossless UYVY 4:2:2 format. Any other format, e.g. YUY2, results in lossy pathways through PP. There are at least two methods that I have verified:

a. Load the Avisynth script in vdub. Set the output under Video - Color Depth to 4:2:2 YcbCr (UYVY). Make sure full processing is selected under Video. Then save as an AVI (F7).
b. Use the following ffmpeg cmd:

ffmpeg -i in.avs -c:v rawvideo -pix_fmt uyvy422 -vtag "UYVY" -an out.avi

Congrats! You now have clean Color Bars that are 8-bit Y'CbCr 4:2:2.

5. Import in PP. Now, be advised that PP doesn't interpret the UYVY footage encoded from either vdub or ffmpeg correctly. In my case, I have to force PP to interpret the footage as progressive. When you look at the vectorscope of your UYVY bars, you should see the following:


[Attachment 37361 - Click to enlarge]

Now you may say, "Wait a minute. What are all those extra dots on my scope?" Well, let's look closely starting with the Red and Magenta dots. If you draw a line between them, you will see that there is a dot exactly in the middle of that line. That dot represents PP's chroma resampling from YUV 4:2:2 to RGB 4:4:4 for display. Since there is only resampling in the horizontal direction, only the vertical boundaries are effected by the rescaling. If you go through the exercise of drawing lines on the scope between each color pair that border one another vertically, you will find a dot at each mid point. This also explains why there is no dot between Yellow and Green, Red and Yellow, and so on: because those borders don't exist. Also, these chroma resampling artifacts only exist for display purposes. Internally, PP will not resample unless it is required when exporting. Which brings us to our final step for a lossless pathway through PP.

6. Export the UYVY bars as AVI (Uncompressed) and be sure to select UYVY (not V210). When you re-import this exported file, you will get back the exact same scope above and this time PP will automatically interpret the footage as progressive.

Now some of you may also ask, "Why not use Avisynth's ColorBars filter?" Well, for some reason these bars (both RGB32 and YUY2) have a color shift.

Lastly (for those of you had the stamina to wade through this entire post: congrads and thank you ), as much as I love these color bars, there is one thing that I find dissatisfying. They are 8-bit, and wading through this thread will give you an understanding why my next goal is to generate 10-bit YUV bars from the RGB bars.

Any ideas?

EDIT: I am also attaching my color bars for anyone who might find them useful, especially for those who don't have PP or AE. Note the UYVY versions are Rec.709 and 1920x1080.

[shekh]

...goal is to generate 10-bit YUV bars from the RGB bars. Any ideas?

Do you think it will be bad idea?

ffmpeg -i in.avs -c:v v210 out.avi

[SameSelf]

Just learning about v210 as all my experience thus far with 10-bit video is with ProRes and DNxHD.

Your ffmpeg cmd uses Avisynth. Why would I need to frameserve the color bars if I can just load the avi directly, -i colorbarsrgb24.avi? Speaking of this, I tried it, and there is a strange color shift in the resulting out.avi, not to mention lots of chroma subsampling artifacts. Any ideas?

[poisondeathray]

Your ffmpeg cmd uses Avisynth. Why would I need to frameserve the color bars if I can just load the avi directly, -i colorbarsrgb24.avi? Speaking of this, I tried it, and there is a strange color shift in the resulting out.avi, not to mention lots of chroma subsampling artifacts. Any ideas?

ffmpeg uses 601 by default for RGB to YUV. You can tell ffmpeg to use 709 by either using -vf colormatrix, or -vf scale using out_color_matrix , set w,h to -1 to keep input dimensions

-vf colormatrix=bt601:bt709
or
-vf scale=w=-1:h=-1:out_color_matrix=bt709

There are other scaling option flags you can play with , such as -sws_dither, -sws flags for scaling (not sure how it affects chroma scaling for 422)

Are you ok with v210 subsampled 10bit422 YUV ? v410 is 10bit 444 YUV full chroma, but few programs support it .

or you can use AE or AME to convert 8bit RGB to v210, or maybe avisynth with dither tools/stack16 or vapoursynth

Or , if you're starting with RGB, why not use RGB lossless intermediate ?

[chris319]

jagabo, what do you think? On my machine there are no color errors (compiled with GCC). Do I have it right or am I missing something?

#include "stdio.h"
#include "stdlib.h"
#include "float.h"

signed long Y, U, V;
int R, G, B, nr, ng, nb;


// BT.709 CONSTANTS
#define KR 21.26
#define KG 71.52
#define KB 7.22


/*
//BT.601 CONSTANTS
#define KR 29.9
#define KG 58.7
#define KB 11.4
*/

/*
//NTSC CONSTANTS
#define KR 30
#define KG 59
#define KB 11
*/

/*
//BT.2020 CONSTANTS
#define KR 26.27
#define KG 67.8
#define KB 5.93
*/

/************************************************************************/
//  Convert an 8 bit RGB value to a 10 bit YUV value. Rec.601.
/************************************************************************/

//void rgb_to_yuv(int R, int G, int B, int *Y, int *U, int *V)
//rgb_to_yuv(int R, int G, int B)
rgb_to_yuv()
{
float fR, fG, fB;
fR = R; fG = G; fB = B;

Y = (KR*fR + KG*fG + KB*fB);
U = (fB*KB) - Y;
V = (fR*KR) - Y;

}

/************************************************************************/
//  Convert a 10 bit YUV value to an 8 bit RGB value. Rec.601.
/************************************************************************/

yuv_to_rgb()
//yuv_to_rgb(Y, U, V)
//, int *R, int *G, int *B)
{
float fy, fu, fv;

fy = (float)Y;
fu = (float)U;
fv = (float)V;

nr = (int)(((fy+fv)/KR)+0.5);
ng = (int)((-(fy + fv + fu)/KG)+0.5);
nb = (int)(((fy+fu)/KB)+0.5);
//printf("%d\n",nb);
}

/************************************************************************/
//  Test all 16 million 8 bit RGB values through a round trip
//  from 8 bit RGB to 10 bit YUV and back to 8 bit RGB.
/************************************************************************/

test_round_trip()
{
//	int r, g, b
//, y, u, v, nr, ng, nb;
	int count, diff;

	printf("\nTesting round trip...\n");

	count = 0;
	diff = 0;

	for (R=0; R<= 1023; R++)
	{
		for (G=0; G<= 1023; G++)
		{
			for (B=0; B<= 1023; B++)
			{
rgb_to_yuv();
yuv_to_rgb();
				//rgb_to_yuv(r, g, b);
				//yuv_to_rgb(y, u, v, &nr, &ng, &nb);
				if ((R != nr) || (G != ng) || (B != nb))
				{
					//printf("%d, %d, %d -> %d, %d, %d -> %d, %d, %d\n", r, g, b, y, u, v, nr, ng, nb);
					diff++;
				}
				count++;
			}
		}
	}
printf("%d colors tested\n", count);
printf("%d colors differed\n", diff);
}

/************************************************************************/
//    return(0);

int main(void) {
test_round_trip();
puts ("Done");
return(0);
}

[shekh]

Your ffmpeg cmd uses Avisynth. Why would I need to frameserve the color bars if I can just load the avi directly, -i colorbarsrgb24.avi? Speaking of this, I tried it, and there is a strange color shift in the resulting out.avi, not to mention lots of chroma subsampling artifacts. Any ideas?

I kept avs input because you had it your example.

Here is v210 encoded from ColorBarsRGB.avi with virtualdub filtermod, using "709" matrix for output. Is it good?

[SameSelf]

jagabo, what do you think? On my machine there are no color errors (compiled with GCC). Do I have it right or am I missing something?

The spirit of your code and your constants look good. Your RGB to YUV conversion however doesn't follow the Y'CbCr convention. I know we use YUV as shorthand for Y'CbCr all the time, but they are not the same. Also, you are missing the scaling factors and offsets. By definition, Y'CbCr conforms to Video levels, and signed values are offset and full/PC range is considered out of gamut aka superwhites/blacks. The good news is 601, 709, and 2020 all follow the same conversion/scaling/offsets (I think), and the only difference is Kr and Kb. You might find this wiki page helpful.

https://en.wikipedia.org/wiki/YCbCr

EDIT: I would also eliminate Kg from your constants and replace all references in your code with 1 - Kr - Kb as you run the risk of over specifying your code.

[SameSelf]

ffmpeg uses 601 by default for RGB to YUV. You can tell ffmpeg to use 709 by either using -vf colormatrix, or -vf scale using out_color_matrix , set w,h to -1 to keep input dimensions

-vf colormatrix=bt601:bt709
or
-vf scale=w=-1:h=-1:out_color_matrix=bt709

There are other scaling option flags you can play with , such as -sws_dither, -sws flags for scaling (not sure how it affects chroma scaling for 422)

Are you ok with v210 subsampled 10bit422 YUV ? v410 is 10bit 444 YUV full chroma, but few programs support it .

or you can use AE or AME to convert 8bit RGB to v210, or maybe avisynth with dither tools/stack16 or vapoursynth

Or , if you're starting with RGB, why not use RGB lossless intermediate ?

pdr the great guru! Thanks so much. I learn things everyday about ffmpeg thanks to you.

I tried both of your rec'd -vf flags. The first or top one, still seems to have a slight color shift. Not as severe without the flag, but present none the less. The second or bottom one is much better with no hue shift from what I can see, maybe just a slight luma shift but only for the Mg and Cy bars.

Of course both have a lot of chroma subsampling artifacts. I wonder if there is a way in ffmpeg to tell it to use point resizing a la Avisynth? I already tried transcoding to V210 using PP, and there are a lot of chroma subsampling artifacts as well.

Unfortunately, an RGB lossless intermediate is not an option because Resolve does not read RGB files.

[poisondeathray]

-sws_flags neighbor is supposed to tell ffmpeg to use nearest neighbor, but it might affect final frame dimension resizing, not chroma plane resizing . -vf scale aslo accepts flags , but again I don't think it affects the chroma resizing, only the final dimensions scaling , you can test it out
https://ffmpeg.org/ffmpeg-filters.html#scale-1

prores4444 is accepted in resolve, but it's not mathematically lossless (probably very close on bars) , but it's not easy to encode on a PC as a MAC . NukeX can do it but I'm not sure if the free version can. There is a ffmpeg variant of the prores4444 encoder (prores-ks), and ffmbc does as well but there are various issues with both of them

Doesn't resolve accept PNG in MOV, or QT Animation in MOV ? Both are lossless RGB, but IIRC there was a bug with one or both of them but they might be fixed by now

[SameSelf]

or you can use AE or AME to convert 8bit RGB to v210, or maybe avisynth with dither tools/stack16 or vapoursynth

You know, it is funny that just last night I was reading the vapoursynth blog because VS supposedly has native 10-bit support. And there is a post titled "R29: Death to swscale!!!". Something about how zimg is much more accurate than swscale. However, I am left wondering if it offers point resizing. I am slowly wrapping my head around VS, and maybe soon I will be trying it out.

[poisondeathray]

You don't need to resize or incur any rounding errors with 8bit RGB coming from an 8bit RGB source. It works in resolve, PP/AE , every program

QTRLE does work (QT animation codec). IIRC , it was older versions that had problems , maybe with the alpha channel or something like that, either RGB, or RGBA worked for PNG or animation. Anyways, newest resolve works fine

Src and encode incl.

[poisondeathray]

or you can use AE or AME to convert 8bit RGB to v210, or maybe avisynth with dither tools/stack16 or vapoursynth

You know, it is funny that just last night I was reading the vapoursynth blog because VS supposedly has native 10-bit support. And there is a post titled "R29: Death to swscale!!!". Something about how zimg is much more accurate than swscale. However, I am left wondering if it offers point resizing. I am slowly wrapping my head around VS, and maybe soon I will be trying it out.

It does, or you can use fmtconv in vapoursynth which also offers lots of control, and a bit faster processing wise (script wise, vapoursynth is 10x more difficult than avisynth IMO for non programming types like me, it's very picky about syntax and conventions)

But this is all pointless when you can avoid all those conversions and use 8bit RGB

[chris319]

Your RGB to YUV conversion however doesn't follow the Y'CbCr convention. I know we use YUV as shorthand for Y'CbCr all the time, but they are not the same. Also, you are missing the scaling factors and offsets. By definition, Y'CbCr conforms to Video levels, and signed values are offset and full/PC range is considered out of gamut aka superwhites/blacks. The good news is 601, 709, and 2020 all follow the same conversion/scaling/offsets (I think), and the only difference is Kr and Kb.

I'm sorry, I'm not understanding what you're getting at. Would you care to modify the code to make it correct? There's not that much code. That might be better than a lot of back and forth trying to explain/understand it.

[SameSelf]

I am just going to modify your RGB > YUV portion. I am not fluent in whatever language that is and my changes are a little messy.

Y = (219/255)*(KR*fR + (1-KR-KB)*fG + KB*fB);
U = (224/255)/2/(1-KB)*(fB - Y*(255/219));
V = (224/255)/2/(1-KR)*(fR - Y*(255/219));

[SameSelf]

NukeX can do it but I'm not sure if the free version can. There is a ffmpeg variant of the prores4444 encoder (prores-ks), and ffmbc does as well but there are various issues with both of them

Doesn't resolve accept PNG in MOV, or QT Animation in MOV ? Both are lossless RGB, but IIRC there was a bug with one or both of them but they might be fixed by now

Sometimes I feel like if I were just to take the time to learn Nuke, all my problems would be resolved.

As for why Resolve doesn't read my RGB .avi, I don't have a clue. I have learned to not expect too much from Resolve beyond what it was built to do. The good news is it reads my UYVY, V210, and ProRes files fine. And thanks for the QTRLE tip. Resolve was able to load both of those as well. I don't have the absolute latest version, running 12.3, not 12.5. Maybe that is part of the problem.

FWIW, I just downloaded Nuke 10.0v2 and have a renewed motivation to look at it again.

[chris319]

I am just going to modify your RGB > YUV portion. I am not fluent in whatever language that is and my changes are a little messy.

Y = (219/255)*(KR*fR + (1-KR-KB)*fG + KB*fB);
U = (224/255)/2/(1-KB)*(fB - Y*(255/219));
V = (224/255)/2/(1-KR)*(fR - Y*(255/219));

I melded your code into mine, checking it carefully to make sure all was correct. No joy. Lots of errors in the round trip.

[SameSelf]

Well, I am not sure then. Those are the correctly scaled formulae. And there is a little math involved to transform the formulae to RGB as a functions of YUV. Did you also include the offsets?

[chris319]

I added the code you posted.

My code is written in the C programming language.

[SameSelf]

I am posting a workflow for transcoding to ProRes using ffmbc. Turns out that ffmpeg/ffmbc have a persistent bug related to swscale that hard codes the RGB->YUV conversion to the Rec.601 colorspace even for HD content. This results in a huge color shift. Additionally, ffmbc 0.7rc8 has some precision/performance issues while 0.7rc7 32-bit uses a much more precise conversion. The result is a very clean vectorscope especially since there is no chroma subsampling (see below).

Workflow:

1. Start with the ColorBarsRGB.avi attached in post #121
2. Using ffmbc 0.7rc7 32-bit

>ffmbc -i ColorBarsRGB.avi -threads 16 -color_primaries bt470bg -vcodec prores -profile hq -b 220000k -pix_fmt yuv444p10 -an ColorBars444.mov

Comments:

The -color_primaries flag somehow bypasses the Rec.601 encoding. For some reason, encoding to yuv422p10le, even with the -color_primaries flag, results in a color shift, even if I use the ProRes444 transcode as input. So, I haven't yet figured our a way to transcode to a ProRes 422 format without incurring a color shift.

I have attached below my ProRes 444 colorbars for anyone who might need them.

[poisondeathray]

So, I haven't yet figured our a way to transcode to a ProRes 422 format without incurring a color shift.

You won't be able to with ffmbc. There is even a slight shift on greyscale (!) YUV input when encoding prores422. It becomes tinted slightly. ffmpeg prores does not have the tint issue, but it has other problems. And they both have problems in various programs.

[SameSelf]

Thanks pdr. Saves me the wasted effort of more testing. I was going to give ffmpeg a try. I guess finding a Prores 422 encoder for the PC really is hard. But I am glad I found this ProRes444 workflow.

[poisondeathray]

Some of the ffmpeg prores compatibility issues in various programs are improved when you force prores hq profile
-profile:v 3

Some of the issues and differences can be related to the method of 8 to 10bit conversion (e.g. +/- dithering ; ffmbc dithers by default, ffmpeg does not), and the method of RGB to YUV conversion if starting from RGB. You might be able to avoid some of the issues by controlling the exact conversion you want in vapoursynth before feeding to ffmbc or ffmpeg . For ffmbc, I'm not sure if the color shift is from the actual encoder or from the bit depth conversion, or dithering or something else. Feeding it YUV takes the RGB to YUV part out of the equation, but if you fed it a known v210 input, that would take everything else out except the encoder. I didn't look into that yet, probably not worth it because they can still have other problems

[chris319]

I'm looking at your ColorBars444.mov file with VLC.

If those are supposed to be SMPTE bars, they're way out of spec, or VLC is mangling them.

[SameSelf]

I'm looking at your ColorBars444.mov file with VLC.

If those are supposed to be SMPTE bars, they're way out of spec, or VLC is mangling them.

Yes, they are SMPTE bars. The only minor difference is the pluge doesn't go to -4.

I would look into how VLC converts YUV to RGB. If you look at the Mediainfo, it says BT.601 for the colorspace, which is wrong. So it could be that VLC is reading the metadata and using BT.601 for the YUV to RGB conversion. You may have to force it to use BT.709.

Which reminds me, is there a way to edit the metadata on a ProRes file?

[SameSelf]

Some of the ffmpeg prores compatibility issues in various programs are improved when you force prores hq profile
-profile:v 3

Some of the issues and differences can be related to the method of 8 to 10bit conversion (e.g. +/- dithering ; ffmbc dithers by default, ffmpeg does not), and the method of RGB to YUV conversion if starting from RGB. You might be able to avoid some of the issues by controlling the exact conversion you want in vapoursynth before feeding to ffmbc or ffmpeg . For ffmbc, I'm not sure if the color shift is from the actual encoder or from the bit depth conversion, or dithering or something else. Feeding it YUV takes the RGB to YUV part out of the equation, but if you fed it a known v210 input, that would take everything else out except the encoder. I didn't look into that yet, probably not worth it because they can still have other problems

pdr, thanks. Here is something to think about maybe. If the problems can be avoided by controlling the conversion in vapoursynth and feeding that to ffmbc/ffmpeg, then wouldn't starting with the ProRes444 bars, which are already in 10-bit Rec.709 colorspace, achieve the same thing? That was the spirit of my testing. My guess is that problem is sadly in the encoder, and that this issue can't be avoided. It really is a shame that the developers of ffmpeg/ffmbc were so sloppy in their handling of color primaries. You would think with all the flags that exist, you could specify your own Kr and Kb values and avoid all this mess. These issues seriously compromise these tools.

[poisondeathray]

The idea was to control/specify what conversions were being done, instead of letting ffmpeg or ffmbc handle it. eg. If you wanted to disable dithering, or use "nearest neighbor" for chroma scaling, or use true 16bit for intermediate steps etc... you have control over every step

But now that I think about it, when I tested this a while back, there were problems with vspipe and 10bit422 piping even with the enable_v210=true switch, and the conversions done by either fmtconv or zimg were different than expected. Many updates since then, so I don't know if these things have been addressed, but I won't have time revisit that topic anytime soon. It's a pain to update vapoursynth because plugin versions change, python version changes (breaks compatibility some other programs).

[SameSelf]

I hear ya.

If I ever get vapoursynth up and running on my PC (using python was a huge mistake imho), I will give it a test. But unless vapoursynth offers true high bit depth support, it looks like it is more trouble than it is worth.

[poisondeathray]

v210 did work on earlier versions (like r10 or something), so something got broken it's probably fixed by now . The thing is every few versions, everything , the API changes and everything breaks. So you have to change everything , plugins, source filters, vpy helper scripts. Growing pains

It does support true higher bit depths and has more control than avisynth, but it's a big PITA to use IMO. Threading is much better than avisynth, it's more stable, no memory issues (64bit) . Programming types love it, non prgramming types like me find it 10x harder than avisynth. It takes like 5 lines to do the same thing that avisynth could do in 1. But it has huge potential

[chris319]

Yes, they are SMPTE bars.

Are they supposed to be within "studio swing" or 0 - 255 or 1 - 254?

Am I correct in assuming they're supposed to be 75% saturation?

Can you list the 8-bit RGB values they are supposed to be without encoding or decoding?

[SameSelf]

This is what AE says the dynamically linked PP generated SMPTE tone and bars RGB values are:

Grey: [191,191,191]
Yl: [191,190,0]
Cy: [0,191,190]
G: [0,191,0]
Mg: [191,0,192]
R: [191,0,1]
B: [0,1,192]
-I: [0,63,105]
+Q: [65,0,119]
W: [255,255,255]
Bl: [0,0,0]
Left pluge: [0,0,0]
Center pluge: [0,0,0]
Right pluge: [10,10,10]

Technically, the pluge is supposed to be -4,0,4 IRE, but -4 gets clipped in RGB space. Also, don't worry that the colors don't appear exactly right, for example, that R is not [191,0,0], because both collapse to the same 8-bit YUV value as do the others.

So, yes, they are 75% saturation (191/255 = 75%); otherwise the vectorscope will look wrong.

Studio swing applies to YUV, not RGB. 8-bit RGB is always [0...255].

FWIW, this page is the most complete explanation of the conversion matrices I have come across (although I would replace Kg in any code with 1-Kr-Kb to avoid overspecifying): http://avisynth.nl/index.php/Color_conversions