Frameserve Color Shift

[SameSelf]

It took me a while to figure this out. In order to properly load into Avisynth colorbars generated in After Effects so as to convert them to a YUV format, I had to specify the following:

Avisource(src, pixel_type="RGB24")
ConvertToYV16(matrix="Rec709",chromaresample="point")

It would seem that ConvertToYV16 defaults to Rec601. My colorbars are HD, so this caused a color shift. Fortunately, the use of colorbars made the color shift obvious. And fortunately I persevered because this produces cleaner colorbars for vectorscope purposes. Previously I was using colorbars that I generated by exporting RGB24 colorbars from PP as UYVY. But these were quite messy due to the chroma resampling that PP uses.

So here is my question. I can't for the life of me figure out why there is a color shift in the bars when I frameserve out of PP as YUY2. I thought after figuring out the above, maybe I could make YUY2 work, but I just can't. I am not talking about a lossless versus lossy problem. I am talking about a color shift that is just plain wrong.

Below are screenshots showing the shift. Does anyone know why this color shift is occuring? Is it a bug in the frameserver? Or is Avisynth loading the signpost incorrectly? Thanks for all the help.

Original

Color Shifted

[jagabo]

The software you are using to convert YUY2 to RGB for the PNG file is converting with the wrong matrix? For example, VirtualDub always converts YUV to RGB with a rec.601 matrix.

[poisondeathray]

What are you doing exactly ? Colorbars in AE, open in PP, frameserve out YUY2 ?

If so, why is AVISource pixel_type="RGB24" ? , not YUY2 ?

And how are you converting BACK to RGB for the screenshot ? (or how are you viewing it)

[SameSelf]

Thanks guys for the responses. To be certain, I did fully transcode the bars and bring them back into PP just in case it was a display issue. It is not, the shift is real. Here is my workflow.

I have a set of bars that I generated as described in my OP. The only thing I didn't mention is I used ffmpeg to transcode them as UYVY. I import these into PP. Generate a timeline. Then frameserve the timeline as YUY2. Then I try to load the signpost in Avisynth but there is always a color shift. I have played around with Avisource settings and Convert, but it doesn't matter.

Here are the bars if you want to test.

EDIT: I should mention that these bars match the RGB bars generated out of AE when I overlay them in a timeline in PP and compare the scopes. The only differences are the edges between the bars. But as I said, these bars are much cleaner than what I was using previously.

[jagabo]

Well, your color shifted image is rec.601 colors converted to RGB with a rec.709 matrix.

ImageSource(...)
ConvertToYV24(matrix="rec709")
ConvertToRGB(matrix="rec601")

Converts the colors back to the expected RGB values. Aside from rounding errors, obviously.

[poisondeathray]

OK, I see

1) RGB bars
2) Avisynth to convert to YUV422, using "point" for the purpose of cleaner bars/lines, less interpolation
3) ffmpeg to convert the avs to UYVY
4) importing that UYVY into PP looks ok colorwise
5) but frameserving out YUY2 looks wrong

Like jagabo says, the screenshot colorwise just demonstrates a 601/709 mismatch. But how are you loading it BACK into avisynth ? Again, how are you "viewing" that frameserved YUY2 output ? How are you taking the screenshot? ie. how are you converting it BACK to RGB for display purposes ?

[poisondeathray]

As an aside, your "blue" seems off too , both in the "original" screenshot, and UYVY. 75% blue should be 0,0,191 in 8bit RGB

I'm seeing 1,1,193 in your "BarsUYVY.avi" , both in Adobe and avisynth. 2,1,194 in your "original" RGB screenshot .

When I convert RGB bars to UYVY, I see 0,0,191 in both programs in RGB display. It suggests you're doing something slightly differently that what you described

Here are RGB bars as a PNG , and the UYVY conversion with avisynth, ffmpeg
(when attaching uncompressed video or images, you should zip it up to reduce the filesize as forum courtesy)

[SameSelf]

I downloaded your Bars+UYVY and they are identical to my RGB and UYVY bars when I import them into PP and scope them up. I don't know why you are seeing slightly different RGB values, but that is a secondary issue for me right now versus the color shift I am trying to resolve. I posted BarsUYVY.avi above. Now, I am posting my UYVY bars that were frameserved out of PP as YUY2 and then transcoded to UYVY in ffmpeg.

Just a recap of the workflow:

1. Import BarsUYVY.avi into PP (make sure PP interprets as progressive)
2. Load into a timeline
3. Frameserve out as YUY2
4. Load the signpost in Avisynth with Avisource(src,audio=false,pixel_type="YUY2")
5. Transcode to UYVY ffmpeg -i signpost.avs -c:v rawvideo -pix_fmt uyvy422 -vtag "UYVY" -an BarsFS.avi

Here are the frameserved bars. Do you see a difference between these and the BarsUYVY.avi? They are different when I scope them up in PP. They are also different when I view the Avisynth script in AvsPmod.

Thanks

[poisondeathray]

Yes, BarsFS.avi are off in the same manner (rec601 vs. rec709) . This suggest a problem with the handing off to the frameserve, or the frameserve itself (My guess is your YUY2 ws treated as RGB and probably got Rec601 treatment). You can "fix" it afterwards by using colormatrix, but there are several lossy steps in there

[poisondeathray]

BarsUYVY.avi and uyvy_ffmpeg.avi are not identical, I'm 100% sure. Both avisynth and Adobe report the same difference

When you get unexpected differences, even so slight, in my experience they can provide clues as to what might really be going on, and might even be related to your original problem - don't be so quick to brush aside

[jagabo]

BarsUYVY.AVI has rec.709 colors.
BarsSF.AVI has rec.601 colors.
SameSelf's files.

[SameSelf]

Thanks, that answers the question on why the difference exists. I suppose it would have been easier to just post the two videos up front and ask why they are different. I still need to find a resolution, but now I have a place to start.

I am beginning to think that the problem is somewhere in PP. I have a hard time believing that the frameserver does any matrix conversions on the fly. I could be wrong though. But this has me very troubled because it means that frameserving is broken in PP if I can't get it to not frameserve as Rec601. Does this mean all my past frameserves (tons of video now on youtube and other places) were frameserved as Rec601. Gasp!

I need to step through this a little more carefully to figure out exactly where the problem lies.

[poisondeathray]

I have a hard time believing that the frameserver does any matrix conversions on the fly. I could be wrong though. But this has me very troubled because it means that frameserving is broken in PP if I can't get it to not frameserve as Rec601. Does this mean all my past frameserves (tons of video now on youtube and other places) were frameserved as Rec601. Gasp!

Why should it be a stretch of the imagination? On import , UYVY gets "special treatment" . YUY2 gets converted to RGB. Probably the same thing is happening, and probably the matrix used is Rec601 for at least one of the conversions (if you specify YUY2, it's probably getting converted twice, once to RGB, back to YUY2 on frameserve)

Didn't you resolve these issues with advanced frameserver instead of dmfs ? Or was this using adv frameserver ?

[SameSelf]

DMFS was giving me the same problem and doesn't offer UYVY, so I deleted it from my build and started using AFS with UYVY. I don't know what happened, but somewhere along the way, frameserving as UYVY stopped working. Now, I can't load the signpost in Avisynth. So I stepped back to YUY2 hoping to see better results only to encounter the same color shift problem in DMFS. But AFS is a fork of DMFS, so I guess I shouldn't be too surprised that it has the same problem.

If only only I could get the UYVY signpost to load in Avisynth, I could avoid all this. I had it working at one point. I have no idea why it stopped.

[poisondeathray]

In my experience, (and many people's experiences) - CS5 was the last stable frameserver version. Every version of frameserver (advanced or dmfs) after that is buggy for PP ; either audio glitches, or corrupted frames, or crashing, or limited use without major workarounds. You are in the minority if you have it working, or you haven't looked or listened closely to your old projects. Nobody has been able to figure out why it works for a handful of people, but not others. (In contrast, DMFS for vegas is very stable. Many satisfied users there.)

I would offer to try help debug , but besides the other problems it's a massive PITA in CC because you have to uninstall / reinstall each time you use it. It's like a 1 use limited trial. Satish has released the source code for DMFS on github about a year ago , so if someone picked up the project - in theory it could be made great

So if this is occurring, an RGB conversion is likely occurring somewhere in the path. So it's a double hit , wrong matrix, colorspace conversion to RGB (with clipping and out of gamut values if coming from a camera source). As mentioned earlier, it's possible to "Fix" in YUV with colormatrix (mode="rec.601->rec.709", clamp=0) - that changes the color in YUV staying in YUV (instead of converting back to RGB) as if it had used 709 instead of 601 when coming from RGB. But that's 3 lossy conversions now with rounding errors each step. Not a big deal for many types of projects, but you seem to be on a quest for true "losslessness."

Most people have moved away from frameserving in PP because 1) it's so buggy 2) lossless intermediate means faster workflow when you have 2pass or multiple destinations 3) you have an "archival" master 4) hdd's have come down in price/GB massively over the last few years

If anything has a better chance of working, it would be UYVY so I would concentrate your effort there or try to remember what you did to get it working in the first place