ConvertToYV12 in Avisynth: Rec601, Rec709, PC.601, PC.709, or Average?

[_Al_]

RGB32 or RGB24, it does not matter much, more about it was explained by jagabo above, I used RGB32, not sure what ccd.vdf needs, in that example it used 32, so I'd use RGB32 as you say. It might be even faster as was said, don't know. Use RGB 32.

LoadVirtualDubPlugin("C:\Users\CZ\Desktop\Video Capture\VirtualDub-1.10.4\plugins32\ccd.vdf","ccd") 
ccd(10,0) # threshold (0-100), multithreading (0=off. 1=on)
ConvertToYV12(interlaced=true, matrix="PC.601")
QTGMC()
#resize to square pixel here if not encoding with --sar 32:27

for that MPG standard resolution video you use: ConvertToYV12(interlaced=true, matrix="PC.601")

QTGMC changes colors? I'd say no, I use it pretty often, and also I tested your video and ConvertToYV12(interlaced=true, matrix="PC.601") does not change colors as well

never used avisynth denoise filters in Avisynth and such much, maybe some tests, if it is a disaster I just use neatvideo in Vegas, that fixes it

When I erased this code from my script, while I frameserved out as YUY2, it would lighten the picture and change it:

LoadVirtualDubPlugin("C:\Users\CZ\Desktop\Video Capture\VirtualDub-1.10.4\plugins32\ccd.vdf","ccd") 
ConvertToRGB32()
ccd(10,0) # threshold (0-100), multithreading (0=off. 1=on)

This scenario does this, Vegas changes your video to RGB, you frame serve YUY2, then you change it to RGB again in Avisynth, and you did not have it included in your script, you HAVE TO go back to YUV for encoding, so you change color spaces a lot.

Because of Vegas nature, working in RGB I export RGB every time, even if I do not need to have RGB in Avisynth, try YUV but I was there before, it did not help. But I worked with DV avi (using SD video), you work with MPG and who knows Vegas might treat those a bit differently, for example I got levels moved a bit off if working with DV, try it yourself. And again, if you do not interleave it with original you might be prone to mistakes comparing videos.

[poisondeathray]

He's already manipulated colors/levels in vegas, so Rec601 should be what he's using to match an internal vegas preview on a sRGB PC monitor (not studio RGB external 2nd monitor) . I think he wants to match what he's "seeing" in vegas post manipulations. If you "eyeball" the 1st post, the Rec601 is the closest there too. If you did no color manipulations, the PC 601 matrix is correct for SD native camera files - that would give you back the closest to original levels and color

He was initially asking what "looks best" which of course is subjective, and the "wrong" question to ask

[CZbwoi]

Thanks _AL_, I see.

If you did no color manipulations, the PC 601 matrix is correct for SD native camera files - that would give you back the closest to original levels and color

What if I only did color curves/color corrector in Vegas and no color manipulating in Avisynth, would PC.601 technically never be the one to use since you said it's only for if I did no color manipulations? Because right now every time I switch from Rec601 to PC.601, the colors get more rich and the darks become darker (honestly more reminiscent of what's in Vegas to me), while in Rec601 everything is more dull and foggy like there's a tint over it. I want great dynamic color range while keeping it *true and realistic* to form, PC.601 seems to expand on what Rec601 does and gives it life, unless I'm completely misunderstanding it.

Speaking of, is it even a good idea to use PC? Like, will the colors look fine on every device it's played on (barring CRT screens or whatever which will never be used)? I'm looking at pages comparing and describing color spaces and even on the Wikipedia page there is no mention of PC: https://en.wikipedia.org/wiki/List_of_color_spaces_and_their_uses

Is this a relatively new thing/used only by Avisynth?

[poisondeathray]

What if I only did color curves/color corrector in Vegas and no color manipulating in Avisynth, would PC.601 technically never be the one to use since you said it's only for if I did no color manipulations?

Yes. If you've corrected for the studio RGB levels, especially the black level, in your color corrections , then you would use Rec matrix

Because right now every time I switch from Rec601 to PC.601, the colors get more rich and the darks become darker (honestly more reminiscent of what's in Vegas to me), while in Rec601 everything is more dull and foggy like there's a tint over it. I want great dynamic color range while keeping it *true and realistic* to form, PC.601 seems to expand on what Rec601 does and gives it life, unless I'm completely misunderstanding it.

PC expands the range compared to Rec. It's basically 1:1 mapping. The matrices dicatate YUV<=>RGB conversions. Back and forth. That's what they do. When native camera formats such as your mpeg2 files are imported into vegas in 8bit mode, it uses a studio RGB conversion from YUV. Studio RGB means Y,Cb,Cr 0,0,0-255,255,255 are "mapped" to RGB 0,0,0-255,255,255 . This is very close to what avisynth calls "PC" matrix (there is a separate studioRGB function in avisynth as well). It will look non standard compared to 99.9% of everything, programs, devices, etc... because 99.9% use Rec matrices. If you just import a native camera file, and do nothing, it will undergo studio RGB conversion, the black level will appear elevated and "washed out" if you're using a normal computer sRGB monitor. If you applied a studioRGB to computerRGB preset only, it will suddenly look exactly like it does in a media player. A "computer RGB" or standard range conversion "maps" Y 16-235, CbCr 16-240 to RGB 0,0,0-255,255,255 or vice versa. This is identical to what avisynth calls a "Rec" matrix .

These are just standards. Rec601 is used for SD RGB<=>YUV conversions, Rec709 is used for HD RGB<=>YUV conversions. "PC" just means full range conversion both ways. When vegas uses "studio RGB" to convert the native YUV camera files, it uses something very close to "PC" levels, thus it looks washed out (black levels will be elevated) . It's just the way vegas does things, pros/cons. On one had you don't clip data. It preserves the head and tail room. A standard Rec conversion will clip superbrights and superdarks. On the other hand, 99% of other programs do it the other way using Rec (for example premiere, which you can "rescue" the lost data because you can use YUV filters and work in YUV, only the preview is converted to RGB, or until you use RGB filters) . It doesn't matter - you can end up with very close to the same thing no matter what program you use, just different workflow

Speaking of, is it even a good idea to use PC? Like, will the colors look fine on every device it's played on (barring CRT screens or whatever which will never be used)? I'm looking at pages comparing and describing color spaces and even on the Wikipedia page there is no mention of PC: https://en.wikipedia.org/wiki/List_of_color_spaces_and_their_uses

Is this a relatively new thing/used only by Avisynth?

Ideally you'd use a 2nd monitor, a grading or broadcast monitor, but only if you're doing this seriously or for a living. You can do fine just using a PC monitor and scopes.

"PC" levels just means full range conversion YUV<=>RGB . And vegas' studio RGB is very close to a "full range" conversion. The problem is different assets get treated differently in vegas. Also some export formats get treated differently too. Some get Rec or "computer RGB" treatment. Others get studio RGB . I already mentioned native camera files get studio RGB treatment. There are presets in vegas that allow you to inter convert. A studio RGB to computer RGB preset, and a computer RGB to studio RGB preset. It's just a quirk of the way vegas does things. Other programs have varous quirks too - you saw the interlaced chroma mishandling and workaround in premiere for example

[_Al_]

EDIT: PDR was faster, I might repeat a thing here or there.

All this rant below will be about Levels are not colors and how Vegas can freak out , it might even treat formats differently, but it has a build in functions to fix it easily. And I am going to confuse you a bit more I guess.

PC601 manipulate levels, not colors, so if you color correct a change levels somehow in Vegas so after loading it into Avisynth it is pretty much legal, sure, you do not use matrix="PC.601".
Again check frame server in avisynth script and histogram, with matrix="PC.601" or without. Check histogram, if it is very close to illegal values, why would you make it "worse" using PC601.

You just use ConvertToYV12() only. No need for matrix="PC.601" Your footage is already in 601 color space. That matrix="PC.601" just fixes levels to full range 0-255 assuming it is not correct.

Frame server exports what Vegas serves to it, levels are off, it needs to be fixed in Avisynth. There is even possibility in Vegas properties to set "pixel format" instead of "8bit" to "32 bit floating point" and either to "normal" or "full levels". If you set full format you'd not use matrix="PC.601" for sure in your case. Internal Vegas export subroutines remember all this and set correct levels but we have to fix it in Avisynth. At the beginning I even thought that it is Avisynth's fault to load video with wrong levels coming from Vegas until I played that frameserver.avi directly in a player and realizing colors were washed out. But again different video format and it might not be true. This is where you should pay an attention what Is happening with levels in Vegas all the time. I also mentioned Sony levels effect that can fix wrong Vegas interpretation for levels. There is even people that say the first thing you should do if loading clip in Vegas is to give it that Sony levels effect, Computer RGB to Studio RGB, where it basically "shrinks" levels, makes it look like washed out. Perhaps they use Vegas internal export modules and they are not happy with levels coming out or they know that Vegas calculates things internally without going into illegal values (like your color correcting), perhaps they treat overburn camcorder footage's, I don't know. It might be a mess for someone, but as soon you start to use histograms somewhere along you fix it As I mentioned there is more than one way to "fix" it.

Use histograms in Vegas, the same for audio, use that "audio meter" that is overthere, make an "executive decision" to always export audio that gives you -6dB for example so your audio levels are legal or that audio levels are at least uniform. The same for video, you should use histogram as well.
like pdr basically says

He was initially asking what "looks best" which of course is subjective, and the "wrong" question to ask

if you changed levels in Vegas somehow with effects, then whatever works for you to fix it

[poisondeathray]

Example to illustrate - look at your 1st post. The PC.601 conversion has lost a lot of detail in the arm of the jacket the levels are crushed, you can no longer see the "folds" clearly. Some people might like that "look". You would call it "contrasty" . But purposely doing that is not "color correction" , it's grading. It's a subjective look that is technically "wrong" , but people do it all they time on purpose for things like music videos etc... for a certain subjective "look" . The Rec.601 example matches what you see in vegas more closely - that's what you should be aiming for. The assumption is you're doing color manipulations in vegas. If you've purposely crushed it in vegas, thrown away details , using a PC matrix will crush it even more. Technically, throwing away stuff is bad.

When you ask "what do you like better", it's very subjective. You ask 10 different people and get 10 slightly different answers as to what they prefer and why.

[CZbwoi]

Yes. If you've corrected for the studio RGB levels, especially the black level, in your color corrections , then you would use Rec matrix

Pardon my elementary level but I just want to be certain, would what I did in here count as correcting black levels and whatnot, and thus result in needing to use Rec?





And here are two clips I just made to show it in action, because I don't feel like screenshots with the Print Screen button does it justice, again - just to be certain. To me it doesn't look like any data is lost in PC.601, it just makes everything more filling, I don't see any details lost here from my point of view compared to other times when I actually did crush the blacks. But maybe your expert eyes will confirm what you said, or maybe side with me here. I'm playing them side to side and it just looks like the PC one is better and doesn't lose anything, but I'm no pro here. Which one would you still go with?

[poisondeathray]

Yes, you didn't actually crush anything there, and the black level of the rec601 is elevated slightly

So I wouldn't go with either, or something in between. The black shadow details are less visible, but the rec601 version has slightly too high black levels . In other words I would adjust the black, but increase the shadow contrast

[CZbwoi]

Lol, so I'm back to square 1 Now even Rec601 has something too high? Oy vey.

So I wouldn't go with either, or something in between.

Well, I thought I had 2 options left to choose from to finally be done with this, and now I can't use either option

So how would I go about fixing this in either Vegas or Avisynth? Because I seem to be in sync where I would need to be between 0-100 of the luminance scale in Vegas, altering any of that to do what you're saying, from what I think I'd need to alter, would put me over or under.


P.s. Did you mean to say Rec601 both times in your last post or were you referring to PC one of the times?

[poisondeathray]

No, the pedestal, the lowest black level is correct at about Y=16 for the Pc601 version. Good.

However, too much data is clustered down low in the PC601 version. It's all clumped together. You're effectively reducing details. When I eyeballed the 1st post I even thought it was crushed (it technicially isn't) . The problem is a lot of those dark details are obscured. Ask yourself if you can see more suit/arm details in the Rec601 version or Pc601 version. This is an outdoor shot in bright sunlight. If I took a "black" suit into the sun, it would reflect light, show folds, details, not be obscured in bright sunlight. Here it looks like there is 1 or 2 "shades" of black, those details are obscured. One generic goal of color correction is spread out or increase tonal range. You're kind of doing that when you say you want more "dynamic range" . So the lowest , blackest black level should indeed be ~Y=16. However, you want to increase the shadow contrast. For example in the curves you can boost the shadow contrast, but keep the blackest black levels still at 0 IRE or Y=16


This is not a PC or Rec thing. Those don't color correct or grade. Those only stipulate how YUV <=> RGB conversions are done

[_Al_]

Do you have original uploaded somewhere?

What just pdr says, should perhaps be fixed or kept (if it was recorded well) in Vegas.

I think you should use PC.601 but do it "differently" in Vegas , because somehow it might not be done correctly in Vegas in the first place, maybe your footage is crushed as recorded at those ends, so you might color correct it after you apply Computer RGB to Studio RGB and then after color correcting, you switch it back on the output, never tried it like that though.

[CZbwoi]

So the lowest , blackest black level should indeed be ~Y=16. However, you want to increase the shadow contrast. For example in the curves you can boost the shadow contrast, but keep the blackest black levels still at 0 IRE or Y=16

Hm, alright...so how I would go about increasing the shadow contrast in the curves without changing where I correctly am at 0 IRE..? Should I move the bottom of the S curve's handle and move it a smudge up..or to the right..? I don't know if moving anything in the curves and in what direction will screw up that perfect Y=16 level I'm at. And I know I sound like a complete n00b saying all this. But I assumed that if I get into the correct areas, not below 0 and not above 100, that I'd be fine, and it seems like moving any of those would alter that...

Do you have original uploaded somewhere?

Here's the original with only AssumeTFF and a spline36resize added:

Edit: and I still don't know what you mean when you say "Computer RGB to Studio RGB" filter or the "Sony levels" plugin, I'm looking at all the plugins I have in Vegas under All and there is nothing with either of those names.

[poisondeathray]

For curves, have you used photoshop or other image editors like gimp? It's analogous to that. You can put "knots" in the curve to hold values in a certain place. In vegas it's right click insert point . You can pull out the bezier handles to shape the curve. If you're still having difficulty getting what you want - you can get it close and just make minor adjustments with levels filter.


In newer versions of vegas, Studio to Computer RGB and vice versa are a levels preset. So apply the levels filter and in the drop down menu of presets they should be listed in one of the presets.

[CZbwoi]

Alright, so I added the Computer RGB to Studio RGB preset in the Levels plugin and placed that at the beginning of my workflow in Vegas, it changed everything so I adjusted the color curves/corrector accordingly.

Here are the two results now...any changes, improved or regressed, and for which..?

[jagabo]

pc601_0.mp4 has about the right levels but there is some minor crushing of darks somewhere along the processing path. Colors are oversaturated and there are some areas with illegal colors.

[_Al_]

That original.mp4 is not original camcorder video,levels are off.

I tested levels of that M2U00012.MPG you posted in the other thread, levels are pretty much ok, it looks ok on my living room plasma, I would not try to fix it.

Also even making mp4 thru frame server you should not change levels (as oppose in that so called original.mp4 you posted), I tested it before (previous page), it could be done without any level change.

And Vegas can render that MPG original losslessly! It changes only parts where needed, it changed only a part with subtitles in it. For me it would be actually no-brainer regarding for all of this.

[_Al_]

here is that rendered clip

[_Al_]

also I posted one screen that tells nothing, this is actual setting for that "NTSC DVD Architect widescreen stream":

[CZbwoi]

pc601_0.mp4 has about the right levels but there is some minor crushing of darks somewhere along the processing path. Colors are oversaturated and there are some areas with illegal colors.

That's what I thought too, more saturated than before. Did you take a look at pc601.mp4 from December 1? That one was from before I added the Computer RGB to Studio RGB effect. Would you say that one is better?

That original.mp4 is not original camcorder video,levels are off.

I needed to take those exact frames from the file to compare, as it's not an indivudal file, so I had to frameserve the project out of Vegas with all the filters off into Avisynth so I could trim the same amount in frames (in half because I turned QTGMC off as well). But like I said, I frameserved it out of Vegas with every filter unchecked so I could get the same frames for the comparison, and in Avisynth the only other thing that was there was AssumeTFF and a spline36resize added like I said. I don't know how the levels would be off, probably because it stayed in Vegas' RGB mode? I didn't convert it to YV12 or YUY2 while encoding.

I tested levels of that M2U00012.MPG you posted in the other thread, levels are pretty much ok, it looks ok on my living room plasma, I would not try to fix it.

So...I should remove that Computer RGB to Studio RGB Levels filter you told me to add at the start of my workflow, correct? And only keep my Color Curves/Corrector plugins?

And Vegas can render that MPG original losslessly! It changes only parts where needed, it changed only a part with subtitles in it. For me it would be actually no-brainer regarding for all of this.

here is that rendered clip

Okay...so frameserving out is what is throwing my levels off now? Are you telling me to color correct in Vegas with Curves/Corrector, export the project out as "NTSC DVD Architect widescreen stream", open that file up in Avisynth to add my ccd.vdf filter, QTGMC, and Spline36Resize...and then encode that? So doing this would be better than frameserving it out..? In the end I am doing the same thing, color correcting in Vegas and applying those few things in Avisynth. Frameserving is at fault here?

[_Al_]

mpeg2cut2 can cut a part of you MPG clip

I do not understand if you take all effects off, frame serve it as I described on the other page and have levels off. I can see same levels, colors but yours clips are changed. I tested frame serving and it was fine.

Regarding that lossless proces in Vegas, I meant that I would not bother at all and export MPG stream which is on the top of things even DVD compatible.
But there is a chance for you to just edit it export lossless MPG movie and as you said load it into Avisynth thru DGindex and fix it there, if for some mysterious reason you cannot make frame server working right.

[jagabo]

Did you take a look at pc601.mp4 from December 1?

I just looked at it. It's oversaturated too, some brights are a tiny bit too bright (I wouldn't reencode it just to fix that), the black level looks good, and it doesn't have the crushed blacks of pc601_0.mp4.

[CZbwoi]

But there is a chance for you to just edit it export lossless MPG movie and as you said load it into Avisynth thru DGindex and fix it there, if for some mysterious reason you cannot make frame server working right.

True.

I just looked at it. It's oversaturated too, some brights are a tiny bit too bright (I wouldn't reencode it just to fix that), the black level looks good, and it doesn't have the crushed blacks of pc601_0.mp4.

Yes, that's what I think I'm going to stick with, no more adding that Computer RGB to Studio RGB Levels filter, it only complicates things for me when I already got it better beforehand.

It's oversaturated too

Are you referring to the red cake in this case? That cake reflected by the sun flies off the radar in every instance I've done - even originally without any color correcting done at all and when the film is all bland that's the only thing that manages to be on the tip of the radar. So barring singular focus on the cake and desaturating just that (which I have no clue on doing), the rest of it is okay, right? Or do you think the entire clip is oversaturated?

[_Al_]

no more adding that Computer RGB to Studio RGB Levels filter, it only complicates things for me when I already got it better beforehand.

I suggested above to use it Computer RGB to Studio RGB (on the whole track on the left) , apply effects, whatever you do changing gamma, colors, do not crush ends though much. And then return it back, with Studio RGB to Computer RGB. For this you use Video Master Bus (Shift+Ctrl+B gets it on the timeline). This basically effects output. So your levels are back. Is that what you did? Not to use Computer RGB to StudioRGB and then try to fix it with color corrections or in Avisynth.
But anyway, there is something going on because you cannot seem to export frame server without effects that matches your video, so you seem to not have a reference point for what you do and just guessing correct output.

[jagabo]

It's oversaturated too

Are you referring to the red cake in this case?

The cake is definitely out of range. But so is the orange bench around frame 260, the lady's blue dress around frame 1240. Even if you darken the clip to get brights below Y=235.

frame 1240 after ColorYUV(gain_y=-5) with illegal (outside the rec.601 RGB cube) YUV combinations highlighted in yellow:


[Attachment 39791 - Click to enlarge]

And just looking at the clip, colorful things look a little too saturated to me.

[CZbwoi]

K, made this, inserted the 2 levels conversions _AL_ was talking about to the start and end, also changed some stuff around the 2 filter plugins to make the luminance for instance more legal. What do you think, improvement? Did I lose too much, did I even do anything significant? Good or bad?

[_Al_]

you can try to compare all of them yourself using histograms and your eye as well (regarding color, saturation) here they are all of them, download them put them in separate folder and click between them, remember you cannot compare two images next to each other, you have to compare them interleaving them , in this case in Windows Photo Viewer clicking next,

1.mp4.JPG looks the best somehow
I included even JPG from your original MPG to show those blacks could be the same, it can be recorded like that
You basically compare only usable ones like 1.mp4.JPG and PC.601_0.mp4.JPG and 1.mp4.JPG has levels not that crushed.
pictures are lined up the other way, from the worst to the best
rec.601.mp4.jpg
rec.601_0.mp4.jpg
PC.601.mp4.jpg
PC.601_0.mp4.jpg
1.mp4.jpg
last one is some other original to compare black levels

[CZbwoi]

Would you say 1.mp4 is best color/saturation wise as well? I think it's the best one yet...and should maybe settle with that.

[jagabo]

Lovels are ok but I think it's still too saturated. I'd reduce saturation by about 10 percent.

[CZbwoi]

Okay, so the saturation level in Vegas' color corrector plugin was set at 1.000, and I lowered it to .900. What do you think about it now, all good? Attached file - 2.mp4


Another question I have for these camcorder videos, this one included, it's hard to choose what to do. I want the best color and the full color range to be there, from 0 to 255 (that is the goal, right?), but it seems to be impossible to do it at the sake of keeping the luminance level in check. The colors at their highest peak don't reach the 255 mark on the RGB parade, almost a bar and a half below the 255 mark, likewise on the bottom when I go to a darker part of the video.



Is it not expected for me to to enable the full color range here up to 255 for the best look possible, in the color range sense? After tinkering with these other things, it's pretty much impossible for me to go up to 255 anymore - if I change anything, the luminance goes back up to being illegal, the saturation goes up sometimes too, I think, etc.

What's more important here, having the full (correct?) color range or keeping the luminance waveform in check? At the moment my colors can't reach their "peak" after all these tweaks and it caps off nearly a bar and a half of a notch before 255. And every time I try to raise the RGB parade's peak to be right at 255, as I was taught, the luminance waveform also jumps up a bar...making that one illegal...while the RGB one is now fully in range.

Which leaves me in a weird situation, because in a way that's only happening because that's the sunlight beaming through those windows making the luminance go up that high ONLY when it points to the windows. The rest of the video has the luminance's peak at around 55-70 with an occasional 80-85. It's all the same in the end if the luminance is a bar higher over the illegal mark in that instance at the sake of having the entire color range, your eyes still hurt looking at it and there's nothing you can do to change the fact that it's a bright window due to the sun, the effect barely changes, right? Or should that not even be a question because correct luminance > color range?

I hope that made sense.

[jagabo]

2.mp4's colors are better. The only illegal colors now are from oversharpening and YV12 blur. That's perfectly normal.

That sample image from Vegas appears to be showing the Y levels correctly, IRE 0-100 (Y 16-235), but the displayed image decoded with pc.601 colors, not rec.601 colors. And hence, the RGB parade is from 16-235, not 0-255. Check the display settings in Vegas. With proper rec.601 decoding RGB values will range from 0 to 255:


[Attachment 39870 - Click to enlarge]

Open that image in a video editor (or use a screen RGB reader) and you'll see the whites are up around 253. Blacks aren't down to zero but that might just be that particular shot (IRE isn't down to 0 either).

Yes, when editing you sometimes let brights blow out in order to bring out detail in the rest of the picture. Or conversely, let black crush to bring out detail in brighter parts of the picture.