Avisynth ColorYUV Gamma is doubling the encoding bitrate, please help

[VideoFanatic]

Here's a short sample MPEG2.

I created a dedicated thread for this issue so I can get more eyes looking at it. This is an urgent problem for me, I've just found out that my last week's h264 encodings took up double (10 Mbps) the bitrate than normal (5 Mbps) due to the gamma line as below so I need to fix them quickly:

Mpeg2Source("Gamma Problem.d2v")
ColorYUV(off_u=10, off_v=-2) 
org=last Overlay(org,last,0,0,GreyScale(org).ColorYUV(cont_y=100)) 
ColorYUV(gamma_y=200, off_y=-32) # Removes darkness blur

I added ColorYUV(levels="PC->TV") and my video now encodes at the normal bitrate but it increases the brightness of the video so I need to know how to remove that extra brightness.

Jagabo suggested that I should "make sure your levels are within the valid range". I don't know how to do that. I thought the ColorYUV(levels="PC->TV") line did that? Maybe ColorMatrix does that also but I don't want to convert to RGB.

Themaster1 suggested "coloryuv pc>tv levels is a wrong choice because it'll lower UV (255->240) so try ylevels instead and lower the gamma or tweak's brightness with coring=false)". What is the consequence of lowering UV? The video is for viewing on a PC so I thought I was supposed to have the ColorYUV(levels="PC->TV") line?

I'm not sure what code I'm supposed to use for ylevels. Tweak already has coring=false but it made no difference.

Should I just use the code I already have at the top of the page and remove the brightness with -15 bright on Tweak?

[jagabo]

Histogram()

[VideoFanatic]

I loaded 2 videos side by side in AvsPmod with histogram. One with ColorYUV(levels="PC->TV") and the other without it. The histogram looks the same on both only the white on the histogram is slightly more bright when ColorYUV(levels="PC->TV") is used. I don't know what I'm supposed to do with that information and I read the Avisynth.org histogram page.

[sanlyn]

No, the histograms don't look "the same". You place the histogram statement in a script after you make level and color changes. If you place the histogram statement before the lines that make brightness changes, you change the brightness of the histogram. Besides, you are using a filter and values that were designed for a different scene with different problems. The first camera shot in your sample video already has chroma values that are outside the 16-240 range, and the longer close-up shot is already too bright.

What does "removes darkness blur" mean? Gamma_y and off_y won't fix blur. Using your script, the darks are crushed.

In the close-up shot, If by "darkness blur" you are referring to the short depth of focus that always occurs with the use of long telephoto lenses, you're fighting a lost cause. All long telephoto lenses used at wide apertures for indoor lighting will have that effect on background objects when focused on something closer to the lens.

ColorYUV(Levels="PC>TV") sets the color matrix. It does not automatically "correct" invalid levels.

Why don't you set your own bitrate? I have no idea what a "normal" bitrate means. You use a bitrate that is adequate and suitable for the video at hand. If you don't like the bitrate your encoder sets, change it.

[VideoFanatic]

OK I've updated the first post with the sample that the script was originally designed for so the gamma setting is correct for that video. I put the histogram code last in the script so yes the histograms now look different but I don't know what I'm supposed to do with the info I see on the histogram.

I use a CRF encoding setting which is adaptive so it will increase or lower the bitrate as required up to a point. Without the gamma line my videos encode at about 5 Mbps max (normal bitrate) and with the gamma line they encode at double that. Adding ColorYUV(Levels="PC>TV") fixes the bitrate problem but it adds brightness. I can remove the brightness by adding -15 bright to Tweak but is this is the best way to fix the problem? You guys mentioned fixing the levels but I don't know what I'm supposed to do, could you please provide a script to fix my problem?

I don't have time for guesswork and to learn about histograms etc. I will try to learn that stuff in a few months when I have the time. I need to fix this problem today so if you guys know the answer then please tell me.

[jagabo]

I don't know what I'm supposed to do with the info I see on the histogram.

You want the blackest blacks in your video to be black, and the brightest brightest whites to be white. But you don't want the histogram to routinely protrude into the yellow areas (Y<16, Y>235) of the histogram. Ignore small excursions by oversharpening halos and noise (you can use coring to eliminate those but it's not really necessary). And use some sense, not every shot is supposed to have full black and/or full white content. Unless you plan to go through your video shot-by-shot just look for a compromise settings that works reasonably well for most shots.

I usually prefer a horizontal graph trace rather than a vertical trace. To do so I use

TurnRight().Histogram().TurnLeft()

or

VideoScope()

The latter gives both horizontal and vertical traces by default.

If you don't understand what a waveform histogram is these posts might help:

https://forum.videohelp.com/threads/293041-Any-harm-in-using-ConvertToRGB24%28%29-by-de...=1#post1792823
https://forum.videohelp.com/threads/296961-TV-Safe-Colors?p=1812482&viewfull=1#post1812482

[VideoFanatic]

Telling me how the histogram works doesn't help with my problem of the brightness issue. Am I supposed to add ColorYUV(Levels="PC>TV")? I did so so my video would encode at the normal bitrate instead of double. However the video is brightened when I add that. What should I do to lower the brightness? Tweak -15 bright seems to do it - is that OK?

[jagabo]

Telling me how the histogram works doesn't help with my problem of the brightness issue.

Really?

[VideoFanatic]

I appreciate your help but I specifically said I don't have time to waste in figuring this stuff out. If you have the solution then please provide me with the script.

[sanlyn]

Well....in this case I'll agree with the histogram problem. YUV histograms are focused on luma, as if color didn't matter, and correcting narrow color ranges in YUV is an exercise in futility (IMHO). You can onbly go so far with that colorspace. Still, it's necessary to fix basic stuff in YUV before proceeding -- there's no other way.

frame 202 in your original mpg. Not too bad, but the major problems are the midlevels and chroma. The black levels are too high, and gamma is too high -- that's why your borrowed statement with gamma_y=200 will not work here. And the image has too much magenta. IF you go into RGB to read more thorough histograms and scopes (which IMO are easier to understand) , and use a pixel sampler to read pixel values("Mod" has one), you could have figured this out a long time ago.

[Attachment 16256 - Click to enlarge]

After running your script, you've blown away the midtones and nearly smashed the darks. Please forget the "remove background blur" business. It's a long-telephoto shot; that will blur the background. Always. Period. Every time. No exceptions. And using Tweak to lower contrast is the wrong move. The color balance has moved from purple to pale pink.

Tweak(Hue=0, Sat=1.5, Bright=0, Cont=0.9, Coring=False)   
ColorYUV(gamma_y=200, off_y=-32) # Removes darkness blur

[Attachment 16257 - Click to enlarge]

Sane gamma effort. I also fixed the right-edge ghosting (in the images above, look at the girl's arm, shoulder, and the side of her face facing the guy. And killed some of that grimy junk on skin edges). It can be tweaked more, even in YUV, but you'd need RGB to finish it off.

ColorYUV(cont_y=10,gain_y=-5,cont_v=-10,cont_u=-30)
Tweak(sat=1.5,hue=-12)

[Attachment 16258 - Click to enlarge]

Coming out of YUV it's still a bit pink, and there's a lot of tape noise. I cleaned all that pretty well in RGB (because it's a chroma gamma problem across a narrow RGB range. Not possible in YUV. Sorry). I didn't post the RGB version.

[VideoFanatic]

Thanks Sanlyn. That helps fix this particular video but that wasn't what I needed help with! I have lots of videos that definately do need the below script such as the video in the first post (this is the brown video from another thread).

ColorYUV(gamma_y=200, off_y=-32)

The problem is that the red text causes my encodings to use up to double the bitrate than what it normally would. I use Simple x264 Launcher (it's portable and doesn't need to be installed). I've got a pre-setup version here if anybody wants to test what I'm saying. I use CRF 17, preset very fast, tuning film, custom parameters: --tff --sar 8:9

I figured out that adding ColorYUV(levels="PC->TV") fixes that problem but that also increases the brightness. What I would like to know is if I should be adding that code or is there some alternative I could use that doesn't increase the brightness? If not, then how should I remove the added brightness? Would -15 brightness with Tweak do it?

[sanlyn]

ColorYUV(gamma_y=200, off_y=-32)

The problem is that the red text causes my encodings to use up to double the bitrate than what it normally would.

You aren't using the script that jagabo originally posted. The original script is:

ColorYUV(off_u=10, off_v=-2)
Overlay(org,last,0,0,GreyScale(org).ColorYUV(cont_y=100))
ColorYUV(gamma_y=100, off_y=-32)

Try this (it should keep things inside the 16-240 range a little better):

ColorYUV(off_u=10, off_v=-2)
Overlay(org,last,0,0,GreyScale(org).ColorYUV(cont_y=100))
ColorYUV(cont_y=-20,gamma_y=100, off_y=-32)

I figured out that adding ColorYUV(levels="PC->TV") fixes that problem but that also increases the brightness.

It doesn't increase brightness. It outputs the YUV original's range as clamped 16=240. Using the original script and the mod posted above should to solve the problem.

Many encoders. when they see a source video that has fits of noise and defects, will raise the bitrate. You should not use low bitrates on pre-encoded, crappy source. Why go thru all this work to clean up, then make it worse?

[VideoFanatic]

I did what you said and used the code below but the video still looks a little too bright and blurred when I add the first line:

ColorYUV(levels="PC->TV")
ColorYUV(off_u=10, off_v=-2)
org=last Overlay(org,last,0,0,GreyScale(org).ColorYUV(cont_y=100)) 
ColorYUV(cont_y=-20,gamma_y=100, off_y=-32)

Also you mentioned that I used low bitrates but 5 Mbps in h264 is the equivalent of 10 Mbps in MPEG2 (DVD has a 9.8 Mbps max) so it's not really low. I'm using McTemporalDenoise & QTGMC as well to remove noise but to keep things simple I just asked for help with the brightness issue.

[sanlyn]

I wouldn't use the PC-> line. Actually it isn't too bright, it just has 0-255 squished into 16-235, so the black levels look a little washed out. They wouldn't look that way on TV, because RGB16 is "black" on TV. Anyway, lowering brightness with Tweak won't work the way you think it will with that code -- you'll end up with crushed blacks again, which is another bit-hungry encoding problem.

You should be able to set your own bitrate. That video still has lots of compression and tape noise; you would have to filter the heck out of it to clean it all, and there wouldn't be much video left. I'd have to say there's a reason why your encoder wants more bits. But think of it this way: noisy or over-sharpened or over-contrasty source = higher bitrate demands. There's really no good way around it, short of getting a bit-starved video. And somebody flying across the ring and taking 6 bounces back and forth = more bitrate. I'm surprised your encodes weren't higher earlier.

5 mbps for h624 isn't all that low for fast-action video. More activity means more bits, or you get with motion artifacts and block noise -- which is a big defect with those low-bitrate MPEG sources to begin with. The modified script (without the PC>TV line) ought to lower contrast and pull in the luma and chroma values closer to 16-235. Other than that, you have a problem video -- which is what I think your encoder is telling you.

All I use is TX264 or TMPGEnc Mastering Works, so maybe someone who uses your software could suggest something.

You can only imagine how much cleaner those old mpeg transfers would have looked with higher bitrates. They would have been easier to denoise and re-encode.

[jagabo]

Unless jagabo objects, I'd say make it 100 instead of 200.

I don't object. The OP should do whatever he thinks looks best. But I'm not going to spend my time fine tuning his videos for him when he isn't willing to spend his own time learning anything.

[jagabo]

RGB16 is "black" on TV.

No, RGB=0 is black on TV. RGB=16 should be very dark grey. That's why you use a rec.601/709 matrix to convert between RGB and YUV.

[sanlyn]

Right. Give me the model numbers of your TVs that can show different shades of black between RGB 0 and RGB 16. The last one I saw that could do that was the Toshiba 27AF CRT series. Even a SONY Wega or Tinitron TV couldn't do that.

[jagabo]

Every TV and computer monitor I have ever had will show the difference between RGB 0 and RGB 16.

Make four color patches at RGB 0, RGB 6, RGB 10, RGB 16 using an sRGB color space.

There's your problem. sRGB is the same levels as YUV. So sRGB<16 should all be black. RGB is full range from 0-255. What you see in VirtualDub is RGB, not sRGB. Of course, there's nothing to stop you from using a PC.601/709 matrix to convert that RGB as if it was sRGB. But you have to go out of your way to do that because it's not "normal".

[poisondeathray]

Clarify if you mean sRGB or Studio RGB . These are 2 different things

I agree with jagabo - every TV - even el cheapo ones - can definitely distinguish between RGB 0,0,0 vs RGB 16,16,16 . Most can distinguish between 0, 4, 8, 12, 16 .

I just tested this on several TV's. Both from a jpg image (some of my tv's can play images directly), and also from video converted to YUV using Rec601. Both display the same

[poisondeathray]

Unless jagabo objects, I'd say make it 100 instead of 200.

I don't object. The OP should do whatever he thinks looks best. But I'm not going to spend my time fine tuning his videos for him when he isn't willing to spend his own time learning anything.

Hahaha I thought maybe you're just sick of watching oiled up, muscular men rolling on the ground together ?

[sanlyn]

Ah. So I don't have to worry about RGB 16-235 ? That being the case, RGB 0-255 should look pretty good on every TV I see, even those with ISF calibration, and no problem encoding with those values. Won't have to admonish others for using RGB 0-255 for video, and don't have to edxplain stuff like crushed detail, etc.

I wonder why it looks like crap on TV, though.

[poisondeathray]

Ah. So I don't have to worry about RGB 16-235 ? That being the case, RGB 0-255 should look pretty good on every TV I see, even those with ISF calibration, and no problem encoding with those values. Won't have to admonish others for using RGB 0-255 for video, and don't have to edxplain stuff like crushed detail, etc.

I wonder why it looks like crap on TV, though.

What looks like crap ? If it's properly calibrated, the pure black point should be black, pure white should be white

You have to worry about legal range video , Y' 16-235 , CbCr 16-240 . RGB histogram is less useful when looking at YUV values for video, because the method in which you convert to RGB can change those values. It's when YUV values are "illegal" that you should worry, because those values will not be seen

RGB 0-255 is fine, when you convert to YUV using Rec 601 matrix or 709 for HD. This "maps" RGB 0,0,0 to Y' 16 , and RGB 255,255,255 to Y'235 . The reverse is done when you view the YUV video on an RGB display in 99.99% of hardware and software - so you see black and white the same as you started . This is why Rec601 and Rec709 are the standard for broadcast, hardware devices, DVD players, blu-ray , flash, even portable devices like ipad, ipod, "ithingys"

The exception is "studio RGB" used by some software (notably Sony Vegas) . This maps RGB 0-255 <=>YUV 0-255

[jagabo]

The exception is "studio RGB" used by some software (notably Sony Vegas) . This maps RGB 0-255 <=>YUV 0-255

MJPEG codecs are another exception.

[poisondeathray]

The exception is "studio RGB" used by some software (notably Sony Vegas) . This maps RGB 0-255 <=>YUV 0-255

MJPEG codecs are another exception.

Yes, and another common exception is Canon DSLR's which record h.264/mov - they record full range (along with full range flag in the bitstream). Using the standard Rec601 (i.e disregarding the full range flag) will clip shadows and highlights , it must be converted to RGB using full range, or a using a decoder that clamps the YUV output to Y'16-235 (e.g. quicktime) before the RGB conversion

[sanlyn]

So when I set up my yuy2 AVI for color correction in VirtualDub, Photoshop, AfterEffects, etc., I ignore their setup? For example, Color Finesse has a template that automatically rolls off brights from RGB 220 on up, rolling off to RGB 240. You can see it happening on their histogram, even when working in YUV. You're saying I should disable this feature and juice up bright white to 255? I set up a black-point in Photoshop for RGB 16, so the darkest black-point won't fall below that value. So I should always set the black point at RGB-000 for TV video? When I set up a TV with a pluge pattern for blacks, I should make certain that a black patch sits at the darkest plug level ("below black")? What's that stuff all about?

Maybe we're talking about different things. I owned one CRT TV 8 years ago that clearly displayed values below RGB 16 when using calibration test patches. That's the only TV I've owned that could do that. The old SONY Wega couldn't even come close, and my new HDTV's don't know RGB 0 from RGB 16, even when I yell at them. My colorimeter set on the TV screen seems to know the difference, but my TV's are ignorant below RGB 16 (well, actually, that patch has a subtitle that says IRE 0, but on my PC monitor that same patch displays as RGB 16 according to my pixel sampler).

Yeah. I think we're talking about different whatchamacallits. For 15 years I've had success working within 16-235 videos on the computer and having no problems with it on any TV I've played them on, even those godawful uncalibrated Panny and Sharp TV's playing in Volcano Mode. I think I'd better stick to what's been working so far.

[jagabo]

Maybe we're talking about different things. I owned one CRT TV 8 years ago that clearly displayed values below RGB 16 when using calibration test patches. That's the only TV I've owned that could do that. The old SONY Wega couldn't even come close, and my new HDTV's don't know RGB 0 from RGB 16, even when I yell at them.

How are you getting RGB to the TVs? Don't you use component cables with all your TVs? Maybe your problems are in the conversion from RGB to YUV for component transmission.

[poisondeathray]

So when I set up my yuy2 AVI for color correction in VirtualDub, Photoshop, AfterEffects, etc., I ignore their setup? For example, Color Finesse has a template that automatically rolls off brights from RGB 220 on up, rolling off to RGB 240. You can see it happening on their histogram, even when working in YUV. You're saying I should disable this feature and juice up bright white to 255? I set up a black-point in Photoshop for RGB 16, so the darkest black-point won't fall below that value. So I should always set the black point at RGB-000 for TV video? When I set up a TV with a pluge pattern for blacks, I should make certain that a black patch sits at the darkest plug level ("below black")? What's that stuff all about?

Maybe we're talking about different things. I owned one CRT TV 8 years ago that clearly displayed values below RGB 16 when using calibration test patches. That's the only TV I've owned that could do that. The old SONY Wega couldn't even come close, and my new HDTV's don't know RGB 0 from RGB 16, even when I yell at them. My colorimeter set on the TV screen seems to know the difference, but my TV's are ignorant below RGB 16 (well, actually, that patch has a subtitle that says IRE 0, but on my PC monitor that same patch displays as RGB 16 according to my pixel sampler).

Yeah. I think we're talking about different whatchamacallits. For 15 years I've had success working within 16-235 videos on the computer and having no problems with it on any TV I've played them on, even those godawful uncalibrated Panny and Sharp TV's playing in Volcano Mode. I think I'd better stick to what's been working so far.

I'm wondering if you're mixing up the readings on the test videos? The videos will be YUV if it's on a DVD. Did the markings say "RGB" ?

The absolute bottom line is you need to adhere to legal range values. Period. This means Y' 16-235. Y' 16 is "digital black" , Y' 235 is "digital white" . (Sure you're allowed a bit of overshoot and undershoot, but you should stick mostly within those values) . Notice RGB isn't even talked about.

Monitoring and measuring RGB is less useful, because you're not delivering RGB. YUV is the delivery format. There are dozens of different ways , in which you can convert the RGB to YUV and back and forth. It's those conversions that are the source of all the headaces.

If you've been using RGB 16,16,16 as "black" and RGB 235,235,235 as "white", then you must convert to YUV using a full range matrix. This will map black at Y'16, white at Y'235.

I doubt that your HDTV can't display between RGB 0,0,0 vs RGB 16,16,16 . This is quite a visible difference. If your TV or blu-ray/dvd payer can read JPG images, feed it a JPG image . Most likely you mean between Y' 0 and Y' 16. I suspect you're mixing something up or didn't do the test properly

For the pluge bar, you should see the visible demarcation between 7.5 and 11.5 IRE - but this is an analog test signal sent by equipment. If it's from another source, it's hard to say what it should be...

[VideoFanatic]

Sanlyn, you're right. The video looks virtually identical on my TV with or without the ColorYUV(levels="PC->TV" line. I take it there's no way to emulate on my PC the exact way it would look on my TV?

The video I posted earlier with the darkness corrections you guys said it didn't look right as that video wasn't meant for the corrections given. I just viewed it on my PC in AvsPmod and thought the corrections looked better but you pointed out all the imperfections caused by those corrections. That's why I was reluctant to meddle with ColorYUV as it's easy to make mistakes like that. I'll just use that darkness correction line on videos with really obvious darkness problems. I don't think I'll be meddling with ColorYUV again.

[themaster1]

Me think you may want to reconsider:

with pc>tv:
http://imageupload.org/en/file/246976/with.jpg.html
without:
http://imageupload.org/en/file/246977/without.jpg.html

[sanlyn]

@jagabo + poisondeathray, as it's impossible (Okay, impractical) for me to hook my PC's to HDTV sets, I'm forced to use a couple of IPS LCD's and Windows PC's to process video. All this viewing is done with monitors/TV calibrated with EyeOne colorimeters and the proper software (XRite for PC's, Calman or HCFR for TV), with accepted SD and HD calibration test patches for grayscale, saturation, gamma, primary/secondary CIE measurements, and all that. Unfortunately I can't afford a $1500 color generator, so I have to use SD and HD material on DVD/BD discs for the TV's. Can't say what JPG photos look like on my TV's that are calibrated for disc playback and cable viewing.

Enjoy your component cable setup while you can (as I do), they are disappearing from TV's and from a/v receivers and disc players alike, replaced by this horrid/freaky/paranoid/dumbass HDMI junkware for the toy crowd and the hearing/seeing/match-the-red-plug-to-the-red-hole impaired who truly believe that all wire and everything "digital" looks and sounds alike. Not using my TV as a photo viewer, essentially because seeing how digital cameras (and especially cheap digital cameras, iphones and whatnot) are so visually crippled compared to film, I find that looking at JPG images on my TV (or even on my PC) is one of those yawn-inducing experiences that just hasn't wedged itself into a niche in my daily schedule.

So a PC and IPS LCD's are what I use to correct and arrange video for playback on my TV and other TVs. I realize there are molti problemi going from YUV -> RGB and around in that circle. I make those conversions only as needed, not haphazardly but in the careful and prescribed manner learned here and elsewhere, even down to using dither() for conversions, never allowing software to make its own permanent conversions a la VirtualDub, setting PC hardware and software to mimic BT601 or 709 display according to instructions, and all that mind-boggling + time-consuming stuff.

So I realize that one of my TV's, and one of my former CRT's, could be used as an RGB 0-255 device. But one of the tv's doesn't fare very well as a full-range PC display and doesn't like cable broadcasts that sneak from the cable boxes with illegal luma and chroma down there in the RGB-0 range (or even lower). Back when I first started ruining my mind with this video hobby, I said bah-humbug to all that invalid video superstition+voodoo....until movies that looked gorgeous on my then-alive CRT started looking like doodoo 75% of the time elsewhere, and looked like blocky, blotchy detritus on LCD's. So I learned to look at video in properly converted RGB with PC stuff that purportedly treats YUV the way a TV would. I realize that it's not an exact-science transition from PC to TV, but I try to get close to the mark. My target is still the "standard" TV that works within a 16-235 box, one that doesn't try to be an all-purpose m/m 'bot-twitter-read-my-mail center.

Anyway...."'God bless us, every one!', said Tiny Tim."