Using Avisyth Overlay for Contrast Masking

[sanlyn]

Looks as if I need a more sophisticated understanding of masking to improve contrast in some DVD videos. The scene I'm working with is from a recording of Trader Horn with low contrast and a dim image. Recorded off SD digital cable at ~9000 VBR with a DVD recorder. I've seen countless old b&w and color movies with this problem. I could apply similar fixes to other recordings. But I don't have the hang of masking yet.

Attached horn_sample1_ivtc.avi is lossless Lagarith. The only processing was TIVTC, nothing else. At this point I don't care about noise or other junk, just levels. My script is rudimentary stuff; take my word for it, I've tried lots of filters, gamma fixes, level dilters, whatever. It appears to me that contrast masking is the best (maybe only) way to improve this puppy. The big problem here is learning how to curb the "hot" brights I'm getting with this script:

AviSource("Drive:\path\to\video\horn_sample1_ivtc.avi")

SmoothLevels(15,1.0,255,15,230,chroma=200,limiter=0,tvrange=true,smooth=200,dither=100,protect=6)
Greyscale()
a=last
mask=a.Invert().ColorYuv(cont_y=-400,gamma_y=60,off_y=-10)
overlay(a,0,0,mask,mode="add")
return last

An example of the original from DVD and the results of that script. Detail on clothing and hands are burned away, despite being below RGB235. The hot spots aren't there in the original. So how would I bring up lows and mids without knocking the heck out of the highlights?


[Attachment 21018 - Click to enlarge]

[Attachment 21019 - Click to enlarge]

I tried all of Avisynth's autogain plugins. Same results, or worse. Big problem with autogains is that they "gain" when they shouldn't. The script attempts to build an inverted contrast mask whose levels are the opposite of those in the original. I don't seem to be doing it correctly.

[poisondeathray]

It appears to me that contrast masking is the best (maybe only) way to improve this puppy.

Usually more than one way to do things....

The big problem here is learning how to curb the "hot" brights I'm getting with this script:

The reason why you're clipping is you're using "add" blending mode . You can modulate the effect by playing with the opacity parameter in overlay. Or another approach would be adjusting the mask

An example of the original from DVD and the results of that script. Detail on clothing and hands are burned away, despite being below RGB235. The hot spots aren't there in the original.

I think you better recheck that - there is definitely clipping >RGB255 in both images. "Add" mode is the the reason for the farther loss of detail - brighter pixel values get brighter . I think avisynth's "add" math might be slightly different than photshop's or AE's but anything >128 will clip because 128+128=256 (I'm referring to the plain operation, without a mask)

So how would I bring up lows and mids without knocking the heck out of the highlights?

One "plain jane" approach in avisynth would be to use hdragc

eg.
hdragc(max_gain=0.5, coef_sat=0, shift=3)

Tweak it by looking at the histogram and playing with the values

Another way would be to use RGB curves e.g. in vdub or avisynth's gimp curves plugin gicocu

But if you're trying to modify your specific script without blowing highlights, and adjusting the opacity parameter or adjusting the mask doesn't quite do it for you, you might need some masktools magic to impose upper limits , or use a secondary luma mask

[manono]

One "plain jane" approach in avisynth would be to use hdragc

eg.
hdragc(max_gain=0.5, coef_sat=0, shift=3)

Another would be to adjust SmoothLevel's 'brightSTR' setting.

[jagabo]

First of all, wouldn't you be better off using gamma to bring up the darks and mids?

You are adding the processed image to itself using that Overlay((last),a,0,0,mask,mode="add") command, not blending it with something. The mask in that case determines how much the second image is added to the first.

You are inverting the image twice to make the mask. First with Invert(), then with ColorYUV(cont_y=-400...). So your result is a positive mask not a negative mask.

The end result is that the processed image is added to itself with the brightest parts getting brighter more than the darker parts.

I'm not sure exactly what you were attempting to do with that overlay. Maybe changing the mask to:

mask=a.ColorYuv(cont_y=-400,gamma_y=60,off_y=-100)

gives something more like what you wanted?

mask:


output:

[sanlyn]

Yipes. Well, thanks to evreyone. Several good ideas here. But first: I played with gamma, curves, and whatnot (very spikey histogram) and AGC did levels changes several times during the remainder of the scene because the camera follows these guys around and encounters big dark objects, big bright spaces, etc.

OK, I see what Inverse is doing, then. That's what I get for copying an old script and modding it without knowing what I was doing. Yes, the first thing I noticed is that the masking here doesn't work per Photoshop or AE. Jagabo's idea looks pretty good. I'll give that one a try later tonite, then maybe tweak if necessary with SmoothAdjust or whatever to get the black levels just-so. Meanwhile my current PC repair client might not like watching me try it on his PC (at least I managed to get his internet connection going again!). Thanks for checking. Let's see if I can learn something new.

[jagabo]

The result of the Overlay(mode="add") is the darkest areas get a=a+a, the brightest areas a=a+0, greys in between something between those two depending on the mask.

[sanlyn]

Yews, that's why I was looking into masktools, where one can set limits with those wild luma and chroma strings. Other than fighting the polish notations, I'm still trying to figure out what those y=+128, -128, +255, etc., are doing. I know they're settimng up tonal ranges, but I don't see any docs or threads telling us how.

[poisondeathray]

Yes, the first thing I noticed is that the masking here doesn't work per Photoshop or AE. Jagabo's idea looks pretty good.

It's actually the same thing .

In Adobe terminology, it's called a "track matte" or a "luma matte". It's called a luma matte because the alpha (transparency) values are shown as luminance. 100% white areas are transparent, 100% black are opaque . ie. 100% white would have the upper layer "show though" , 100% black areas would have the bottom layer "show through"

That's the exact same thing in avisynth when using mt_merge() or overlay() with mask layer - the mask layer is the "luma matte"

As for the layer blending modes, IIRC there are slight differences in the math operations between avisynth and the way photoshop / AE does it . You can also have "brighter than white" and "darker than black" in 32bit float , and there is higher internal precision for the calculations

[poisondeathray]

Yews, that's why I was looking into masktools, where one can set limits with those wild luma and chroma strings. Other than fighting the polish notations, I'm still trying to figure out what those y=+128, -128, +255, etc., are doing. I know they're settimng up tonal ranges, but I don't see any docs or threads telling us how.

Totally agree - I'm still fighting those "Polish notations" , lutxy business...

That's one area that needs way more documentation and examples

I use premade functions like lumamask() , where you can specify the black and white points with b=, w= . As for the "mechanics" behind it, no idea. I just use functions, don't really code them . Another useful one is maskhs() where you can specify hue ranges (it's like color keying really)

[sanlyn]

Thanks for the extra notes, jagabo and pdr. When I get home later I'll try to un-confuse myself with some pre-made masking routines. One thing: working with grad curves, etc., I ran into the same problem: blown out highlights, but evreything else looking OK. But what convoluted curved and spikey/dipping histograms I came up with, made the tonal range look like just so much noise and grit.

[jagabo]

I'm still fighting those "Polish notations" , lutxy business...

You can use standard math notation by using mt_polish().

Reverse polish notation uses a stack concept. Numbers are stacked up, operators act on the top two number of the stack. A sequence like:

3 2 1 + *

results in 3 being put on the stack. 2 being placed above it. 1 above that. (If you were doing this on an HP calculator you would have to press ENTER between 3 and 2, and between 2 and 1.)

1
2
3

+ then acts on the top two values (1 and 2) resulting in 3 which is left on top of the stack:

3 (the result of 1+2)
3 (the original 3 we entered)

* now acts on the two values at the top of the stack (3 and 3) and results in 9 being left at the top of the stack.

I suspect the main reason the author used RPN was because it's easier to parse.

http://www.calculator.org/rpn.aspx

[poisondeathray]

Thanks jagabo .

But I'm going have to add entry # 450937 to my text list of things to read up on .... (you have a quite few posts on that list).

One day I'll tackle it.. I keep on saying that LOL

But what convoluted curved and spikey/dipping histograms I came up with, made the tonal range look like just so much noise and grit.

There is only so much you can do with noisy, overcompressed, 8bit footage....

[jagabo]

But I'm going have to add entry # 450937 to my text list of things to read up on .... (you have a quite few posts on that list).

I learn a lot from your posts too.

[sanlyn]

Learning from both of you, and having the devil of a time trying to keep up[. But making progress. Getting closer to what I want here, considering the 1930 vintage, a bad theatrical print, and a low-bitrate broadcast. I managed to get a little more brillaince into the scene and kept most of the highlights without creating a "glow" up there or crushing the murky darks. Even did a little denoising (but that river shot is going to take more work). Borrowed some work from Didee and others on doom9.

AviSource("Drive:\path|to|vieo\horn_sample1_ivtc.avi")
Grayscale()
YLevelsG(20, 2.0 ,255, 12, 225)
QTGMC(preset="medium",InputType=1)

a1=last
a2=a1.Dfttest().ColorYUV(gamma_y=5)
LumaMask(a2,a1,b=16,w=80,upper=true)
# ---- minjor tweaks to results of above -----
SmoothLevels(16,1.1,255,12,255,chroma=200,limiter=0,tvrange=true,smooth=200,dither=100,protect=0)
FineSharp(sstr=0.02)
grainfactory3(g1str=4, g2str=4, g3str=4)
AddGrainC(1.5, 1.5)
return last

function YlevelsG(clip clp, int a, float gamma, int b, int c, int d)
       { wicked = gamma > 1.0
          \      ? "x "+string(a)+" - "+string(b)+" "+string(a)+" - / 1 "+string(gamma)+" / ^ "+string(d)+" "+string(c)+" - * "+string(c)+" + x * x 255 x - * + 255 /"
          \      : "x "+string(a)+" - "+string(b)+" "+string(a)+" - / 1 "+string(gamma)+" / ^ "+string(d)+" "+string(c)+" - * "+string(c)+" + 255 x - * x x * + 255 /"
         return( clp.mt_lut(Yexpr = wicked, U=2,V=2) )  
         }

function LumaMask(clip filtered, clip raw, int "b", int "w", bool "upper", bool "show"){

raw      = default(raw, filtered)

LO       = string(default(b, 24))
HI       = string(default(w, 48))
upper    = default(upper, false)
show     = default(show,  false)

code     = upper ? "x "+LO+" < 255 x "+HI+" > 0 255 x "+LO+" - 255 "+HI+" "+LO+" - / * - ? ?" : \
                   "x "+LO+" < 0   x "+HI+" > 255 0 x "+LO+" - 255 "+LO+" "+HI+" - / * - ? ?"

msk      = raw.mt_lut(code,u=-128,v=-128)

show  ?  msk : mt_merge(raw, filtered, msk, luma=true, U=3,V=3)
}

Will keep at it.

[2Bdecided]

I think you just need some straight "tone curve"-style levels manipulation.

Applying this curve to the full range data gives the following result.





(done in a photo editor - I've never done this in AVIsynth, but there must be a way. You need something with a decent GUI to figure the correct values out though - a small change to some of the points on the curve can give a big change to the picture.)


Cheers,
David.

[Deter]

This is an interesting post. Is it possible to clean up some of the picture bleeding or dullness and give the picture or movie a clean detailed look? Working with dull video is a real challenge, now I went and messed around with a ton of different filters and came up with this pic, however this was done in corel paint and it is still terrible. However my goal was to sharpen the hell out of the picture, fix some of the color problems, to try to fix the dullness. The end result is my next post.

FYI (it is the lighting that gives the brightness on the guys hand)

[Deter]

Look at the picture I posted and it looks terrible, reduce the size add boarders and change the tint and you get this out of the crap I posted above..

[Deter]

Than you just recolor it back to B&W, fix it a bit more so it kind of looks like a photo than some crappy VHS screen capture. Next you get all the frames to look like this (I did this fast you want a better model to use), clean up all the errors in the video. Play it back on a projector, recapture the entire footage on film. Kind of like what they did with the old WWII footage. Than work like hell on trying to restore the mono recorded audio. That would be in a perfect world and cost a ton of $$$$.

However if they still had the original film that would bypass some of these steps. The above posts would be the idea if the source film was lost or damaged. Just for personal use, you would be crazy to do what I wrote about.

check out these links:

http://trekcore.com/blog/2012/09/exclusive-trekcore-interviews-cbs-digital-part-i/


http://trekmovie.com/tng-remastered/

[sanlyn]

Still working on it. Thanks for the notes. Many things to consider.
BTW, this scene is posted on the net by TCM at http://www.tcm.com/mediaroom/video/496538/Trader-Horn-Movie-Clip-They-Have-A-Telegraph.html . You have to tolerate a 29-sec ad before the clip starts. If you scroll ahead in the posted clip, this specific sequence begins at 3min 55secs. Two other scenes are posted.

[sanlyn]

I think you just need some straight "tone curve"-style levels manipulation.

Applying this curve to the full range data gives the following result.
Cheers,
David.

That's exactly where I started, but with Photoshop curves. I had similalr results. S0 I went with Overlay(), which gave me something closer to Deter's post and exaggerated contrast. I guess it depends on what you're working for. Everyone will want a different result. Comparisons:

Original, from the otherwise unprocessed ivtc'd AVI posted earlier (YUV and Rec601-RGB histograms):

[Attachment 21082 - Click to enlarge]

From the above you can see there's no low-level detail, so you might as well leave the darks where they are. The scene was apparently shot without the aid of reflectors and fills -- who's going to lug that hardware around in the African jungle in 1930 with 500 untrained natives all over the place and the camera tracking for a long journey through a real-life jungle village? So you have strong overhead lighting and deep shadows. To me it would be desirable to get some luma in the RGB 32-100 range or so, but not a lot of it; the highlights are already rather bright (they already exceed RGB 255). 2B's curves version has a Hollywood lighting effect of sorts. Should have been lighted more along these lines anyway, but...

[Attachment 21083 - Click to enlarge]
I did that the same thing in Photoshop first, then copied the curve settings to VirtualDub's gradation curves. Nothing really wrong here, but I think it lacks some punch and is missing an impression of depth or dimension. The strong lighting contrast is somewhat undone, but of course a curves filter can be tweaked. So I tried masking to hit only the RGB 32-100 range:

[Attachment 21084 - Click to enlarge]

I don't think you can "stretch" the intended values much farther. The histograms show spikes and dips that would result in banding (but I intend to apply some dithering later). Right now it looks over filtered and lacks some "grit", but I can fix that. The image above came from the second script I posted, with some very mild help from gradation curves in the RGB 16-64 range to keep some contrast in the texture of those straw huts and some background depth. Only problem with these demo images is the way the forum image viewer darkens pictures in a browser; these are brighter in photo apps, players, and editors with a calibrated monitor. All of these images are brighter on a TV, to the point where the original, looks washed out.

Wait til I get to the night sequence, where you can't see a damn thing:

[Attachment 21085 - Click to enlarge]

[Deter]

Sharping the mask would help. To my knowledge the best restore job doing what you are kind of trying is
DOCTOR WHO AND THE SILURIANS

[2Bdecided]

The scene was apparently shot without the aid of reflectors and fills -- who's going to lug that hardware around in the African jungle in 1930 with 500 untrained natives all over the place and the camera tracking for a long journey through a real-life jungle village? So you have strong overhead lighting and deep shadows.

It probably looked great projected from the original Nitrate film in the 1930s. In a cinema / movie theatre, your eyes adapt to the light level on screen. Many parts of real films were often far darker than what we expect on TV, but in that darkness they had great detail and charm which is lost when viewed on TV. It's hard to know (and highly subjective) how best to translate them.

Cheers,
David.

[sanlyn]

All true. A CRT has better luck with this kind of image. But I did see this movie on the big screen when it was revived in the early 1950's. I thought it was such a cool movie, I watched it 3 times. In the big cinema house I recall that it was not all that sharp, and not all that bright, but the darker areas were indeed not as dense as the TCM broadcast or the VHS tape (and the tape is so godawful I won't work with it). Our home's beloved calibrated CRT died a couple years back, but I saw this recording on my parents' fairly decent Sharp CRT and -- I remain convinced: LCD's might have better geometry, but otherwise they really really really suck.

[MattiasN]

i have done some testing to here is a function script i did do but HDRAGC still maby works best

function hdr1(clip movie)
{
movie_high=movie.ConvertToRGB().ColorBalance(rs=150,gs=150,bs=150,rm=150,gm=150,bm=150,rh=19,gh=19,bh=19,keep_luma=false,clone_gimp=false,highcolor=false).ConvertToYV12()

mask=movie.Grayscale().ColorYuv(cont_y=-400,gamma_y=-252,off_y=64).GaussianBlur(VarY=100).ConvertToRGB().ColorBalance(rs=150,gs=150,bs=150,rm=150,gm=150,bm=150,rh=0,gh=0,bh=0,keep_luma=false,clone_gimp=false,highcolor=true).ConvertToYV12().Levels(0,1.000,175,75,175,true)

film=overlay(movie,movie_high,mask=mask,opacity=1.0,greymask=true,mode="blend",pc_range=true).Tweak(sat=1.45)

return film
}

you need a plugin called ColorBalance_0.26.zip i did find it here

http://www.avisynth.info/?%A5%A2%A1%BC%A5%AB%A5%A4%A5%D6#z066d47f

or get it here

[sanlyn]

Thanks for the tip. You can't use autogain or autocolor controls for most video. Pumping of contrast and fluctuations of color usually result, especially with camera motion and various objects and lighting angles changing constantly. They work best for static scenes with little motion and where objects in the scene don't change very much. There's also a problem with converting high-noise low-bitrate sources to RGB and back to YV12 again.

I've since found some better masking ideas but haven't had time to post.

[manono]

Yes, a very nice link. What's killpulse do? The text file is in Japanese. What I'm looking for is something to stop luma fluctuations over several second intervals. The kind of stuff you get with old silent films or films that haven't been taken care of properly.

[sanlyn]

I was going to try killpulse on a video I'm cleaning that has the same problem you describe -- although I don't know that the fluctuations last for "seconds", but they spread over 30 to 45 frames or so. I've used autocolor and autowhite on those shots, but you end up with the second problem I described: If a dark or bright object enters the scene or the camera angle changes, etc., you have several seconds that you have to mask with different versions of the scene that you work in/out with dissolves, etc., etc.,. And autowhites tend to blow out darks or brights and have to be corrected again. I read where Disney shops have a device that levels that problem, but you could buy two BMW's for what it costs. So I'm still looking for a workaround.

[manono]

Yeah, I wasn't clear. It'll get bright for 2-6 frames or so, and these fluctuations occur every several seconds, but not in any sort of a regular cycle. It's the kind of thing fixed by Diamant's DFlicker filter:

http://www.hs-art.com/files/diamant/samples/DFlicker.swf


You'd think something like this would be possible with AviSynth. If anyone wants to attempt it, I have lots of samples.

[MattiasN]

Hi hmm this did work very good to deflicker


http://forum.doom9.org/showthread.php?t=166355
Post nr: 19


But you get little color bleeding i think it's called :/

[manono]

Thanks. They're black and white films so color bleeding isn't a problem. I've used similar deflicker scripts before and, while they reduced the flicker they also softened the picture noticeably. I'll give these a try. Thanks again.