How to correct temporal luma variations from capture

[anegroo]

Hi everybody!

I shot a video (with static background) with automatic gain (exposure) controll. When a white object appears, the background becomes darker.

I would like to keep constant the background luminosity (on a darker level).

My idea: I want to analyze the "v1" luma (luninosity?) value of a certain area -with x,y coodinates- from a captured picture with darker background and "v2, v3, v4, ..." of the same area of all frames. Than change the luma of the same area in all frames, and accordingly to the difference of this two values, change the value of all pixels from all frames... I don`t know how...?

In the picture from this link: http://aycu40.webshots.com/image/35239/2000265358422948564_rs.jpg I put together three stages of variation (frame [0,11,22] - [min., middle, max.])

I started a topic on neuron2.net forums too with this problem but with no answers. Maybe here...

My insatisfying solution still is an avisynth script:
code:
###
v1=AVISource("video.avi")
v2=ImageReader("GoodCapure.png",0,1425,25). Crop(12,4,-594,-460). ConvertToYV12 # Cropping the desired "static" area from captured image
v3 = v1.Crop (12, 4, -594, -460). ConvertToYV12 # The same area to compare

ScriptClip (v1, "levels(0,1+ AverageLuma(v2)/AverageLuma(v3), 255,0,255)")
###

"GoodCapture.png" here is a capture from a frame with the desired darkness. The value of gamma is influenced by "AverageLuma(v2)/AverageLuma(v3)".

Perhaps AverageLuma measures different thing than gamma, but the variations are reduced a bit, but not enough.

Any idea?

[jagabo]

Perhaps AverageLuma measures different thing than gamma.

Yes. Off the top of my head, I suspect you will want to use ColorYUV() to adjust the off_y and gain_y values. Keep in mind that the arguments to ColorYUV() are scaled integers, not floating point.

This sounds like a fun little project. Can you supply a short clip?

[anegroo]

Here is a test file:

http://www.bestsharing.com/files/0RLRMk372454/Test-Clip.avi.html

[jagabo]

This script will keep the green patch at the top of the screen the same average intensity and retain the black level:

########################
#AviSynth script

BLACK = 16
DESIRED = (96-BLACK)

source = AVISource("Clip.avi").ColorYUV(off_y=-BLACK)
sample = source.Crop(8, 16, -8, -472)
adjusted = ScriptClip(source, "ColorYUV(gain_y = (DESIRED/AverageLuma(sample))*256-256)")
return(adjusted.ColorYUV(off_y=BLACK))
########################

BLACK is the black level of the source. 16 is the normal black level for digital video in YUV. Subtracting out the black level before adjusting the gain keeps the black level from changing. And, of course, it needs to be added back in at the end.

DESIRED is the luminance of the green area at the top of the frame at frame 34. I use this as a base line -- all other frames are adjusted to match this. It is the value before BLACK has been subtracted from the frame. I got the value using AverageLuma() on the cropped section.

ColorYUV(gain_y=int_gain) uses scaled integers:

int_gain = (fp_gain - 1.0) * 256.

So a unity gain (1.0) would be 0, a 2x gain would be 256, etc.

The last line adds the black level back into the image. The black level should be 16 so you might add 16 rather than 20. Many camcorders have slightly high black levels.

The final result keeps the green area at the same luminance but the white level doesn't stay real constant (note the brightness of the labels on the keyboard). To get better results you'll probably have to adjust the gamma as well as the gain.

The output as a small XVID AVI:

output.avi

[anegroo]

Hi jagabo!!! Many thanks for the script and the great explanation!

It is a nearly perfect solution. So, it remains to adjust the gamma.
In this line from my script: ScriptClip (v1, "levels(0,1+ AverageLuma(v2)/AverageLuma(v3), 255,0,255)") I tried that thing. But probably the "1+AverageLuma(v2)/AverageLuma(v3)" value of gamma is totally wrong. I don`t know how to measure the gamma value of original video and the difference between this and the sample image - or the variation of it. The next: "1+(AverageLuma(v2)-AverageLuma(v3))" also not work. I have to take in account something else that AverageLuma? Do you have an idea?

[jagabo]

Some reading about gamma correction:
http://en.wikipedia.org/wiki/Gamma_correction

In short, changing the gamma changes the brightness and contrast in the mid tones without effecting the darkest and brightest areas.

I fiddled around with changing the gamma as well as the gain and got this for the ColorYUV() line:

adjusted = ScriptClip(source, "ColorYUV(gain_y=(DESIRED/AverageLuma(sample))*140-140, gamma_y=(DESIRED/AverageLuma(sample))*100-100)")

Note the change in gain from "*256-256" to "*140-140". The two numbers should change together. We're adjusting how much the gain is scaled. And we're now changing the gamma in a similar manner.

The output is a little better but there's still a color saturation change. The camera was probably adjusting the white balance too. You can try adjusting the gain and gamma parameters to further improve the video.

If you want perfect results you'll need to reshoot with the camera's auto white balance and auto exposure settings disabled.

[anegroo]

Yes, I see that this is not a simple problem as I thought initially. I`ll try to play with this values but probably - as you wrote - I have to be reshot the video with manually adjusted white balance ... Anyway, jagabo, thanks for your time and help!

[jagabo]

Good luck!