Restoring/ compensating overexposed, heavily clouded weather, video's

[Rockel83]

I'm working on some video's. And using Pinnacle Studio Ultimate 24 for editing.

Now I came across some vacation video's which were recorded in heavily clouded weather. It was high up in the Andes, so the sun is verry sharp there. But it seems I didn't noticed verry well because of the clouds. But it seems the camera sensor was quite sensitive to it.

At some points the video gets almost dark and colors dissapear. I would like to try to compensate this to get some more usable images. But I'm quite struggeling with it. The video is quite dynamic, because I'm turning around. When turning over the square, colors and brightness is normal again. So I can't put some effects over the whole clip, but I need to use some keyframes. Sliding "exposure" up at some intervals makes a difference. But still colors look realy bad at some points.

Should I keep adjusting things here with Pinnacle and does someone have some setting reccomendations for me? Or is there another alternative which works better on restoring/ compensating the colors and brightness of this kind of overexposed video?

https://www.youtube.com/watch?v=c1BH2ka4eKM

Here is a video sample of it:
https://we.tl/t-RTdqO48f4p

This are the settings and keyframes I was trying so far within Pinnacle:

[Attachment 65912 - Click to enlarge]

[themaster1]

adjust the gamma it works rather well ,i just tried (i used sony vegas secondary color plug-in); gamma to max nearly 1,789 and limit luminance highlights for the sky to 217,421. Adjust saturation aswell, just a tad

[Rockel83]

That's quite some difference yes. That was one of the darkest parts. But unfortunately I can't find some gamma-settings in Pinnacle. Could it have another name here?

edit:
Is this "sony vegas secondary color plug-in" a Pinnacle plug-in?

[johnmeyer]

I don't know what features Studio has added since I last used it twenty years ago. Vegas has a whole host of features for correcting exposure including not only several gamma functions (which gains the intermediate levels without affecting deep shadows or highlights), but also has a "color curves" function which lets you create your own custom histogram for each color channel. Normally I create a histogram for all three channels and, for something like your video, I'd increase the gain on pixels in the darkest 1/3 of the video while leaving the brighter sections untouched.

AVISynth has some autogain functions, but I've never gotten them to work reliably.

[Rockel83]

Thanks! I think it's something like the "Tone Curve" and "HSL Tuning" tab in Pinnacle which you are talking about here. I'm playing around with the "Tone Curve" section now, being able to get some nice colors.

But I'm strugeling getting some decent coors back on the almost black parts. I need to increase exposure here up to 70, going up much furthe my colors disapear and I get more of a B&W image. Guess I need to play around a bit here.

I want to change to other editing oftware, because this Pinnacle Studio is also verry buggy. Loosing a lot of time with it sometimes. Maybe I will look after Vegas too. But it looks it only has subscriptions nowaday?

[Sharc]

You could apply level corrections with Avisynth.
Problem is that some scenes have crushed blacks and clipped whites. While the crushed blacks can be fixed with gamma correction for example, the clipped out-of-range whites can be legalized and visually somewhat improved. But what is lost by the clipping is lost (Frame 44+) and can't be fully recovered as I see it.

[Attachment 65916 - Click to enlarge]

[themaster1]

with avisynth you can try this:

LWLibavVideoSource("C:\VID_20220219_135613910.mp4" )
AutoAdjust(auto_gain=true,gain_mode=0,avg_safety=1 .00,scd_threshold=0,input_tv=false,output_tv=true, dark_limit=17.0,bright_limit=1.50,gamma_limit =4.0,dark_exclude=1.00,temporal_radius=2,high_qual ity=true,high_bitdepth=false)
tweak(sat=1.3,coring=false)

[Rockel83]

with avisynth you can try this:

LWLibavVideoSource("C:\VID_20220219_135613910.mp4" )
AutoAdjust(auto_gain=true,gain_mode=0,avg_safety=1 .00,scd_threshold=0,input_tv=false,output_tv=true, dark_limit=17.0,bright_limit=1.50,gamma_limit =4.0,dark_exclude=1.00,temporal_radius=2,high_qual ity=true,high_bitdepth=false)
tweak(sat=1.3,coring=false)

Thanks, I'm trying this now. But I'm getting an script error in Line 2, column54. It says there's a "," or a ")" expected. But I don't see annything strange.

[Rockel83]

You could apply level corrections with Avisynth.
Problem is that some scenes have crushed blacks and clipped whites. While the crushed blacks can be fixed with gamma correction for example, the clipped out-of-range whites can be legalized and visually somewhat improved. But what is lost by the clipping is lost (Frame 44+) and can't be fully recovered as I see it.

[Attachment 65916 - Click to enlarge]

Thanks, however I'm not that verry advanced with Avisynth. Do I need to adjust to normall parts and underexposed parts manually here? Or is it automated?

[Sharc]

Thanks, however I'm not that verry advanced with Avisynth. Do I need to adjust to normall parts and underexposed parts manually here? Or is it automated?

Manual adjustments, scene by scene. There are auto-level scripts available in avisynth, however I have not much experience with these. I will give it a shot to see how it goes .....

[Sharc]

... and here a tentative quick test with Avisynth's autolevels. Not optimized at all.
Left original / right autolevels for comparison.

[Rockel83]

... and here a tentative quick test with Avisynth's autolevels. Not optimized at all.
Left original / right autolevels for comparison.

Oh wow! This actually looks much better and more stable with less flickering! How do I achieve this?

So it's not optimized, so I could even tweak it a bit more?

[jagabo]

with avisynth you can try this:

LWLibavVideoSource("C:\VID_20220219_135613910.mp4" )
AutoAdjust(auto_gain=true,gain_mode=0,avg_safety=1 .00,scd_threshold=0,input_tv=false,output_tv=true, dark_limit=17.0,bright_limit=1.50,gamma_limit =4.0,dark_exclude=1.00,temporal_radius=2,high_qual ity=true,high_bitdepth=false)
tweak(sat=1.3,coring=false)

Thanks, I'm trying this now. But I'm getting an script error in Line 2, column54. It says there's a "," or a ")" expected. But I don't see annything strange.

The forum software added extra spaces in the long line because themaster embeded the script in a quote block rather than a code block. Use this:

AutoAdjust(auto_gain=true,gain_mode=0,avg_safety=1.00,scd_threshold=0,input_tv=false,output_tv=true, dark_limit=17.0,bright_limit=1.50,gamma_limit=4.0,dark_exclude=1.00,temporal_radius=2,high_quality=true,high_bitdepth=false)

[Rockel83]

Thanks Jagabo!

Guess I'm missing something here.
"There's no function named "AutoAdjust"".

[Sharc]

So it's not optimized, so I could even tweak it a bit more?

Perhaps a little better / less washed out colors.

autolevels(autogamma=true,input_low=0,input_high=255,output_low=7,output_high=232,coring=false)

You need to download and put autolevels (or autoadjust) in your avisynth plugins folder
http://avisynth.nl/index.php/Autolevels

[Rockel83]

with avisynth you can try this:

LWLibavVideoSource("C:\VID_20220219_135613910.mp4" )
AutoAdjust(auto_gain=true,gain_mode=0,avg_safety=1 .00,scd_threshold=0,input_tv=false,output_tv=true, dark_limit=17.0,bright_limit=1.50,gamma_limit =4.0,dark_exclude=1.00,temporal_radius=2,high_qual ity=true,high_bitdepth=false)
tweak(sat=1.3,coring=false)

Cool! Got it working! Great results!

So it's not optimized, so I could even tweak it a bit more?

Perhaps a little better / less washed out colors.

autolevels(autogamma=true,input_low=0,input_high=255,output_low=7,output_high=232,coring=false)

You need to download and put autolevels (or autoadjust) in your avisynth plugins folder
http://avisynth.nl/index.php/Autolevels

Got it working too! Thanks!

[Selur]

Using Retinex (https://github.com/Asd-g/AviSynth-Retinex) might also be worth a shot, depending on how bright on wants it.

[Sharc]

Using Retinex (https://github.com/Asd-g/AviSynth-Retinex) might also be worth a shot, depending on how bright on wants it.

Interesting. A quick shot with Retinex here. The sky comes out pretty dark (how did it really look?). Maybe there are more apropriate settings.

converttoYV24()
MSRCP(fulls=false,fulld=false,lower_thr=0.0,upper_thr=0.0001,chroma_protect=2)
converttoYV16()

[Selur]

You could also use a luma/limit mask to apply retinex to everything but the sky.
Just wanted to point out that retinex might offer a different view.

[Rockel83]

I really like the results of:
- AutoAdjust
- autolevels
- Retinex

But it just depends on the type of video, it's collors and it's "exposure".

I like the vibrant colors of AutoAjust, but sometimes it's a bit too much. Then autolevels fits better. But to my opinion autolevels is a bit darker and lacks a bit of detail or depth in some video's over AutoAdjust. Both probably just needs a little bit of tweaking.
And in some shots Retinex works pretty well, it gets a lot of detail back and it brightens verry dark spots verry well. But it's just at the cost of the sky colors in all of my video's. In other video's the colors get a bit too vibrant, even more then with AutoAdjust.

Maybe it just needs some tweaking or maybe I should just try on which video what plugin works best

[Sharc]

You could also use a luma/limit mask to apply retinex to everything but the sky.
Just wanted to point out that retinex might offer a different view.

Good idea applying a mask.
Here a sample. Not perfect, but perhaps something to begin with.

lmask=clip.mt_binarize(110).Blur(1.5).Blur(1.5).Blur(1.5).Blur(1.5).invert() # build a mask based on luma
Overlay(clip, converttoYV24().MSRCP(fulls=false,fulld=false,lower_thr=0.0,upper_thr=0.0001,chroma_protect=3), mask=lmask)
smoothlevels(input_low=16,input_high=255,output_low=16,output_high=235) #adjust luma to TV range 16 ..... 235

Edit: same but with stronger mask blurring:

lmask=clip.mt_binarize(110).converttoYV12().GaussianBlur(varY=5000).invert() # build a mask based on luma
Overlay(clip, converttoYV24().MSRCP(fulls=false,fulld=false,lower_thr=0.0,upper_thr=0.0001,chroma_protect=3), mask=lmask)
smoothlevels(input_low=16,input_high=255,output_low=16,output_high=235)

[Sharc]

Maybe it just needs some tweaking or maybe I should just try on which video what plugin works best

Take a look at my last Retinex example in post#21.

[Rockel83]

Maybe it just needs some tweaking or maybe I should just try on which video what plugin works best

Take a look at my last Retinex example in post#21.

Oh wow! This is quite some improvement! Thanks!

I'm really amazed what this Avisynth is capable of. First some people helped me here removing some timestamp. Something I never thought which was possible without cropping. And now this!

[Rockel83]

You could also use a luma/limit mask to apply retinex to everything but the sky.
Just wanted to point out that retinex might offer a different view.

Good idea applying a mask.
Here a sample. Not perfect, but perhaps something to begin with.

lmask=clip.mt_binarize(110).Blur(1.5).Blur(1.5).Blur(1.5).Blur(1.5).invert() # build a mask based on luma
Overlay(clip, converttoYV24().MSRCP(fulls=false,fulld=false,lower_thr=0.0,upper_thr=0.0001,chroma_protect=3), mask=lmask)
smoothlevels(input_low=16,input_high=255,output_low=16,output_high=235) #adjust luma to TV range 16 ..... 235

Edit: same but with stronger mask blurring:

lmask=clip.mt_binarize(110).converttoYV12().GaussianBlur(varY=5000).invert() # build a mask based on luma
Overlay(clip, converttoYV24().MSRCP(fulls=false,fulld=false,lower_thr=0.0,upper_thr=0.0001,chroma_protect=3), mask=lmask)
smoothlevels(input_low=16,input_high=255,output_low=16,output_high=235)

I wanted to try this one too, but I guess I'm missing something again here?


[Attachment 65962 - Click to enlarge]

I'm using the masktools2_v2.2.30 plugin btw

[jagabo]

When you don't specify a stream by name the default name, "last", is used. Both for inputs and outputs. Your script doesn't name the output of LWlibavVideoSource() so it is automatically named last. When you call mt_binarize you are specifying a stream called clip -- which hasn't been created. Use last instead. You can do so explicitly with:

lmask=last.mt_binarize(...

or implicitely with:

lmask=mt_binarize(...

It's the same for the next line with Overlay(clip, ...:

Overlay(last, ...

In other words, Sharc explicitly named the input stream with something like:

clip = LWlibavVideoSource('filename.ext")

And later referred to that stream by its name, clip.

[Sharc]

In other words, Sharc explicitly named the input stream with something like:

clip = LWlibavVideoSource('filename.ext")

And later referred to that stream by its name, clip.

Yes.
The source which I used was the OP's re-encoded .mp4 sample from post#1 rather than his .avi. I had some doubts regarding the color matrix (601 vs 709) though.
Anyway, for what it's worth here the full script:

DGSource("D:\Temp\Rockel83\VID_20220219_135613910.dgi") #or use a source filter of your choice

converttoYV16(interlaced=false,matrix="Rec709") #conversion depends on the source and on requirement of the subsequent filters
assumeFPS(25)
spline36resize(1920/2,1080/2) #for later SBS display on TV with 1920 pixels total width, and faster processing
clip=last  #naming the input stream

#retinex with luma mask:
lmask=clip.mt_binarize(110).converttoYV12().GaussianBlur(varY=5000.0).invert() # build a mask based on luma
#return lmask #for debugging the mask only
Overlay(clip, converttoYV24().MSRCP(fulls=false,fulld=false,lower_thr=0.0,upper_thr=0.0001,chroma_protect=3), mask=lmask)
smoothlevels(input_low=12,input_high=255,output_low=16,output_high=235)  #adjust the levels

#select the output:
StackHorizontal(clip.subtitle("  original"), last.subtitle("  retinex with mask")) #display before and after, side by side
#StackHorizontal(clip.histogram().subtitle("  original"), last.histogram().subtitle("  retinex with mask")) # same with waveform histogram for luma range checking

#for subsequent x264 encoding only:
ConvertBits(8)
ConvertToYV12()

The GaussianBlur is from the variableblur plugin.

[Rockel83]

Thanks for pointing out and explaining! So when text is colored red I'm probably missing a plugin?

I've downloaded the Variableblur plugin here http://avisynth.nl/index.php/VariableBlur#External_Links (version 0.7/x64). And made the adjustments in the script.

But now I'm getting this error:


[Attachment 65967 - Click to enlarge]

edit:
Well, found this in the txt-file so far. Found the libfftw3f-3.dll file in the zip from website below and placed it in the SysWOW64 dir. But still getting the same error.

"The gaussian filter uses fftw version 3.3 available from http://www.fftw.org/install/windows.html - 32-bit version.
Specifically, libfftwf3-3.dll needs to be in the search path (C:\Windows\SysWOW64 64-bit OS or C:\windows\system32 32-bit OS)
and variableblur wont run or load without it."

edit 2:
Nevermind, got it working!
Guess the System32 folder did the trick...

[Sharc]

edit 2:
Nevermind, got it working!
Guess the System32 folder did the trick...

Great.
By the way you could simplify the lmask line somewhat by dropping the invert(), like

lmask=clip.mt_binarize(threshold=110,upper=true).converttoYV12().GaussianBlur(varY=5000.0)

[Rockel83]

Ah I see! Because "invert contains no value here?

How about "converttoyv12" then?

Results of this script look verry cool in some really dark places, verry helpfull. Especially with the new sky (mask?)

I only encounter some extreme inversions in the upper right part of the sky around 0:09 while applying this script in this video? Is there a way to soften this a bit?
The results are verry much better then the orriginal video ofcourse.

Edit:
I see that playing around with the "mt_binarize" value is helpfull here.

[Sharc]

You may reduce the varY value of the Gaussian blur to say 500 and/or play with the threshold (110) of mt_binarize, or play with the settings of MSRCP and so on ....
You could also add other post processing filters. Have fun!