ffmpeg Lift

[chris319]

Does anyone know of an ffmpeg video filter which acts as a true "lift" control, i.e. which raises the black levels while keeping the white levels clamped? I've looked in the docs and have not found anything which matches this description. Shotcut has something like this but I'm looking for something that can be used in an ffmpeg script.

I've used lutyuv and brightness but these do not clamp the whites. With lutyuv it is possible to clip the blacks without affecting the whites but as best I can tell this is a hard clip and does not change the black level.

Thank you.

[poisondeathray]

By white levels "clamped" did you mean "not affect" or "not move", sort of how resolve does it ? or did you mean clamp, as in "clamp" or "squish" to certain level?

For ffmpeg, if working in RGB you can use the curves filter. You can apply a photoshop .acv curve file. If you don't have photoshop, vdub's gradation curve can export a photoshop acv. Or you can "map" the channels directly using key point values for each R,G,B curve - thus you can place "hold" knots to "hold" the values over a certain range (e.g. the "whites") or do basically anything to any range.
https://ffmpeg.org/ffmpeg-filters.html#curves-1

The lutyuv filter should be able to do it in YUV, but you would need to use some expressions and math functions for more control to do exactly what you want.
https://ffmpeg.org/ffmpeg-utils.html#Expression-Evaluation

[chris319]

By white levels "clamped" did you mean "not affect" or "not move", sort of how resolve does it ?

Yes.

For ffmpeg, if working in RGB

No, YUV it is. The conversions between RGB and YUV kind of killed my efforts at r 103 compliance a while back. r103 is written for RGB but the conversion to YUV throws everything off kilter.

I'll look further into lutyuv, thanks.

[poisondeathray]

They really need a levels filter in ffmpeg, because it's one of the most basic, almost every software has it. "output black" is basically "lift" . But lift and levels are basically linear in operation ; curves can do everything they can, and much more

If using vapoursynth , you can use YUV or RGB curves, or avisynth has smoothcurves, and both have levels filter

[chris319]

This might do what I want. It is pseudocode; I haven't run it on ffmpeg yet:

if y < 16; * (16 / y)

so if y = 4

y = 4 * (16 / 4)

= 16

[richardpl]

Can be done with ffmpeg already, just use curves filter and cast YUV input to RGB and back with mergeplanes filter.

[chris319]

This works after a fashion but is still a hard clip.

lutyuv=y='if(lt(val,19),21,val)'

Or I can use:

lutyuv=y='clip(val,30,230)'

I'm not sure if one is preferable to the other.

Due to the vagaries of ffmpeg I have to fuss with those numbers to get the actual clip levels I want. You can't just specify, say, 16 and have it clip at 16.

[richardpl]

Also you can use lut1d filter if you create lut manually in text file.

[poisondeathray]

Can be done with ffmpeg already, just use curves filter and cast YUV input to RGB and back with mergeplanes filter.

This works, but it seems like a convoluted way to get the result of a simple levels filter . I took a full range YUV 4:2:0 gradient to test

ffmpeg -i input.mp4 -filter_complex "[0:v]extractplanes=y+u+v[y][u][v];[y]mergeplanes=0x000000:gbrp[gbr];[gbr]curves=g='0/0.0625 1/1'[gbrf];[gbrf][u][v]mergeplanes=0x001020:yuv420p[out]" -map [out] -c:v libx264 -crf 18 -an output.mp4 -y

FFmpeg's curve filter is normalized 0-1 . In this example, the "hold" knot is at x=1, y=1

16/256 code values = 0.0625 , so on a [0,1] interval scale, x=0, y=0 is remapped to x=0, y=0.0625

This behaves correctly as the output_low or output_black for a levels filter, or a "lift" in resolve . You can place more control points if you wanted to

I think I'm messed something in the calculation for 0.0625, because it actually ends up "15" , not "16" . But whatever, it's just a proof of concept that it's possible. Really crappy way to do the manipulation IMO, when a simple levels filter would suffice

src

[Attachment 47721 - Click to enlarge]

ffmpeg x=0, y=0.0625 ; x=1, y=1

[Attachment 47722 - Click to enlarge]

ffmpeg x=0, y=0.5 ; x=1, y=1

[Attachment 47723 - Click to enlarge]

[chris319]

I'm trying to bring out some dark detail. If I use brightness it interacts with my gamma setting which I do not want, so I think it must be conditional and restricted to < 16, maybe something like:

lutyuv=y='if(lt(val,19),(val*1.1875)+1,val)'

I add 1 because 0 is reserved for sync.

[poisondeathray]

I'm trying to bring out some dark detail. If I use brightness it interacts with my gamma setting which I do not want, so I think it must be conditional and restricted to < 16, maybe something like:

lutyuv=y='if(lt(val,19),(val*1.1875)+1,val)'

I add 1 because 0 is reserved for sync.

If I apply this to a test gradient, there is a "notch" . You usually want a smoother curve or interpolation between points, otherwise you'll get quantization and banding artfacts there. Some filters apply dithering algorithms to smooth everything out

If you look at the general shape of math function curves, like log, ln, etc.. they can be used if you're going this route , but it takes a lot of testing and massaging. Unless you're a math saavy, it's going to take a lot of work.



"Bringing out dark detail" usually implies increasing contrast at some shadow range. Normally a curve would be used. Most cameras "squish" shadow detail (partly because of sensor latitude, partly because of compression codec, partly because of the transfer) . ie. A bunch of values are compressed in a narrow range (you can only make out a few dark "shades")

Levels , lift will not do that, because the contrast is actually decreased since the adjustment is linear. ie. You will "wash out" the image as you increase lift, because the slope of the curve is actually decreasing.

You usually want to anchor the darkest "black" at 16, but increase the shadow contrast - which means a curve at a point above that level. You can place "hold" knots at different levels (e.g. if you were happy with midtones , you can keep them there) .

But doing this with 8bit footage with consumer cameras will usually expose a lot of noise , junk and compression artifacts. There are limits to what you can do with 8bit footage. You might have to denoise the shadows a bit as you increase the contrast and exposure there. But even a small boost can often reveal shadow details that would otherwise be obscured or crushed

[chris319]

If I apply this to a test gradient, there is a "notch" .

Is this notch at or around 16? Keep in mind that in BT.709 anything below 16 is considered out of bounds. Applying this filter to real-world footage, the results look satisfactory. We're dealing with some very dark stuff.

The other option is to simply do a hard clip but that won't improve detail.

[poisondeathray]

If I apply this to a test gradient, there is a "notch" .

Is this notch at or around 16? Keep in mind that in BT.709 anything below 16 is considered out of bounds. Applying this filter to real-world footage, the results look satisfactory. We're dealing with some very dark stuff.

The other option is to simply do a hard clip but that won't improve detail.

Yes, just a bit above 16 , a little notch. Generally you'd want to smooth it out instead of sharp inflexion points or bumps, because those will cause artifacts

If it was really dark stuff, you'd usually want to brighten it up there a bit more; the notch is small, so tells me the magnitude of the effect you are applying is relatively small. (you might be limited by other factors, light noise)

Yes, clipping is usually not desired, unless there is a purposeful good reason (e.g. maybe some bad noise or something, that couldn't be addressed with other methods, and you want to "hide" it by clipping it)

[chris319]

If it was really dark stuff, you'd usually want to brighten it up there a bit more

Not really. In several decades of professional video work I've become averse to turning black into dark gray. Well, maybe a teensy bit.

[poisondeathray]

If it was really dark stuff, you'd usually want to brighten it up there a bit more

Not really. In several decades of professional video work I've become averse to turning black into dark gray. Well, maybe a teensy bit.

That's what you were doing with "lift" . I suggested that you NOT do that, because you'd "wash out" the image and turn black to grey

I suggested you to anchor black at 16 , and to increase the shadow contrast. Black will still be black, but you could reveal more details. So you will brighten up the shadows and black will still be black .
https://forum.videohelp.com/threads/391595-ffmpeg-Lift#post2538817

[chris319]

That's what you were doing with "lift" . I suggested that you NOT do that, because you'd "wash out" the image and turn black to grey

I wasn't using lift; I was asking about it. I don't know where you got the idea that I was using it. I looked at lift in Shotcut just for fun and it was easy to make the darks look artificially washed out, so it didn't make the cut.

I suggested you to anchor black at 16

The ffmpeg code I posted only works on values < 16. Take another look.

[poisondeathray]

I suggested you to anchor black at 16

The ffmpeg code I posted only works on values < 16. Take another look.

Then it's not working as you thought. Take another look

[chris319]

Then it's not working as you thought.

Look, you and I know that you never get out of ffmpeg what you tell it to give you. You tell it to clip at n and it clips at some other level. You end up pissing away hours and hours zeroing in on the value you want by doing pass after pass with different values in the code until it spits out the right value. Been there, done that. After a while it gets to be more trouble than it's worth.

The code I wrote is conditional upon y < 16 but of course ffmpeg gives me something else.

[poisondeathray]

Then it's not working as you thought.

Look, you and I know that you never get out of ffmpeg what you tell it to give you. You tell it to clip at n and it clips at some other level. You end up pissing away hours and hours zeroing in on the value you want by doing pass after pass with different values in the code until it spits out the right value. Been there, done that. After a while it gets to be more trouble than it's worth.

The code I wrote is conditional upon y < 16 but of course ffmpeg gives me something else.

Yes, often there are issues with ffmpeg.

But clipping works properly

[jagabo]

As was discussed in your other thread about lift in AviSynth, lift is defined as

Y' = Y * (1.0 - lift) + lift

where everything is in unit values. Using ffmpeg for full range (0-255) YUV:

set RANGE=255
set LIFT=19
ffmpeg -y -i %1 -vf lutyuv=y='clip((val*(%RANGE%-%LIFT%)/%RANGE%)+%LIFT%,0,255)' -c:v rawvideo -an "%~n1.avi"

I'm not sure if RANGE should be 255 or 256. I used Clip() here but with the range set from 0 to 255 it does nothing. Use whatever clipping values you need.

Before:

[Attachment 47726 - Click to enlarge]

After LIFT=19:

[Attachment 47727 - Click to enlarge]

[chris319]

I'm not sure if RANGE should be 255 or 256.

255 I think.

Thanks a million for this. Works great!

My camcorder was putting the darkest regions at 10 so I used your "lift" at 14 to bring them up to 17. Also brought the gamma down a little and resized from 1920 x 1280p to 1280 x 720p.

As you can see I was fighting a setting sun.

set RANGE=255
set LIFT=14
D:\Programs\ffmpeg\BroadcastVideo\ffmpeg -y -i "D:\Videos\Trains\December_6_2018_San_Jose\C0073.MP4"  -crf 1  -c:v libx264  -pix_fmt yuv420p  -s 1280x720  -r 59.94  -vf eq=gamma=0.83,lutyuv=y='clip((val*(%RANGE%-%LIFT%)/%RANGE%)+%LIFT%,0,230)' -c:v rawvideo -c:a copy "lifted.mp4"