[Avisynth+] How to display a luminance map

[butterw]

I'm trying to display a luminance/brightness map of a video clip in Avisynth+.

1. How can I posterize Y in n evenly distributed or custom levels (ex n=5) ?
Input is YV12. Options for output would be grayscale or keep UV.
something like posterize(vid_yv12, n=5) or posterize(vid_yv12, levels=[25, 50, 100, 200.0])

2. GrayScale(vid_rgb, matrix="Rec709") provides a grayscale output from a rgb input
How can this operation be performed with a custom LumaCoef vector (or how to calculate a custom dot product in Avisynth) ?
LumaCoef=float4(0.18, 0.41, 0.41, 0); //Agfa 200X
dot(col_rgba, LumaCoef);

[jagabo]

You can use Tweak twice:

function Bandify(clip v, int bands)
{
    v.Tweak(cont=float(bands-1)/256.0, sat=0, coring=false)
    Tweak(cont=256.0/float(bands-1), coring=false)
}

Call with:

Bandify(5)

Note that assumes full range YUV.

[hello_hello]

Is the last equation pixel shader related? It looks a little familiar, but I'll confess don't understand it

This uses MaskTools2 and requires a patience for reverse polish notation I don't have. There's possibly a simpler expression that'd achieve the same thing, but this at least works. Hopefully "N" doesn't need to be a variable.... this'd be the equivalent of N=8 though.

In a more familiar form, at least to me, it's doing this, where X is the pixel value.

X = (X < 32) ? 16 : (X < 64) ? 48 : (X < 96) ? and so on...

mt_lut(last, "x 32 < 16 x 64 < 48 x 96 < 80 x 128 < 112 x 160 < 144 x 192 < 176 x 224 < 208 240 ? ? ? ? ? ? ?", Y=3, U=-128, V=-128)

Greyscale
Y=3, U=-128, V=-128


[Attachment 56669 - Click to enlarge]

Process U & V
Y=3, U=3, V=3


[Attachment 56670 - Click to enlarge]

Copy U & V
Y=3, U=2, V=2


[Attachment 56671 - Click to enlarge]

[hello_hello]

You can use Tweak twice:.

&%$ me!
That's fairly inventive. And so simple.

[jagabo]

The problem with mt_lut() is that it is a pre-computed lookup. So ti doesn't accept runtime variables. But you can use it for a fixed number of bands.

mt_lut("x 5.0 1.0 - 256.0 / * round 256.0 5.0 1.0 - / *")

Change both 5.0 to whatever number of bands you want.

Oh, if you want Bandify with an optional greyscale:

function Bandify(clip v, int "bands", bool "greyscale")
{
    greyscale = default(greyscale, true) # if not specified default to true

    greyscale ? v.greyscale() : v
    Tweak(cont=float(bands-1)/256.0, coring=false)
    Tweak(cont=256.0/float(bands-1), coring=false)
}

[butterw]

Is the last equation pixel shader related? It looks a little familiar, but I'll confess don't understand it

Yes it's an optimized vector math operation commonly used on gpu/pixel shaders. It may not be used in Avisynth ?
It's used for instance to convert RGB to Y (grayscale):
dot(color_rgb, LumaCoef): Y = R*LumaCoef.r + G*LumaCoef.g + B*LumaCoef.b
For Rec709, LumaCoef would be the top line of the rgb2yuv 3x3 colormatrix.

My question was really about per component math and it seems like Masktools2 is the place where to look (if at all).

[jagabo]

Is the last equation pixel shader related? It looks a little familiar, but I'll confess don't understand it

Yes it's an optimized vector math operation commonly used on gpu/pixel shaders. It may not be used in Avisynth ?
It's used for instance to convert RGB to Y (grayscale):
dot(color_rgb, LumaCoef): Y = R*LumaCoef.r + G*LumaCoef.g + B*LumaCoef.b
For Rec709, LumaCoef would be the top line of the rgb2yuv 3x3 colormatrix.

My question was really about per component math and it seems like Masktools2 is the place where to look (if at all).

You can use mt_lutxyz() after splitting the RGB into three images with ShowRed(), ShowGreen(), ShowBlue(). Given an RGB image:

# assuming RGB video

R = ShowRed().ConvertToYV12(matrix="pc.601") # red to Y
G = ShowGreen().ConvertToYV12(matrix="pc.601") # green to Y
B = ShowBlue().ConvertToYV12(matrix="pc.601") # blue to Y

mt_lutxyz(R, G, B, "x 0.299 * y 0.587 * z 0.114 * + + 219 * 255 / 16 +") # 8 bit limited range rec.601 Y value

[hello_hello]

Yes it's an optimized vector math operation commonly used on gpu/pixel shaders. It may not be used in Avisynth ?
It's used for instance to convert RGB to Y (grayscale):
dot(color_rgb, LumaCoef): Y = R*LumaCoef.r + G*LumaCoef.g + B*LumaCoef.b
For Rec709, LumaCoef would be the top line of the rgb2yuv 3x3 colormatrix.

I think I'm understanding now.

The problem with mt_lut() is that it is a pre-computed lookup. So ti doesn't accept runtime variables. But you can use it for a fixed number of bands.

I guess if for "Bands", a huge range from which to choose isn't necessary.....

mt_Bandify(5, -128)

function mt_Bandify(clip Vid, int "Bands", int "UV")
{

Bands = default(Bands, 5)
UV = default(UV, -128)

return \
(Bands <= 2)  ? mt_lut(Vid, "x 1.0 256.0 / * round 256.0 1.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 3)  ? mt_lut(Vid, "x 2.0 256.0 / * round 256.0 2.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 4)  ? mt_lut(Vid, "x 3.0 256.0 / * round 256.0 3.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 5)  ? mt_lut(Vid, "x 4.0 256.0 / * round 256.0 4.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 6)  ? mt_lut(Vid, "x 5.0 256.0 / * round 256.0 5.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 7)  ? mt_lut(Vid, "x 6.0 256.0 / * round 256.0 6.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 8)  ? mt_lut(Vid, "x 7.0 256.0 / * round 256.0 7.0 / *", Y=3, U=UV, V=UV) : \
(Bands == 9)  ? mt_lut(Vid, "x 8.0 256.0 / * round 256.0 8.0 / *", Y=3, U=UV, V=UV) : \
(Bands >= 10) ? mt_lut(Vid, "x 9.0 256.0 / * round 256.0 9.0 / *", Y=3, U=UV, V=UV) : nop()

}

[butterw]

The problem with mt_lut() is that it is a pre-computed lookup. So ti doesn't accept runtime variables. But you can use it for a fixed number of bands.

Good to know. But n would most likely be a fixed parameter in the script rather than a variable.

mt_lut("x 5.0 1.0 - 256.0 / * round 256.0 5.0 1.0 - / *")

Is it possible to to avoid the reverse polish notation using mt_polish as a translator ?

[jagabo]

mt_lut("x 5.0 1.0 - 256.0 / * round 256.0 5.0 1.0 - / *")

Is it possible to to avoid the reverse polish notation using mt_polish as a translator ?

Yes:

mt_lut(mt_polish("round(x * (5.0 - 1.0) / 256.0) * (256.0 / (5.0 - 1.0))"))

Or precalculating all the constants:

mt_lut(mt_polish("round(x * 0.015625) * 64.0"))

[hello_hello]

I've yelled, cried, tried bargaining with the devil, but I just can't get mt_polish to work. All I get is a black output.

Even something really simple that works this way:
mt_lut("x 20 +")
Doesn't work like this:
mt_lut(mt_polish("x + 2.0"))
Or this:
mt_lut(mt_polish("x + 2.0"), y=3, u=-128, v=-128)
Is MaskTools2 currently broken, or is it me?

90 seconds after posting, I found the reason, even though I'd already scanned the wiki several times.

http://avisynth.nl/index.php/Masktools2#MaskTools2_v2.2.x
At the moment "mt_polish" functionality is not available on XP builds.

It'd be nice if something like that was highlighted in red to make it hard to miss, or even by just making the text bold. Sigh...

[butterw]

MaskTools2.dll: Avisynth+ high bit depth support (incl. planar RGB, color spaces with alpha plane are supported from v2.2.7
- In the RGB case I assume mt_lut(Y, U, V=) would mean R, G, B
- RGB32 isn't planar RGB.

[butterw]

Does Avisynth know what the range of a video is (based on tag: Limited or untagged vs Full) ?
There doesn't appear to be anything related in Clip_properties or LSMASHVideoSource.

I'm used to mpc-hc RGB pixel shaders were this isn't an issue because the range is always Full, but I can see how this could be troublesome for processing in avisynth or mpv yuv shaders.

Video File yuv420 (range tag: Limited or Full) >> YUV processing >> yuv2rgb conversion in video player (with range expansion unless Full range tag) >> RGB pixel Shaders >> Screen: Full Range RGB

[jagabo]

I wasn't able to figure out how to use mt_lut with RGB data. You need to use ConvertToPlanarRGB() or ConvertToPlanarRGBA() first but beyond that all the manipulations appear to work only within individual channels.

The RGB conversion to Y in post #7 in algebraic notation:

mt_lutxyz(R, G, B, mt_polish("(x*0.299 + y*0.587 + z*0.114) * 219.0 / 255.0 + 16.0")) # 8 bit limited range rec.601 Y value

YCbCr equation:
https://software.intel.com/content/www/us/en/develop/documentation/ipp-dev-reference/t...ml?language=en

[butterw]

The problem with mt_lut() is that it is a pre-computed lookup. So ti doesn't accept runtime variables. But you can use it for a fixed number of bands.

mt_lut("x 5.0 1.0 - 256.0 / * round 256.0 5.0 1.0 - / *")

You can't use a variable name in the mt_lut string other than x (it just gets evaluated to 0). It is possible to pre-write the value in the str though:

lut_str = "x " +string(bands)+" 1.0 - 256.0 / * round 256.0 "+string(bands)+" 1.0 - / *"
vid.mt_lut(lut_str)

To clarify what a luminance or brightness map is:
- luminance means Y (ex: rec709)
- brightness could be any scalar expression. ex: Y*Y, fn(R, G, B)

The purpose of the map is either as a Luma visualization tool or to to apply different effects according to the brightness band. It works decently as a realtime Cartoon Effect. Value for bands should be adjusted between 4 and 10 depending on the scene. Above this, ugly posterization artefacts are visible.

[jagabo]

The problem with mt_lut() is that it is a pre-computed lookup. So ti doesn't accept runtime variables. But you can use it for a fixed number of bands.

mt_lut("x 5.0 1.0 - 256.0 / * round 256.0 5.0 1.0 - / *")

You can't use a variable name in the mt_lut string other than x (it just gets evaluated to 0). It is possible to pre-write the value in the str though:

lut_str = "x " +string(bands)+" 1.0 - 256.0 / * round 256.0 "+string(bands)+" 1.0 - / *"
vid.mt_lut(lut_str)

Thanks for that. So you can create a function like:

function mt_bandify(clip v, int "bands")
{
    lut_str = "x " +string(bands)+" 1.0 - 256.0 / * round 256.0 "+string(bands)+" 1.0 - / *"
   mt_lut(v, lut_str)
}

and call it with:

mt_bandify(5)

or to verify two different values work:

StackHorizontal(mt_bandify(5), mt_bandify(25))
TurnRight().Histogram().TurnLeft()