floating chroma shift issue?

[cinephil]

The horizontal line is the midpoint indicator for histogram() - it's normal . It might have displayed slightly differently in older avs versions. You can test it yourself on a black synthetic video, it's still there

I need to stop posting questions that late at night

But I still struggle to get similar looking picture with new ContrastMask And I have some strange artifacts.

new

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ffms2.dll")
FFVideoSource("E:\SAMPLE\video\03.avi")

ConvertBits(16) 

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\aWarpsharpMT.dll")
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\nnedi3.dll")
SeparateFields()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0) #gain_y=-10
Tweak(bright=20, coring=false) #coring=false
Tweak(Hue=-16)

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\vsTCanny.dll")
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\masktools2.dll")
Import("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ContrastMask2.avsi")
SeparateFields() 
ContrastMask(enhance=1.8)
Weave()

Tweak(cont=0.9)

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\fft3dfilter.dll")
FFT3DFilter(bt=-1, sharpen=0.3, interlaced=true)

ConvertBits(bits=8, dither=1)
ConvertToYV12(matrix="rec601", interlaced=true)

old

[Attachment 82684 - Click to enlarge]
new

[Attachment 82685 - Click to enlarge]

And for some reason - today, banding in the script with old ContrastMask not look that bad So maybe I stick with it...

[Attachment 82686 - Click to enlarge]

old

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ffms2.dll")
FFVideoSource("E:\SAMPLE\video\03.avi")

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\aWarpsharpMT.dll")
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\nnedi3.dll")
SeparateFields()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

ConvertToYV12(matrix="rec601", interlaced=true)
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\variableblur.dll")
LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\masktools2.dll")
Import("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\ContrastMask.avsi")
SeparateFields() 
even = SelectEven(last).ContrastMask(enhance=3.0)
odd = SelectOdd(last).ContrastMask(enhance=3.0)
Interleave(even, odd)
Weave()

ConvertBits(16) 

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0) #gain_y=-10
Tweak(bright=3, coring=false) #3 for ContrastMask(enhance=3.0)
Tweak(cont=0.9)
Tweak(Hue=-16)

LoadPlugin("M:\edit\MeGUI-6666.2308_x64\tools\avs\plugins64\fft3dfilter.dll")
FFT3DFilter(bt=-1, sharpen=0.3, interlaced=true)

ConvertBits(bits=8, dither=1)

[Selur]

Can't make heads or tail out of your script. (doesn't make any sense to me; too unsorted and unreadable)

Since the new version I posted can't use GaussianBlur from VariableBlur it uses vsTCanny, so there are bound to be differences.
How does it look if you replace:

    # Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=gblur)

with

    # Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=gblur,mode=-1)

or

    # Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=gblur, sigmaU=gblur, sigmaV=gblur, mode=-1)

?

mode=-1 should make it the outputs more similar.

Cu Selur

[cinephil]

Since the new version I posted can't use GaussianBlur from VariableBlur it uses vsTCanny, so there are bound to be differences.
How does it look if you replace:

    # Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=gblur)

with

    # Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=gblur,mode=-1)

or

    # Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=gblur, sigmaU=gblur, sigmaV=gblur, mode=-1)

?

This way it looks much closer to the old ContrastMask.
new+

[Attachment 82689 - Click to enlarge]

[Selur]

Happy that worked, there might be other parameter to get it even closer to the old version, but I leave that to motivated others.

[cinephil]

Happy that worked, there might be other parameter to get it even closer to the old version, but I leave that to motivated others.

If assume that old ContrastMask used all default values for VariableBlur/GaussianBlur,

GaussianBlur (clip, float "varY", float "varC", int "border", bool "integrate", int "Y", int "U", int "V", int "gfunc", int "gfuncc", int "pcr", int "pcrc", int "nthreads")

then what is "50.0" in this line meaning?

v2=v2.gaussianblur(50.0,50.0+gblur)

[Selur]

v2=v2.gaussianblur(50.0,50.0+gblur)

I overlooked that!
varY=50
varC=50+gblur
=> vsTCanny should probably use:

v2 = vsTCanny (v2, sigmaY=50, sigmaU=50.0+gblur, mode=-1)

(not totally sure if the 50 needs to be adjusted for vsTCanny)

[cinephil]

=> vsTCanny should probably use:

v2 = vsTCanny (v2, sigmaY=50, sigmaU=50.0+gblur, mode=-1)

(not totally sure if the 50 needs to be adjusted for vsTCanny)

This giving error:

[Attachment 82707 - Click to enlarge]

[Selur]

No clue, my guess you would probably have to test a bit to figure out the mapping between GaussianBlur and TCannys GaussianBlur.

[Selur]

try:

v2 = vsTCanny (v2, sigmaY=5, sigmaU=5+gblur, mode=-1)

[cinephil]

try:

v2 = vsTCanny (v2, sigmaY=5, sigmaU=5+gblur, mode=-1)

new++

[Attachment 82709 - Click to enlarge]

It's lost even more details in dark areas. I will try later to compare GaussianBlur vs TCannys GaussianBlur on some kind test pattern. Maybe find out something useful.

[Selur]

Fingers crossed.
I like:

# Apply Gaussian blur
    v2 = vsTCanny (v2, sigmaY=50+gblur, mode=-1)

the most, so far.

[cinephil]

Apparently there is something else missing...
I tried this way, but not get close enough to the old one.

v2 = v2.vsTCanny(sigmaY=7.1, sigmaU=7.1+gblur, mode=-1)

I'm not exactly understand what this mean in vsTCanny:

sigmaU = sigmaY / horizontal_chroma_subsampling_factor

[Selur]

sigmaU = sigmaY / horizontal_chroma_subsampling_factor

Since this is about the default values, it means that if sigmaU is not specified explicitly, it is set to sigmaY divided by the horizontal chroma subsampling factor.
For "4:4:4"-content, it's divided by 1.
For "4:2:2"-content, it's divided by 2.
For "4:2:0"-content, it's divided by 2.
For "4:1:1"-content, it's divided by 4.
see: https://en.wikipedia.org/wiki/Chroma_subsampling


Cu Selur

[cinephil]

Seems like chroma treated differently in vsTCanny than in VariableBlur/GaussianBlur. I can't match color test pattern using vsTCanny if chroma parameters involved. Or just didn't find the right ones.

[Selur]

Is the original Avisynth version you are using identical to:

function ContrastMask(clip v, float "gblur", float "enhance")
{
  enhance = default (enhance, 10.0)
  tvcolor = default (tvcolor, true)
  gblur = default (gblur, 20.0)
  enhance = (enhance>=0.0 && enhance<=10.0) ? float(enhance*0.1) : 1.0
  v2=v.Tweak(sat=0)
  v2=v2.invert()
  v2=v2.gaussianblur(50.0,50.0+gblur)
  photoshop_overlay=mt_lutxy(v,v2,"x 127.5 > y 255 x - 127.5 / * x 255 x - - + y x 127.5 / * ? ")
  merged=overlay(v,photoshop_overlay,opacity=enhance)
  return merged
}

? I suspect it is not. (especially since this script should not work, since 'tvcolor' is never defined; so at least a '#' should be before the tvcolor line)

Is

v2 = vsTCanny(v2, sigmaY=50.0, sigma_vY=50.0 + gblur, sigmaU=0.0, sigma_vU=0.0, sigmaV=0.0, sigma_vV=0.0, mode=-1)

more similar to the original ContrastMask for you? (this should keep the chroma planes untouched)

[cinephil]

at least a '#' should be before the tvcolor line

Yes, it's with '#' before the tvcolor line.

I found mistake in my script - I applied Tweak brightness before ContrastMask

So all vsTCanny variations looks much closer to old ContrastMask. Except highlights looks little bit brighter.

Thanks, Selur!

[Selur]

About the highlights: a. try lowering enhance a bit or decreasing the 50 in the vsTCanny call

[cinephil]

About the highlights: a. try lowering enhance a bit or decreasing the 50 in the vsTCanny call

Well, that affect not only the highlights...

But maybe it's just me being stuck to this old ContrastMask look, and the new one looks better?
old

[Attachment 82730 - Click to enlarge]

new with

 # Apply Gaussian blur
    v2 = vsTCanny(v2, sigmaY=gblur, mode=-1)

[Attachment 82731 - Click to enlarge]

no ContrastMask

[Attachment 82732 - Click to enlarge]

old

SeparateFields()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

ConvertToYV12(matrix="rec601", interlaced=true)

SeparateFields() 
even = SelectEven(last).ContrastMask(enhance=3.0)
odd = SelectOdd(last).ContrastMask(enhance=3.0)
Interleave(even, odd)
Weave()

ConvertBits(16) 

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0)
Tweak(bright=3, coring=false)
Tweak(cont=0.9)
Tweak(Hue=-16)

FFT3DFilter(bt=-1, sharpen=0.3, interlaced=true)

ConvertBits(bits=8, dither=1)

new

ConvertBits(16) 

SeparateFields()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

SeparateFields()
even = SelectEven(last).ContrastMask(enhance=3.0)
odd = SelectOdd(last).ContrastMask(enhance=3.0)
Interleave(even, odd)
Weave()

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0)
Tweak(bright=3, coring=false)
Tweak(cont=0.9)
Tweak(Hue=-16)

FFT3DFilter(bt=-1, sharpen=0.3, interlaced=true)

ConvertBits(bits=8, dither=1)
ConvertToYV12(matrix="rec601", interlaced=true)