Avisynth x64 VHS oversharpnening reduction script for anyone who needs it.

[useraxe]

I don't know how useful this is to put here, but i modified the code (very messily, i know ) from https://forum.doom9.org/showthread.php?t=140623 (Thank you, Avisynth_challenged), for VHS haloing/oversharpening reduction, as my VCR doesn't have the ability to adjust sharpness and boosts it like crazy, so this may come handy for someone else as well.

This code is probably not well optimized, but should work pretty well to reduce the oversharpened appearence of a video without making it look like an oilpainting. Would be good if the dehaloing could be set stronger, but I don't know how to do it, although this already removes it pretty well I believe. In addition, the script also includes BWDIF deinterlacing; can remove dropouts a bit (can be left as is, I think, it shouldn't do damage); and does a little noise reduction that shouldn't destory detail, but needs to be changed or configured depending on the video and your preference. If possible, use NeatVideo and remove the ".HQDN3D().FFT3DFilter(sigma=5)".

If the halos aren't reduced for you, try adjusting the VSLGhost's "shift", and the "mask.Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)"

Some samples:

[Attachment 80874 - Click to enlarge]

[Attachment 80875 - Click to enlarge]

The code: (outdated, see below)

source = AviSource("C:\..\file.avi").AssumeFieldBased().AssumeTFF().ConvertToYUY2().DePulse()
#source = LWLibavVideoSource("C:\..\file.mkv", cache=false, prefer_hw=2).AssumeFieldBased().AssumeTFF().ConvertToYUY2().DePulse()
#source = FFVideoSource("C:\..\file.avi").AssumeFieldBased().AssumeTFF().ConvertToYUY2().DePulse()
############

start=source.spatialsoften(1,255,127).converttoyv12.greyscale.Tweak(0.0, 1.0, 0, 1.0, true, false)
ghostA=source.spatialsoften(1,255,127).converttoyv12.VSLGhost(mode=3, shift=4, intensity=-128).greyscale.Tweak(0.0, 1.01, 0, 1.25, true, false)
diffA=Subtract(ghostA, start).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()

ghostB=source.spatialsoften(1,255,127).converttoyv12.VSLGhost(mode=3, shift=1, intensity=-128).greyscale.Tweak(0.0, 1.01, 0, 1.25, true, false)
diffB=Subtract(ghostB, start).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()


diff2 = Subtract(ghostA, ghostB).Invert()

# Assuming 'diff2' is the final grayscale clip for halo detection
# Using 'diff2' to create a mask for halo removal
mask = diff2.Levels(50, 0.5, 255, 0, 255) # Adjust levels to create a mask

# Shift the mask if necessary
mask = mask.Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)

# Remove haloing from the original video using the mask
halo_removed = Overlay(source, diff2, mask=mask, mode="subtract", opacity=0.5)
# Adjust the levels of the final output to counteract the washed-out effect
halo_removed2 = halo_removed.Tweak(bright=12)
halo_removed3 = halo_removed2.ConvertToYV16()
halo_rem4 = halo_removed3.Crop(12, 0, -2, -0).AddBorders(12, 0, 2, 0).Bwdif(field=3).Dehalo_Alpha(rx=2,ry=2,lowsens=50,highsens=100).HQDN3D().FFT3DFilter(sigma=5)
return halo_rem4

By the way I thought QTGMC was broken so I used BWDIF. I realized AssumeFieldBased() was messing with it, so remove it and use QTGMC if you want to. Don't forget to adjust the noise filtering.

[jagabo]

Put your script in a code block to prevent the additional spaces.

[useraxe]

Okay, thanks.

[jagabo]

A quick attempt from your original image with downscale, dehalo_alpha, upscale:


[Attachment 80941 - Click to enlarge]

Oversharpened, still some artifacts where the strong haloes were. Would have worked better with the original SD video.

[useraxe]

Previous script is now outdated. Try this. It now does 4 passes making it more effective, and the colors shouldn't shift anymore. I'm actually surprised of how well it works. Maybe some very dark detail will be lost, but it looks fantastic now and doesn't have that oil painting look as Dehalo Alpha does (this script includes it, but it doesn't make much difference). I think this just looks like if you adjust the sharpness slider on older Panasonics, and I hope somebody can make this into an adjustable plugin!
Also chatgpt was used, as i don't have a lot of experience on avisynth, so I hope it's not overly unoptimized.

source = LWLibavVideoSource("C:\...\file.mkv", cache=false, prefer_hw=2).AssumeTFF().ConvertToYUY2(interlaced=true).SeparateFields()#.DePulse()
X = LWLibavVideoSource("C:\Users\Admin\Downloads\2.mkv", cache=false, prefer_hw=2).AssumeTFF().ConvertToYUY2(interlaced=true)#.DePulse()
#X = AviSource("C:\Users\Admin\Downloads\2.mkv").AssumeTFF().ConvertToYUY2(interlaced=true)#.DePulse()
#source = AviSource("C:\...\file.avi").AssumeFieldBased().AssumeTFF().ConvertToYUY2(interlaced=true).SeparateFields()#.DePulse()

# Preprocess source for reuse
preprocessed = source.spatialsoften(1, 255, 127).converttoyv12.greyscale.Tweak(0.0, 1.0, 0, 1.0, true, false)
ghostA = preprocessed.VSLGhost(mode=3, shift=4, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
ghostB = preprocessed.VSLGhost(mode=3, shift=1, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
diffA = Subtract(ghostA, preprocessed).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB = Subtract(ghostB, preprocessed).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diff2 = Subtract(ghostA, ghostB).Invert()

# Repeat preprocessing and halo removal steps 4 times

# First pass
mask1 = diff2.Levels(50, 0.5, 255, 0, 255).Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)
halo_removed1 = Overlay(source, diff2, mask=mask1, mode="subtract", opacity=0.5)
halo_rem1 = halo_removed1.Tweak(bright=12)

# Second pass
preprocessed2 = halo_rem1.spatialsoften(1, 255, 127).converttoyv12.greyscale.Tweak(0.0, 1.0, 0, 1.0, true, false)
ghostA2 = preprocessed2.VSLGhost(mode=3, shift=4, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
ghostB2 = preprocessed2.VSLGhost(mode=3, shift=1, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
diffA2 = Subtract(ghostA2, preprocessed2).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB2 = Subtract(ghostB2, preprocessed2).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diff22 = Subtract(ghostA2, ghostB2).Invert()

mask2 = diff22.Levels(50, 0.5, 255, 0, 255).Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)
halo_removed2 = Overlay(halo_rem1, diff22, mask=mask2, mode="subtract", opacity=0.5)
halo_rem2 = halo_removed2.Tweak(bright=12)

# Third pass
preprocessed3 = halo_rem2.spatialsoften(1, 255, 127).converttoyv12.greyscale.Tweak(0.0, 1.0, 0, 1.0, true, false)
ghostA3 = preprocessed3.VSLGhost(mode=3, shift=4, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
ghostB3 = preprocessed3.VSLGhost(mode=3, shift=1, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
diffA3 = Subtract(ghostA3, preprocessed3).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB3 = Subtract(ghostB3, preprocessed3).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diff23 = Subtract(ghostA3, ghostB3).Invert()

mask3 = diff23.Levels(50, 0.5, 255, 0, 255).Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)
halo_removed3 = Overlay(halo_rem2, diff23, mask=mask3, mode="subtract", opacity=0.5)
halo_rem3 = halo_removed3.Tweak(bright=12)

# Fourth pass
preprocessed4 = halo_rem3.spatialsoften(1, 255, 127).converttoyv12.greyscale.Tweak(0.0, 1.0, 0, 1.0, true, false)
ghostA4 = preprocessed4.VSLGhost(mode=3, shift=4, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
ghostB4 = preprocessed4.VSLGhost(mode=3, shift=1, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
diffA4 = Subtract(ghostA4, preprocessed4).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB4 = Subtract(ghostB4, preprocessed4).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diff24 = Subtract(ghostA4, ghostB4).Invert()

mask4 = diff24.Levels(50, 0.5, 255, 0, 255).Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)
halo_removed4 = Overlay(halo_rem3, diff24, mask=mask4, mode="subtract", opacity=0.5)
halo_rem4 = halo_removed4.Tweak(bright=12).ConvertToYV16()

# Weave back + final processing
afinal_output = halo_rem4.Weave().Dehalo_Alpha(rx=2, ry=2, lowsens=50, highsens=100).Blur(1,0).ConvertToYUY2()#.Crop(12, 4, -12, -18)#BicubicResize
final_output = MergeChroma(afinal_output, X).Crop(8, 0, -8, -0).AddBorders(8, 0, 8, 0)#.Prefetch(12)
return final_output

[Selur]

From the looks of it, you process the first source, and then take the Chroma from the second source.
How does the first source:

 LWLibavVideoSource("C:\...\file.mkv", cache=false, prefer_hw=2)

relate to the second source:

LWLibavVideoSource("C:\Users\Admin\Downloads\2.mkv", cache=false, prefer_hw=2)

?

Also, there are tons of code lines that don't contribute to the output at all.

...
diffA = Subtract(ghostA, preprocessed).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB = Subtract(ghostB, preprocessed).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
...
diffA2 = Subtract(ghostA2, preprocessed2).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB2 = Subtract(ghostB2, preprocessed2).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
...
diffA3 = Subtract(ghostA3, preprocessed3).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB3 = Subtract(ghostB3, preprocessed3).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
...
diffA4 = Subtract(ghostA4, preprocessed4).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()
diffB4 = Subtract(ghostB4, preprocessed4).Levels(127, 1, 129, 0, 255).invert().RgTools_RemoveGrain()

Those lines are calculated, but never used.


Cu Selur

[useraxe]

The source was used twice to have MergeChroma work, as I tried using SeparateFields() and Weave() (don't know why). But Separate and Weave() aren't necessary I think.

I really don't understand most of the code. Interesting it works at all.

[useraxe]

Ok, remembered I used X for QTGMC, as it didn't work for some reason without SeparateFields() and weave, so X source was used. But this is meant if you wanna keep interlaced. Also it runs 3x faster.

# Uncomment the desired source
# source = LWLibavVideoSource("C:\...\file.mkv", cache=false, prefer_hw=2).AssumeTFF().ConvertToYUY2(interlaced=true)#.DePulse()
source = AviSource("C:\...\file.avi").AssumeTFF().ConvertToYUY2(interlaced=true)#.DePulse()

# Preprocess source for reuse
function Preprocess(clip src) {
    src.spatialsoften(1, 255, 127).ConvertToYV12().Greyscale().Tweak(0.0, 1.0, 0, 1.0, true, false)
}

function ProcessPass(clip src, clip preprocessed) {
    ghostA = preprocessed.VSLGhost(mode=3, shift=4, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
    ghostB = preprocessed.VSLGhost(mode=3, shift=1, intensity=-128).Tweak(0.0, 1.01, 0, 1.25, true, false)
    diffA = Subtract(ghostA, preprocessed).Levels(127, 1, 129, 0, 255).Invert().RgTools_RemoveGrain()
    diffB = Subtract(ghostB, preprocessed).Levels(127, 1, 129, 0, 255).Invert().RgTools_RemoveGrain()
    diff2 = Subtract(ghostA, ghostB).Invert()

    mask = diff2.Levels(50, 0.5, 255, 0, 255).Crop(2, 0, -0, -0).AddBorders(0, 0, 2, 0)
    halo_removed = Overlay(src, diff2, mask=mask, mode="subtract", opacity=0.5)
    halo_removed.Tweak(bright=12)
}

# Apply preprocessing and halo removal passes
preprocessed = Preprocess(source)
halo_rem1 = ProcessPass(source, preprocessed)
preprocessed2 = Preprocess(halo_rem1)
halo_rem2 = ProcessPass(halo_rem1, preprocessed2)
preprocessed3 = Preprocess(halo_rem2)
halo_rem3 = ProcessPass(halo_rem2, preprocessed3)
preprocessed4 = Preprocess(halo_rem3)
halo_rem4 = ProcessPass(halo_rem3, preprocessed4).ConvertToYV16()

# Weave back + final processing
afinal_output = halo_rem4.Dehalo_Alpha(rx=2, ry=2, lowsens=50, highsens=100).Blur(1, 0).ConvertToYUY2()
final_output = MergeChroma(afinal_output, source).Crop(4, 0, -8, -0).AddBorders(4, 0, 8, 0).Prefetch(12)#.BicubicResize(0, 0)
return final_output