Quick Dot Crawl /Antialiasing Avisynth Question

[Vaengence]

I have a recurring artifact in these DVD's which occurs on straight lines with high contrast borders - I have isolated a small fragment video that has an example of it here https://drive.google.com/file/d/1zLfx0f5AFpQu4HjlHXTLf5-9ZRfXyiPn/view?usp=sharing. I don't know if its the best example of it, and can do more searching for more egregious examples if needed, but I have a question on the best way to deal with it. Its also an unprocessed extract of the raw VOB file, I have done nothing with it but save in AviSynth to extract it.

If you look at the control panel in the background a couple seconds into it, you will see along the light panel borders a kind of rolling dot crawl.

So far, every dot crawl removal tool I have tried either does nothing to it, or has a very small effect but still very noticeable. I accidentally found out that using Antialiasing tools seem to have a far more dramatic effect but often cause issues elsewhere on settings to remove it.

The only thing I have seen remove entirely like magic was this code I stumbled across one day:
Spline36Resize(width/2,height).Spline36Resize(width,height)

But it fixes it by (unless I am mistaken) drastically removing detail.

I know these kind of artifacts cannot often be fixed without damaging other details, but as a non expert that usually just copy pastes others scripts till something works, I haven't been able to find a really good solution (if one exists).

Does anyone here have a solution that may greatly reduce this specific kind of artifact without "too" much damage to the rest of the video?

Thanks in advance.

[jagabo]

The only thing I have seen remove entirely like magic was this code I stumbled across one day:
Spline36Resize(width/2,height).Spline36Resize(width,height)

But it fixes it by (unless I am mistaken) drastically removing detail.

Yes, that is ok for fuzzy VHS but significantly blurs sharp DVDs. You can try a compromise: downscale a little, noise/buzz reduction, upscale, and sharpen. Something like:

Mpeg2Source("C:\Users\John\Desktop\Short Extract.d2v", CPU2="ooooxx", Info=3) 

TFM(d2v="C:\Users\John\Desktop\Short Extract.d2v") 
TDecimate() 
src = last

BilinearResize(480,height)
QTGMC(InputType=1) # or some other temporal noise reducer
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=src.width, fheight=src.height)
Sharpen(0.3, 0.)

Other things you can try are to build a mask of areas that have dot crawl and filter only those areas. Or only sharp horizontal edges.

[Selur]

Not using Avisynth, but Vapoursynth:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# source: 'C:\Users\Selur\Desktop\Short Extract.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Selur\Desktop\Short Extract.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/Short Extract.mkv", format="YUV420P8", cache=0, prefer_hw=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# DEBUG: vsTIVTC changed scanorder to: progressive
# cropping the video to 708x480
clip = core.std.CropRel(clip=clip, left=6, right=6, top=0, bottom=0)
from vsrealesrgan import RealESRGAN
# adjusting color space from YUV420P8 to RGBS for VsRealESRGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# resizing using RealESRGAN
clip = RealESRGAN(clip=clip, device_type="cuda", device_index=0) # 1416x960
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=708, h=540, kernel="lanczos", interlaced=False, interlacedd=False)
# adjusting output color from: RGB48 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()

It's just TIVTC + Crop + RealESRGAN (attached reencode), but there are tons of ways to deal with this.
You might want to read https://forum.videohelp.com/threads/399360-so-where-s-all-the-Topaz-Video-Enhance-AI-discussion later in the discussion there is someone converting StarTrek content to 4k,... In general some deblocking and for example MCDegrainSharp would also work nice when it's mainly getting rid of the block noise.

(this is just TIVTC + Crop + MCDeGrainSharp)

Cu Selur

[Vaengence]

@Selur

Just to clarify, the specific noise mentioned is that which appears in places like this:

https://i.imgur.com/5FGtMtq.png

I currently have a wide array of methods dealing with the videos directly, and some very intensive (hours and hours to process) methods that reduce or mostly eliminate this specific set of artifacts, but I have had difficulties in getting rid of it with the least amount of processing.

If I shorten that last example clip you uploaded even more, to this attached and look to the top right of the background characters head, you will see artifacts that I can best describe in laymans terms as "dancing lights" on the panels. These occur vertically, horizontally but critically, anywhere where there is hard difference in contrast between the edges (like the lights on the panels) and usually only when the sections are relatively small compared to the image as a whole.

When you blow them up to a TV size and watch the result, it is incredibly distracting. I have some methods that remove or reduce these, bit the only ones I have had so far that work do significant damage to the source, take a very long time and still remove a great deal of detail or barely touch them at all.

In essence I thought I would poke a question to more experience people with a specific example to see if anyone had a better solution that might work. Note that I really have no idea how to start with Vapoursynth - took a long time to work out how to use Avisynth and I use that with AVSPmod so I can't make a great deal of progress on the example script, though thanks.

@jagabo

This script (avoiding QTGMC for the moment):

    BilinearResize(480,height)
    nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=src.width, fheight=src.height)
    Sharpen(0.3, 0.)

Does seem to greatly reduce those artifacts and so far, seem to damage the image very little, at least enough for me to struggle to tell the difference so far. I have also not tried this process specifically before, so I am definitely going to play around with it a bit and see how that turns out, thanks. Note I would definitely love to restrict some of the sections with masks where possible, I am still learning however and many of these scripts go over my head as to what they do - I just look at output and compare if it made it worse or better.

This specific set of artifacts, however, are unique in location and duration in each episode - and not every episode has them - so individual masks sounds like it would take me a while on each episode.

[Selur]

Ah, okay. You could try mClean (https://forum.doom9.org/showthread.php?t=174804), it should at least lessen the effect a lot.
"mClean(deband=0,depth=1)" in Avisynth gave me:

You probably can get better results by tweaking the settings.

Cu Selur

[Vaengence]

Thanks for that ^^

mClean(deband=0,depth=1)

This immediately had a significant lessening effect on those artifacts - I will play around with that and see how I go ^^

[jagabo]

Here's script that uses several masks to limit the blurring:

Mpeg2Source("Short Extract.d2v", CPU2="ooooxx", Info=3) 
TFM(d2v="Short Extract.d2v") 
TDecimate() 
src = last

# a blurred version of the video
blurred = BilinearResize(360,height).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=src.width, fheight=src.height).Sharpen(0.3, 0.0)

emask = Overlay(blurred.mt_convolution(horizontal="1", vertical="0 1 -1", chroma="-128"), blurred.mt_convolution(horizontal="1", vertical="-1 1 0", chroma="-128"), mode="add").mt_expand().mt_expand().ColorYUV(off_y=-3).ColorYUV(gain_y=10000).GreyScale()
smask = src.SaturationMask().mt_expand(mode="vertical").ColorYUV(off_y=-5).ColorYUV(gain_y=5000).GreyScale()
mmask = src.mt_motion(thy1=5, thy2=5, thT=255).mt_expand(mode="vertical").mt_expand().GreyScale()

fmask = Overlay(emask, smask, mode="multiply")
fmask = Overlay(fmask, mmask, mode="multiply").Blur(1.0)
Overlay(src, blurred, mask=fmask)
TemporalDegrain2()

emask is a mask of horizontal edges. smask is a mask of areas with color (dot crawl is caused by incomplete removal of the chroma carrier from a composite signal, it will only occur in/near colored areas). mmask is a mask of areas that differ from frame to frame. The dot crawl artifacts change from frame so this help locate those areas. The three masks are then essentially logically and'd together to fmask. Finally the blurred image is overlaid only where the mask indicates.

I didn't spend a lot of time tuning the masks. But this should give you an idea of how masks are used.