Avisynth filters to remove mosquito noise without softening

[AlanHK]

I'm working on an animation AVI that has rather a lot of mosquito noise around the edges.

It has the animated parts (people, animals, etc.) as outline and solid colour, and the background looking more painted, with few outlines.

I want to clean it up in Avisynth.
I can use the MSmooth filter to get rid of mosquito noise from the animated parts quite nicely, but that ends up softening the background detail a lot.

I've tried FFT3DFilter and Fluxsmooth, but neither remove the noise as well as MSmooth, and both also soften.

Adding sharpening after helps somewhat, but still there is a noticeable loss of detail.

Also it's 25fps, but almost all is 12.5 fps with frames duplicated (11 22 33 44...) so I have used Dup() with blending which cuts the noise a little.

Any suggestions?

This image is unfiltered, except for resize and Dup :major.png

[jagabo]

DeBlock() and UnDot().

[AlanHK]

DeBlock() and UnDot().

I tried Undot, had a very small effect. But at least it didn't do any harm.

There is some blocking, but that's not my major concert.

My best result so far is with:
FFT3DFilter(sigma=5.0,bw=32,bh=32,bt=4,ow=16,oh=16 ,sharpen=0.4,plane=0)
and that does some deblocking.

[jagabo]

Be sure to run them both before any resizing or cropping.

[poisondeathray]

Is your primary concern the edge noise without softening background like the strokes in the trees? ie. In your example, the dog outline edge noise is what you're concerned about?

If it is, you can use filters who target edges with edge masks, like Dehalo_alpha and Edgecleaner , and adjust the strength/options to your tastes. If you still want more denoise, or specfically target the minor blocking, you can add a general denoiser with reduced strength, and/or a deblock filter into the filter chain to your tastes.

You only posted a still, so it's difficult to guess or see what other things you might be looking at, and some filters work temporally on moving noise patterns. Obviously these can't be tested on stills

ImageSource("major.png")
ConvertToYV12()
Dehalo_alpha(darkstr=0.5,brightstr=0.5,rx=3,ry=3)
Edgecleaner(strength=20)
Toon(strength=0.5)
LSFMod(strength=50)

Using your FFT3Dfilter settings above:
fft3d.png

Using the strategy targeting edges:
chain.png

[AlanHK]

Is your primary concern the edge noise without softening background like the strokes in the trees? ie. In your example, the dog outline edge noise is what you're concerned about?

Yes.

If it is, you can use filters who target edges with edge masks, like Dehalo_alpha and Edgecleaner , and adjust the strength/options to your tastes. If you still want more denoise, or specfically target the minor blocking, you can add a general denoiser with reduced strength, and/or a deblock filter into the filter chain to your tastes.

You only posted a still, so it's difficult to guess or see what other things you might be looking at, and some filters work temporally on moving noise patterns. Obviously these can't be tested on stills

Thanks, these do give a better result on the animated cartoony parts, and less blurring of the background.
I tried temporal filters but none seemed to make much difference; the noise is only slightly different in succeeding frames.

[darkdream4]

Be sure to run them both before any resizing or cropping.

Should that apply to all the filters that smooth or change video?

[AlanHK]

Be sure to run them both before any resizing or cropping.

Should that apply to all the filters that smooth or change video?

In general, resize last.
Especially after "cleaning" filters, as resizing can blur dirt marks and other problems you're trying to remove and make them harder to fix.

But you may crop off bad edges or especially letterboxing earlier, though some filters require the size to be a multiple of 8.

[darkdream4]

Be sure to run them both before any resizing or cropping.

Should that apply to all the filters that smooth or change video?

In general, resize last.
Especially after "cleaning" filters, as resizing can blur dirt marks and other problems you're trying to remove and make them harder to fix.

But you may crop off bad edges or especially letterboxing earlier, though some filters require the size to be a multiple of 8.

Thanks, I do find the output less blurry when done that way.