Is there a way to remove all Dotcrawling & Rainbow, and Combing?

[themon-master]

I'm currently trying to restore this quality, I have little to no solutions on how to remove the Dotcrawl & Rainbow as well with combing on this source.
The source doesn't have too much Dotcrawling & Rainbow, since it's a little difficult to find them on most scenes (some scenes are REALLY heavy)
But on the other hand, the combing is a huge problem because it ruins most of the quality, and the sad part is, the source is progressive, and not interlaced.
Can anyone help me out here? thanks!

[Attachment 91973 - Click to enlarge]

[Attachment 91974 - Click to enlarge]
https://drive.google.com/file/d/1tR7tXInOPRtSKCQCp61kOinIgv934uTv/view?usp=sharing
(NOTE: I'm using AviSynth
If anyone can lead me to a site that has a lot of description that can restore quality from AviSynth please?,
I'm going to need A LOT of research to restore quality like Dotcrawling & Rainbow I'm trying to remove, thank you!)

[Sharc]

Take a look here
https://forum.videohelp.com/threads/420071-How-to-remove-DotCrawl-and-Rainbow-artifact...m-an-Anime-DVD
https://forum.videohelp.com/threads/419542-How-to-remove-this-interlaced-artifact-when-deinterlacing

[themon-master]

Take a look here
https://forum.videohelp.com/threads/420071-How-to-remove-DotCrawl-and-Rainbow-artifact...m-an-Anime-DVD
https://forum.videohelp.com/threads/419542-How-to-remove-this-interlaced-artifact-when-deinterlacing

These are completely different sources, and the codecs people sent me, doesn't do a good job removing all of it or makes the source have different artifacts.

[jagabo]

That file is progressive but it was poorly upscaled from a poorly deinterlaced standard definintion source. If you want decent results you need to start with that SD video, not this crap. There are many other threads here dealing with that show.

[Selur]

I don't see any combing in the provided source which seems to be wrongly flagged as tff, but seems to be progressive.
+ I agree using a SD source would be preferable, alternatively using descale before filtering seem to be a good idea.

Avisynth:
a. Crop to 1068x1070 +Resize to 640x640, SpotLess(r=1), ReduceFlicker(strength=3, aggressive=1), MCTemporalDenoise, Upscale (spline)
b. same as a. but with RealCUGAN (mlrt_cugan)

side note: using Vapoursynth and BasicVSR++ (script)

Cu Selur

[themon-master]

That file is progressive but it was poorly upscaled from a poorly deinterlaced standard definintion source. If you want decent results you need to start with that SD video, not this crap. There are many other threads here dealing with that show.

Well, here's the thing, this is an original (UNFILTERED) Japan TV airing from a transport stream file, they aired this in progressive instead of using 3:2 interlaced pulldown (which is weird they did that, the channel often doesn't air in progressive), that's also why you see a logo on the top right.

[themon-master]

I don't see any combing in the provided source which seems to be wrongly flagged as tff, but seems to be progressive.
+ I agree using a SD source would be preferable, alternatively using descale before filtering seem to be a good idea.

Avisynth:
a. Crop to 1068x1070 +Resize to 640x640, SpotLess(r=1), ReduceFlicker(strength=3, aggressive=1), MCTemporalDenoise, Upscale (spline)
b. same as a. but with RealCUGAN (mlrt_cugan)

side note: using Vapoursynth and BasicVSR++ (script)

Cu Selur

The first file ("messing around") looks really good because the artifacts I mentioned seemed to be removed! (and, I don't see to much flickering on the text, nice job!), but there is one major down flaw, and that seems to be the blurring between frames.
I'm sorry that I didn't really gave out a good sample of the combing, but you seemed to fix the combing anyways

[Selur]

'messing around' is using Vapoursynth and BasicVSR++ and Reduceflicker, one could still sharpen thinks,
Also attached an example when using just RealESRGAN and ReduceFlicker (both available in Avisynth too)

=> there are probably other filter combination which can help too, just wanted to showcase a few options that directly came to mind.

Cu Selur

[jagabo]

It also needs to be decimated back to 24p.

[themon-master]

'messing around' is using Vapoursynth and BasicVSR++ and Reduceflicker, one could still sharpen thinks,
Also attached an example when using just RealESRGAN and ReduceFlicker (both available in Avisynth too)

=> there are probably other filter combination which can help too, just wanted to showcase a few options that directly came to mind.

Cu Selur

This render is some how worse, the frame blending is more noticeable and it makes the quality look really bad, rainbow can still be spotted, which wasn't there in the Vapoursynth render. Yes, there is no more Dotcrawl, but you made everything else look bad now, worse than the original.
What is causing this terrible frame blend?
(Is there literally any script that can ONLY remove Dotcrawl & Rainbow, and possibly Combing? nothing else?)
Another thing, If possible, could you only do VapourSynth or AviSynth? (If you were able to remove more Dotcrawl & Rainbow, and possibly Combing, that's fine, but I don't want anything else)

[Attachment 91992 - Click to enlarge]

[Selur]

Probably the ReduceFlicker

[themon-master]

Probably the ReduceFlicker

Can I see how it looks like without the ReduceFlicker, please?

[Selur]

attached a version where I just down scaled and then upscaled with RealESRGAN.

Cu Selur

Ps.: Yes, TDecimate() should be added

[themon-master]

attached a version where I just down scaled and then upscaled with RealESRGAN.

Cu Selur

Ps.: Yes, TDecimate() should be added

It feels like we're getting nowhere with this, how about we start fresh by ONLY using AviSynth (or VapourSynth), we'll remove the situations of the quality one, by one (unless if the models have more than one).
Let's start with the rainbow first, how much rainbow we can remove by ONLY using Rainbow Smooth?.
If you want, I can give you more samples.

[Selur]

(Aside from BasicVSR++, everything I posted should be available in Avisynth.)

Let's start with the rainbow first, how much rainbow we can remove by ONLY using Rainbow Smooth?.

Come on, you should be able to figure that out by yourself.

[themon-master]

Come on, you should be able to figure that out by yourself.

I'm trying to get Rainbow Smooth to work, but for some reason, it's not working correctly, so before I start using a lot of time to fix this situation,
I thought this would be a good time to see how this will work, instead of the final output isn't how I want it to look like (and I could have wasted a lot of time), but I understand if you don't want to do it

[Selur]

Using Decimate().RainbowSmooth():

Using Decimate().RainbowSmooth(radius=7)

[themon-master]

Using TFM().RainbowSmooth(radius=7)

This looks really good!, I still see some rainbow if that can be fixed

[Selur]

I still see some rainbow if that can be fixed

there are tons of filters that are aiming to remove rainbows, also denoisers applied just to the chroma plane can help, some machine learning based stuff can help,..
Personally, I find the crawlers and flickering more annoying than the rainbows.

Small correction: didn't use TFM(), but TDecimate().

=> simply try some filters and use whatever you like the best

[themon-master]

I still see some rainbow if that can be fixed

there are tons of filters that are aiming to remove rainbows, also denoisers applied just to the chroma plane can help, some machine learning based stuff can help

I know there's a lot of filters that can remove rainbow, the problem is that I can't find all of those filters, do you know a site that lists all the rainbow filters?
Or any guide that can describe how to remove all the Dotcrawling & Rainbow, and Combing?

[johnmeyer]

GamMac can be used for rainbows. See my post:

https://forum.videohelp.com/threads/398864-Need-help-on-Avisynth#post2595644

[Selur]

Since you use Avisynth, I would first look over at: http://avisynth.nl/index.php?title=External_filters#Rainbow_.26_Dot_Crawl_Removal

Or any guide that can describe how to remove all the Dotcrawling & Rainbow, and Combing?

Sorry, no clue, haven't looked for Avisynth guides or similar for ages,... (I would google for 'Anime Avisynth filtering' or 'Cartoon Avisynth filtering')

[themaster1]

I like mtedge for dotcrawl:

main=last
main
bicubicresize(480,320)# source NTSC
spline64resize(720,480) # source NTSC
lsfmod
filtered=last
main
mt_edge(mode="prewitt",chroma="copy").mt_expand() # prewitt ->more agressive
mymask=last
mt_merge(main, filtered, mymask)

[Sharc]

GamMac can be used for rainbows. See my post:

https://forum.videohelp.com/threads/398864-Need-help-on-Avisynth#post2595644

You mean Spotless rather than GamMac?

[johnmeyer]

GamMac can be used for rainbows. See my post:

https://forum.videohelp.com/threads/398864-Need-help-on-Avisynth#post2595644

You mean Spotless rather than GamMac?

Oops. Yes, Spotless. StainlessS has too many tools.

[Sharc]

@OP: Maybe try SRFComb.avsi or SRFComb2.avsi
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/SRFComb.avsi
https://forum.doom9.org/showthread.php?t=186908