Sharpening leads in making object smaller and general Filtering-Question

[Platos]

Im new to Avi- and Vapoursynth. I use Hybrid-software and so i just tried out some filters because i have basically no clue, what i should use

Anyway, i just checked alot of them and the picture went better, but the Objects went smaller. So i testet out, which filter it is and its aWarpSharp2. Actually it's also the main-filter which gave the most quality-boost in my subjective eye.

Here a comparing with original and sharpening (and other filter): https://imgsli.com/MTczNDQ5

And here comparing original vs realCUGAN with Waifu Extension GUI no denoise: https://imgsli.com/MTczNDUx

In the RealCUGAN picture it looks (in my opinion) very good. But the problem is, it doesnt looks always that good. Often it looks overlighted/Corny/childisch. And lines looks "too dark" and thick. When i use sharpening after RealCugan it looks much better, but still have these effect, that the object gets smaller. So that's why i dont just use realGUAN, because it changes the artstyle a bit.

So my question is: Does someone has an idea, which settings in the sharping-settings i could change to still makes it look "sharper" without making obect smaller ? Or maybe i should use another sharpener than aWarpSharp2 ?

Also: Should i sharp before or after upscaling (with RealCUGAN) ?

And maybe does someone have an advice, what filter i should use ? I tried out a bit but the biggest effect has sharpening. Ist there any Vapour-syth guide? I mean not a guide which explain (only) the syntax, i mean explaing, how to make looking an old anime good again and which filter i have to use and so on.

[Selur]

aWarpSharp2 always reduces the width of lines.
If you want to get into filtering anime with VapourSynth some more, you should understand masking and field handling (deinterlaceing, ivtc,...).
Some links that might be interesting:
https://guide.encode.moe/index.html
https://www.l33tmeatwad.com/vapoursynth101/using-filters-functions
https://blog.kageru.moe/legacy/
old + AviSynth, but might still be worth looking into it: https://www.aquilinestudios.org/avsfilters/, https://www.animemusicvideos.org/guides/avtech31/post-qual.html
If you want to use ai based stuff, VSGAN and the models from https://upscale.wiki/wiki/Model_Database might also be interesting.

Also: Should i sharp before or after upscaling (with RealCUGAN) ?

Since RealCUGAN is quite heavy with the filtering, I would probably usually only sharpen after RealCUGAN.

Cu Selur

Ps.: no there is no Hybrid specific guide afaik.
Pss.: Hybrid also does allow some basic masking and scene/region based filtering, but the options for this are disabled by default. (can be enabled under: Filtering->Vapoursynth->Misc->UI) Hybrid does not allow scene based resizing. Hybrid does allow to add filters (aside from crop, resize, deinterlace to add multiple times if FilterQueue is used instead of FilterOrder (https://forum.selur.net/Thread-INFO-About-Vapoursynth-Filter-Order-Queue).

[Platos]

Ok, thank you, will read the stuff.

About VSGAN: There is no GUI with that integrated, right? Because im not that good with Commands.

And what do you mean with "heavy with the filtering" ? You mean realCUGAN tend to produce artefacts, when using filtered stuff? Because i used RealCUGAN after filtering and now it flickers around like if my graphics card is defect. Strange is, that in the filtered video (before upscaling) i see none of these strange flickering. So you are probably right.

Do you think, sRestore will also make problems, when i use it before RealCUGAN ? Because RealCUGAN has problem with frameblending. I believe it uses infromation in frames before the actual processing frame. So it takes Infromation from a blending frame maybe and then the Frames get really ugly. Because RealCUGAN sometimes upscale even the "good" frames bad, when there is a scene with frame-blending.

At least in the 2 minute testvideo i did not saw any problem with using RealCUGAN after sRestore and Deinterlacing.

[Selur]

Do you think, sRestore will also make problems, when i use it before RealCUGAN ?

no

And what do you mean with "heavy with the filtering" ?

depending on the source it sometimes remove quite some of the small details.

About VSGAN: There is no GUI with that integrated, right? Because im not that good with Commands.

There is an addon for Hybrid which, atm. adds support for:

• VSGAN
• vsDPIR
• vsRIFE (torch)
• vsGMFSS union
• vsBasicVSR++
• vsRealESRGAN
• vsSwinIR
• vsHINet
• vsAnimeSR
• vsFeMaSR
• vsSCUNet
• vsCodeFormer
• vsGRLIR

-> will send you links via pm in ~30min

Cu Selur

[Platos]

Thank you very much man. You can write me in german in PM if you want.

[Selur]

Don't really care whether I write German or English.

[Platos]

I have a Question about VSGAN. Does 32bit mean 32 bit color or is that 32bit file system ? Because if 32 Bit color it is a bit overpowered and if 32 Bit filesystem, why is this default setting then ?

And general question: I guess 2x in upscale-models is better than 4x ? But what does that mean? I mean i already set the resolution, so what does that number mean?

But VSGAN is quite slow. I tried some and all of them i tried was 0.1-0.5 fps speed. While RealESRGAN 4xPlus Anime is 1.7FPS. So we all now RealESRGAN is super slow, but VSGAN is even more super slow

Some of them does a good job, but because of the speed its no option for Videos. Maybe if you want to upscale some picture.

Or do you think, it has sth. to do with hybrid? Maybe its faster, if i would use it with commands and without a gui ?

[Selur]

I have a Question about VSGAN. Does 32bit mean 32 bit color or is that 32bit file system ? Because if 32 Bit color it is a bit overpowered and if 32 Bit filesystem, why is this default setting then ?

It's about calcluation precision 32bit (double) vs 16bit (float). Normally 16bit is faster, but for some unknown reason usually 32bit VSGAN is faster here.

And general question: I guess 2x in upscale-models is better than 4x ? But what does that mean? I mean i already set the resolution, so what does that number mean?

VSGAN models have fixed scaling factor Xx, to archive your chosen target resolution, Hybrid then additionally uses the chosen resizer.
Usually higher scaling factor means more time since more pixels get created.

But VSGAN is quite slow. I tried some and all of them i tried was 0.1-0.5 fps speed.

yup

While RealESRGAN 4xPlus Anime is 1.7FPS. So we all now RealESRGAN is super slow, but VSGAN is even more super slow

you do not know what SLOW is,..

Some of them does a good job, but because of the speed its no option for Videos.

with your setup

Or do you think, it has sth. to do with hybrid? Maybe its faster, if i would use it with commands and without a gui ?

No, it's simply your system. Using the command line projects isn't really faster afaik.

[Platos]

What do you mean with my setup? I mean RealCUGAN works like 100 times faster on my setup (10-20FPS vs 0.1-0.2FPS most of time, maybe 0.5FPS). Sure, you can say at the end my hardware is to slow, but compared with others (RealCUGAN for example) its just 50-200x slower on same setup

And thanks for clarification. if its double precision (32bit) i will let it on 32bit.

[Selur]

Yes, VSGAN is slower than RealCUGAN, but that it's only 0.1-0.2fps depends on your hardware, drivers,... and the resolution of the input.

[Platos]

Hmm, the resolution of the input is 720x576.

What FPS do you have usually and which CPU and GPU do you have ? Is my result quite bad ? I have an RTX 4070Ti and i5 12400f CPU. I could try update my drivers.

[Selur]

I'm using:
OS Version Windows 11 x64 (22621) [UTF-8]
CPU AMD Ryzen 9 7950X 16-Core Processor [5.52GHz] (16C/32T)
GPU #0: NVIDIA GeForce RTX 4080 (9728 cores, 2505 MHz)[PCIe4x16][531.61]
CPU isn't really used in this, it's decoding, filtering, encoding is all done in the gpu.
Speed totally depends on the other filters and the general filter order you use too.
Using a 720x576 source, resizing to 1440x0180 with VSGAN and using NVEnc AV1 as encoder
I just got:

• 19.94 fps. with 2x_LD-Anime_Compact_330k_net_g.pth
• 0.38fps with 2x_LD-Anime_Skr_v1.0.pth
• 0.5fps with Using 2x_SHARP_ANIME_V1.pth
• 19.74 fps with 2x_Bubble_AnimeScale_Compact_v1.pth
• 15.84 fps with 4x-anifilm-compact.pth

-> remember, compact models are usually a lot faster.

With RealESRGAN and x4plus_anime (TensorRT+FP16) I got: 11.51 fps

Cu Selur

[Platos]

Ah ok, these models have a huge difference between speed. I will have to try the 3 fast ones.

It seems i had just no luck. I got:

- 0.1-0.2 fps and nearly no visible improvement with "Anime2x_Loyaldk-Keroro_650000_V1.0"
- 0.1-0.2 fps and tends hard to make 2 lines out of 1 blurry (useless for me) with "4x-AnimeSharp"
- 0.1-0.21fps (It looked not bad, but it has a yellow filter, which is maybe good for these very old cartoons) with "4x_OLDIES_290000_G_FINAL_interp_03" and 4x_OLDIES_ALTERNATIVE_FINAL"
- 0.5FPS, looked not bad, but to slow. With: "2x-UniScale_CartoonRestore-lite" (downloadet it)

Crash with 2/4x-anifilm-compact.

Only the first one and one of the "OLDIES" where in Hybrid. The rest i have downloaded.

[Selur]

If you use external model, you might want to rename to the form Xx_... since Hybrid can read the scale factor this way.