Best AI Upscaler for Real Footage (Films)

[Platos]

Im wondering, if there is any AI upscaler model, which does NOT make a real footage look artificial

I am using atm the VSGAN Model "LD-Anime Compact" for Anime and it does a really good job in "not changing artstyle", not washing out textures and still improve the sharpness of the image significantly.

But on real-footage for sure there is a general problem, Animes/Cartoons dont have: All models i used so far had 2 problems: 1. either it did make the real footage looks not "real" anymore (like computer generated) or 2. it did wash out textures (= denoise).

So without denoise i could never really improve the feeled "sharpness" of a real film with an AI upscaler. On anime this is "very" easy.

So my question is, does anybody knows a model, which does not denoise (=texture killer) and still is able to make a real film looking "sharper" without making the image looking computergenerated ?

[Selur]

In general, I would recommend to not use VSGAN&Co, but rather NNEDI3 + CAS.
Also the problem with VSGAN is that basically all the models have no temporal awareness, this for real footage can cause inconsistencies.
If you share a small sample which looks typical for the content you want to upscale, others might suggest something.

Cu Selur

[rrats]

do not use ai upscalers nnedi3 and cas are way better

[Platos]

I will try it on real footage. On anime the effect from nnedi3 and cas was always a joke for me (not visiable on modern media player, because they already use similair things per default). But i will give a try.

But on real footage on the other hand AI upscaler do bad job (all i tested).

But i will do also more research on AI upscaling.

[Selur]

Here's a simple example (tweaked settings, stepped resize, etc.), what a few different resizers do https://imgsli.com/MTg0MTM3
(I tried to stick to resizers which would not create a plastique look.)
Usually when upscaling Animes you want cleaning, sharpening, contrast adjustment, which are relatively easy to define since (especially older animes/cartoon) have simple structures.
Real world content is a totally different ballpark, with real content.
Most folks will way easier spot too strong filtering (over sharpening/smoothing/...) since you know how this stuff looks in reality.

Cu Selur

[sophisticles]

For me, in the samples posted by Selur, RealESRCAN looks the best, as in more like the original.

[Platos]

Thanks very much Selur. (Edit: Which Waifu2x Model did you use? And which BasicVSR++ ?)

For me BasicVSR++ looks best. NEDI3 looks horrible. It looks sharper, but also you can see in the Face there appeares now these "corny" structure. I dont like that.

On the Other hand RealSR and RealESRGAN looks too "plastic". This word describes it good. And when you look on her hair, you can see it looks not good for me.

BasicVSR++ does make the Image slightly sharper, but does not make so much "corny" like the the two CAS version. Waifu2x is very similair, but overall a bit less sharp. So i like BasicVSR++ best in your example.

Thank you, i will try that out on sth.

Always important is the moving-part. In this kind of stuff you easly tend to look hours for hours on single images. But at the end you forget to check it on a real video scene xD Often it looks very good then.

For me, in the samples posted by Selur, RealESRCAN looks the best, as in more like the original.

Why ? The Hair looks very "plastic" like.

[Selur]

I think when he asked:

So my question is, does anybody knows a model, which does not denoise (=texture killer) and still is able to make a real film looking "sharper" without making the image looking computergenerated ?

He asked for freely available models.
Afaik. Topaz Video Enhance AI does use .onnx models, but they are encrypted, but yeah he could try a trial of Topaz Video Enhance AI and if he likes it buy it.

Cu Selur

[rgr]

I think when he asked:

So my question is, does anybody knows a model, which does not denoise (=texture killer) and still is able to make a real film looking "sharper" without making the image looking computergenerated ?

He asked for freely available models.
Afaik. Topaz Video Enhance AI does use .onnx models, but they are encrypted, but yeah he could try a trial of Topaz Video Enhance AI and if he likes it buy it.

Cu Selur

But if you want to pay for a denoiser, it's better to buy Neat Video. Much, much better than Topaz. And cheaper.