VSGAN/vs-mlrt/avs-mlrt model filtering temporal stability?

[Selur]

Since more folks are using models from https://openmodeldb.info/ what do you use for temporal stability to not end up with a result like the file I attached?
QTGMC with InputType=3 seems to help somewhat, but I was wondering what others came up with.

Cu Selur

[SaurusX]

Do not use lossless encodes. I find that an encode at CRF 13 or 14 provides much more stable results than feeding in a lossless video with a bunch of random noise preserved. That noise gets converted to “detail” which causes a lot of temporal instability.

[Selur]

Hmmm,.. so basically smoothing out details from the source.

[SaurusX]

Hmmm,.. so basically smoothing out details from the source.

Essentially yes. At CRF 13 a video is going to be pretty much "lossless" to human eyes, but it makes a noticeable difference to AI upscaling since it's much more temporally consistent. And as we know, these AI upscalers do not have any temporal coherence, so I think you need to add it in somewhere.

[Quint]

My idea: Don't use A. I. upscalers.

[Selur]

My idea: Don't use A. I. upscalers

Don't know of any non ml/ai based upscaler that comes anywhere near those results when upscaling the attached source to 1280x1024.
(especially when codeformer is added)
So if you know an alternative, please share.

Cu Selur

[Quint]

Definitely not. I didn't know, it was that bad.
In really most cases I'd always prefer non-A.I.-resizing plus unsharp masking(s) - but here I pass.
It seems you'll need some A.I.-based temporal smoother...

[Selur]

Problem is that both '4x_Faces_N_250000.pth' and 'CodeFormer' only work frame-by-frame with no temporal concepts, so this is an extreme case.
(Hoping that there will be a follow up paper&co in the next few years that addresses video.)

Cu Selur

Ps.: Attached a QTGMC input type 3 version of the Face+CodeFormer file.

[Quint]

QTGMC input type 3 sometimes works well, sometimes there remains a lot of combing, didn't find out yet for what reason exactly. Deinterlacing here looks fine, but much too much temporal smoothing (how does it look with stabilizenoise=false, tr2=0 ?) Sorry, no time to test myself, early tomorrow big voyage.

[Selur]

There was no deinterlacing, on my way to bed myself, but if I find some time I'll look at it tomorrow.

[Quint]

Ok, residual de-combing and artefacts-removing.

[Selur]

Adding "stabilizenoise=false, tr2=0" doesn't really make a difference. (see attachement)

Cu Selur

[Selur]

btw. can someone who used Topaz and has some experience with it show what Topaz will do when upscaling suzie_lossless..mp4 ?