Avisynth+ advice needed for video with issues

[elektro]

I'm trying to get the best quality out of this video that was unfortunately saved as progressive despite being interlaced. I need to apply the setting for the whole 55 minutes video, so the advice of Avisynth+ experts is welcomed.

My temporary script for now:

SetFilterMTMode("QTGMC", 2)
LWLibavVideoSource("Aurora.mkv")
AssumeTFF()
AssumeFPS(29.97)
QTGMC(Preset="Slow", FPSDivisor=1)
SRestore(23.976)
Deblock_QED(quant1=60, quant2=60)
TemporalDegrain2()
Prefetch(threads=4)

Still doesn't look right. What would you suggest?

[Sharc]

Try something like

LWLibavVideoSource("Aurora.mkv")
AssumeFPS(29.97)
tfm().tdecimate()  #returns 23.976fps frames
QTGMC(InputType=2)  #cleaning up

[Alwyn]

Is that an unusual telecine pattern? Never seen it before, not that I've seen a lot!

Certainly could fool the young players, with 4 interlaced frames in a row.

[elektro]

The clip is telecined. One has to IVTC (Inverse Telecine) it rather than deinterlace to restore the progressive 23.976fps.
Try something like

LWLibavVideoSource("Aurora.mkv")
AssumeFPS(29.97)
tfm().tdecimate()  #returns the progressive 23.976fps frames
QTGMC(InputType=2)  #cleaning up

https://en.wikipedia.org/wiki/Telecine#2:3_pulldown

Thanks for reply. I noticed that the video is telecined, that's why I used QTGMC (that also does some denoising) + SRestore to get 23.976fps. I know about TFM, the problem is that the video is a mess and the frames are hard to be reconstructed (there are visible artifacts where is movement, e.g. at hands; see the image below). I'd prefer to get rid of these artifacts as much as possible. Would be a good idea to use DeBicubic to a lower resolution and then enlarge it with nnedi3_rpow2? If yes, what would the exact optimal settings? I tried this but I'm not 100% content with it:

DeBicubic(640, 360)
nnedi3_rpow2(rfactor=2, nns=3, qual=1, fwidth=1280, fheight=720, cshift="Spline36Resize")

[elektro]

Is that an unusual telecine pattern? Never seen it before, not that I've seen a lot!

Certainly could fool the young players, with 4 interlaced frames in a row.

The biggest worry are those annoying artifacts (see the image above).

[Sharc]

Is that an unusual telecine pattern? Never seen it before, not that I've seen a lot!

It's just borked (I edited my original post mentioning telecine; certainly not properly telecined).

[elektro]

Is that an unusual telecine pattern? Never seen it before, not that I've seen a lot!

It's just borked (I edited my original post mentioning telecine; certainly not properly telecined).

Indeed. It was a livestream and whoever was responsible with it was an amateur. I'm trying to salvage this footage.

[Sharc]

.....Would be a good idea to use DeBicubic to a lower resolution and then enlarge it with nnedi3_rpow2? If yes, what would the exact optimal settings? I tried this but I'm not 100% content with it:

DeBicubic(640, 360)
nnedi3_rpow2(rfactor=2, nns=3, qual=1, fwidth=1280, fheight=720, cshift="Spline36Resize")

Something like this would have been my next attempt. One has to find the "best" downsize/upsize experimentally, sometimes it helps to downscale to the original resolution - if known ....
Decimation still needed to remove the duplicates.

[elektro]

.....Would be a good idea to use DeBicubic to a lower resolution and then enlarge it with nnedi3_rpow2? If yes, what would the exact optimal settings? I tried this but I'm not 100% content with it:

DeBicubic(640, 360)
nnedi3_rpow2(rfactor=2, nns=3, qual=1, fwidth=1280, fheight=720, cshift="Spline36Resize")

Something like this would have been my next attempt. One has to find the "best" downsize/upsize experimentally, sometimes it helps to downscale to the original resolution - if known ....

Thanks. I think I learned from you this trick before. Basically trial and error. This seems to generate good results (the artifacts are reduced a lot) but the resolution is too low. I'll try more numbers.

DeBicubic(480, 270)
nnedi3_rpow2(rfactor=2, nns=3, qual=1, fwidth=960, fheight=540, cshift="Spline36Resize")

[poisondeathray]

In terms of the cadence, if you separate the fields, each of even and odd field sets have every 4th, 5th field duplicated, but also with dropped fields in no discernable pattern. Thus the real content sampling rate is at least 47.952 if you decimated the dup fields. If you double rate deinterlace that, it's smoother in terms of a higher sampling rate, but still has jerks from the dropped fields.

(You cannot use filldrops derivatives, because the drop location is displaced from the duplicates. You could smoothskip, or john meyers dupes & drops script, and you could modify them to use rife instead of mvools2. But the problem is interlace - you can only interpolate progressive material. You could treat them as 2 separate even/odd field streams, but they will at least sometimes get the metrics incorrect and it will be jerky in some spots when weaved and deinterlaced. Reverse motion is far worse than gaps/dropped frame motion. Deinterlacing first won't work before smoothskip or johns's script - because of the duplicate fields, you end up with displaced duplicate frames (1 extra frame apart) that will not be detected by conventional means )

The main reason for my posting is you don't want to throw away 1/2 the real motion data for 23.976

orig=DGSource("Aurora.dgi")
e=orig.assumetff().separatefields().selecteven().tdecimate()
o=orig.assumetff().separatefields().selectodd().tdecimate()

interleave(e,o)
weave()
#yadif(1,1)
assumetff().qtgmc(preset="fast", border=true, sharpness=0.5) #or some other deinterlacer 47.952
#some other filters for the artifacts

For the artifacts - I don't think there are any great treatments

[poisondeathray]

The chroma "ghosting" is from improper interlaced handling of the chroma .

Here is an example that uses mfdin - a machine learning deinterlacer - I wanted to see if it would handle those chroma ghosting issues . There is a vapoursynth port of the lower quality model trained for damaged low quality source, and a larger HQ model (MFDIN-L) that is not publically released. The main issues with mfdin are that it's very very slow ( maybe 10-40x slower than QTGMC, depending on the GPU), and the LQ model damages fine high frequency details even more than QTGMC.
https://github.com/HolyWu/vs-mfdin

This used the script above up to the weave() without deinterlacing, and downscaled to "SD PAL" 720x576 using IResize(720,576), then applied either assumetff().qtgmc(preset="fast", border=true, sharpness=0.5) , or vs-mfdin



[Attachment 85005 - Click to enlarge]

cropped apng of the ghosting qtgmc (left), MFDIN_old_2P.pth (right) , it should animate in most browsers


5 sec sample

[elektro]

^ Interesting. Never heard of mfdin until now. But that slow processing speed is a deal breaker. I guess I'll stick to lowering resolution in order to make some artifacts go away.

[elektro]

In terms of the cadence, if you separate the fields, each of even and odd field sets have every 4th, 5th field duplicated, but also with dropped fields in no discernable pattern. Thus the real content sampling rate is at least 47.952 if you decimated the dup fields. If you double rate deinterlace that, it's smoother in terms of a higher sampling rate, but still has jerks from the dropped fields.

(You cannot use filldrops derivatives, because the drop location is displaced from the duplicates. You could smoothskip, or john meyers dupes & drops script, and you could modify them to use rife instead of mvools2. But the problem is interlace - you can only interpolate progressive material. You could treat them as 2 separate even/odd field streams, but they will at least sometimes get the metrics incorrect and it will be jerky in some spots when weaved and deinterlaced. Reverse motion is far worse than gaps/dropped frame motion. Deinterlacing first won't work before smoothskip or johns's script - because of the duplicate fields, you end up with displaced duplicate frames (1 extra frame apart) that will not be detected by conventional means )

The main reason for my posting is you don't want to throw away 1/2 the real motion data for 23.976

orig=DGSource("Aurora.dgi")
e=orig.assumetff().separatefields().selecteven().tdecimate()
o=orig.assumetff().separatefields().selectodd().tdecimate()

interleave(e,o)
weave()
#yadif(1,1)
assumetff().qtgmc(preset="fast", border=true, sharpness=0.5) #or some other deinterlacer 47.952
#some other filters for the artifacts

For the artifacts - I don't think there are any great treatments

I don't necessarily want 23.976fps, just the right fps for this video. Is that code you posted an optimal solution for this video to get the right fps?

Meanwhile, after some trial and error this code seems to do a decent job of cleaning some artifacts but the fps is 23.976:

SetFilterMTMode("QTGMC", 2)
orig=DGSource("Aurora.dgi")
e=orig.assumetff().separatefields().selecteven().t decimate()
o=orig.assumetff().separatefields().selectodd().td ecimate()
interleave(e,o)
weave()
tfm(mode=1, slow=2, pp=7)
QTGMC(InputType=2)
Prefetch(threads=4)

[Sharc]

Meanwhile, after some trial and error this code seems to do a decent job of cleaning some artifacts but the fps is 23.976:

QTGMC(InputType=2) is for smoothing and cleaning up of progressive video (what you get out of tfm(....) in your script). If you want to exploit the temporal resolution of an interlaced source you should skip tfm(....) but double-rate deinterlace instead, like QTGMC(InputType=0), or simply QTGMC(), or as pdr suggested.
At the end you decide what looks best to you (on PC or on TV).

[poisondeathray]

Big (field) drops are pretty consistent, every 8th frame (after the field decimation ,weave, double rate deinterlacing) . You can fill and interpolate those and that gives you 53.946fps. There are other little drops but more random harder to detect and fix

The way this sample was cut , big jumps (dropped frames) occur between 1-2, 9-10, 17-18....and it's consistent for the sample.

You can use selectevery and interleaveevery to interpolate the large gap dropped frame. The example uses RIFE because the quality is better than mvtools2 with fewer artifacts, but you could use framerateconverter/svpflow etc... or similar . You should filter the artifacts with whatever you decided on before interpolation step, otherwise the artifacts will be propogated into the new interpolated frames

It would look something like this

orig=DGSource("Aurora.dgi")
e=orig.assumetff().separatefields().selecteven().tdecimate()
o=orig.assumetff().separatefields().selectodd().tdecimate()

interleave(e,o)
weave()
#yadif(1,1)
assumetff().qtgmc(preset="fast", border=true, sharpness=0.5) #or some other double rate deinterlacer 47.952
#whatever other filters to fix artifacts
deint=last

deint.rifewrap(model=44)#.levels(0,0.5,255,0,255) #levels for testing visualization
#deint.framerateconverter(output="flow")#.levels(0,0.5,255,0,255) #levels for testing visualization
selectevery(16,3)
interp=last

interleaveevery(deint, interp, 9,2)