best interpolation to make bad stutter conversion smooth

[skh]

hello,

this is a nightmare but maybe some of the expert had something like this before and already figured out what would be the best avs way+threshold to fix this https://workupload.com/file/mZdb34CAFa2

thanks a lot for looking into it

[davexnet]

This is a little better, but not perfect by any means.

vid=lwlibavvideosource("C:\Users\davex\Downloads\Die.xue.jiang.nu.mkv")
aud=lwlibavaudiosource("C:\Users\davex\Downloads\Die.xue.jiang.nu.mkv")
audiodub(vid,aud)
tdecimate(cycler=3,cycle=25)
changefps(23.976)

[skh]

thanks that is much better so far. Now I we just have to find a clever way to interpolate the remaining dupes

[jagabo]

How's this?

function DoubleFPS_RIFE(clip source)
{ 
    source.z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
    Rife(model=5, sc=true, sc_threshold=0.10)
    z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l")
}

LWLibavVideoSource("Die.xue.jiang.nu.mkv", cache=false, prefer_hw=2) 
Crop(0,4,0,0).AddBorders(0,4,0,0) # remove the flickering line at the top of the frame

alt = DoubleFPS_RIFE().SelectOdd() # double frame rate, keep only the newly interpolated frames
test = mt_lutxy(last, last.Loop(2,0,0),"x y - abs", chroma="-128") # abs diff between frames, for duplicate detection

ConditionalFilter(test, alt, last, "AverageLuma", "lessthan", "0.5") # replace nearly identical frames with the motion interpolated frames

If you don't like RIFE use another frame rate doubler like FrameRateConverter() or InterFrame().

[skh]

....................If you don't like RIFE use another frame rate doubler like FrameRateConverter() or InterFrame().

that looks super smooth and amazing. I heard nothing but good things about RIFE and just installed it right after I saw your sample.

One last question about it. You didn't apply any conversion to 23,976 such as the one davexnet suggested or a deblend.
So would you say that 25fps is the right framerate and this is running in 25fps speed and the cause for the stuttering wasn't the result of a wrong framerate conversion?

[jagabo]

You didn't apply any conversion to 23,976 such as the one davexnet suggested or a deblend.
So would you say that 25fps is the right framerate and this is running in 25fps speed and the cause for the stuttering wasn't the result of a wrong framerate conversion?

Since I only replaced duplicate frames with interpolated frames there was no change of frame rate. It looked to me like 25p was right. I didn't see any need for deblending (I didn't see any blended frames in the source). In my opinion it was just a bad cap where dropped frames were replaced with duplicates. If it comes from a film source and you want to restore the 24p film frame rate you can do that easily with an AssumeFPS(24) or AssumeFPS(24000, 1001) -- and you'll have to change the length of the audio too.

[skh]

Since I only replaced duplicate frames with interpolated frames there was no change of frame rate. It looked to me like 25p was right. I didn't see any need for deblending (I didn't see any blended frames in the source). In my opinion it was just a bad cap where dropped frames were replaced with duplicates. If it comes from a film source and you want to restore the 24p film frame rate you can do that easily with an AssumeFPS(24) or AssumeFPS(24000, 1001) -- and you'll have to change the length of the audio too.

alright sounds great. I think I got it all running now and will start the encode tomorrow. I stayed with RIFE 1.1 as I read that newer versions don't work right in one of your other posts. I also added gpu_id = 0 to the RIFE command to make shure that it uses my main GPU which is the GeForce RTX 4070 and not the onboard weak one. I experimented a bit by changing gpu_thread = 4 but it wasn't faster than the default 2 so I guess there is no use.

[Sharc]

How's this?

function DoubleFPS_RIFE(clip source)
{ 
    source.z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
    Rife(model=5, sc=true, sc_threshold=0.10)
    z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l")
}

LWLibavVideoSource("Die.xue.jiang.nu.mkv", cache=false, prefer_hw=2) 
Crop(0,4,0,0).AddBorders(0,4,0,0) # remove the flickering line at the top of the frame

alt = DoubleFPS_RIFE().SelectOdd() # double frame rate, keep only the newly interpolated frames
test = mt_lutxy(last, last.Loop(2,0,0),"x y - abs", chroma="-128") # abs diff between frames, for duplicate detection

ConditionalFilter(test, alt, last, "AverageLuma", "lessthan", "0.5") # replace nearly identical frames with the motion interpolated frames

If you don't like RIFE use another frame rate doubler like FrameRateConverter() or InterFrame().

Amazing solution, amazing also the duplicates detection.
Btw. is the selection of the model in RIFE a matter of experience (~trial and error ....), or are there guidelines?

[jagabo]

I haven't found a good synopsis of the various models in RIFE. I mostly just use trial and error. If you find one let me know!