Is it possible to interpolate this 10,000 fps video?

[DMFOREVER98]

From a video of 2004 of 320x230 at 10,000 fps, I achieved in my opinion a decent quality of visualization (although I may be wrong) but its fps speed prevents that at least you can see well, in these cases what can I do to make the fps speed look better and less stuck to improve its visualization?

Captures of video "improved if you can say"

[Attachment 75251 - Click to enlarge]

[Attachment 75252 - Click to enlarge]

The only thing I would like is that the frame rate is better so that it looks at least good, but I don't know what to do in these cases.
I leave an excerpt of the video to get an idea and the video speciations. I can provide the source file if needed

[betwixt]

Thats ten frames per second, not 10,000 !

Interpolation might help.

Brian.

[ProWo]

The original 320x230 video has 25 fps.
You should correct your upsize workflow and work with the original frame rate.


[Attachment 75267 - Click to enlarge]

[DMFOREVER98]

The original 320x230 video has 25 fps.
You should correct your upsize workflow and work with the original frame rate.


[Attachment 75267 - Click to enlarge]

Hello, thank you for responding, I will provide you with the original file if it can be of any use since I would not know what to do about it.

[ProWo]

Get a better source here
https://player.vimeo.com/video/746077596

[johnmeyer]

Years ago I wanted to improve the only version (even to this day!) of a wonderful performance by the late Geoffrey Lewis (Clint Eastwood sidekick) and his group, Celestial Navigation. It had been released in 1992 on the very first multimedia CD-ROM subscription magazine called "Nautilus." It was 160x120 at only 12 fps. You can follow what I did by reading this 2011 post I made in the doom9.org forum:

Ideas for restoring early generation (really old) digital video

To my surprise, the link to my short clip of the original video still works, twelve years later!

If I were to re-do the same thing today, I would perform pretty much the same operations.

You'll find the link to the result of my efforts in post #9 in that thread. As you will see, because of the primitive nature of 1992 digital video as well as the really low (160x120) resolution, the spatial resolution is pretty awful. I was lucky that there wasn't too much motion, but in the cases where there is (especially at transitions), you'll see some pretty nasty morphing artifacts. That is pretty much unavoidable when you get to frame rates below 24 fps. I've done similar work on 10 fps video (the same as you have) and for portions of the video the result looks pretty good and on others, it looks awful. If I were to do this work for hire I'd do several versions using different techniques: repeating frames; blending frames; and the same thing as I did, but using totally different algorithms (I used the motion estimation found in MVTools2). When the original motion estimation fails (creates artifacts), I'd briefly switch to best of the results from the other algorithms.

That requires a lot of manual intervention so, as I said, I'd only do that work if someone was willing to pay me for it.

[Selur]

@johnmeyer: waiting for a codeformer successor for video with temporal consistency

[johnmeyer]

Wow. What did you use to create that MKV video? I want it! You cleaned up the blocking, lack of spatial detail, and the banding from the low-bit color quantization.

The individual frames are absolutely stunning.

I assume that when you said, "waiting for a codeformer successor," that you meant that you don't yet have a significantly better solution for generating the additional two frames between each existing frame to go from 10 to 30 fps.

[Selur]

Didn't look at the interpolation, thought that cleaning up the source would be more important.
I used Hybrid with the torchaddon, used dpir for the denoising, then codeformer nnedi3 for upscaling and qtgmc (input type = 3) trying to get some stability.
In the other version I used face enhancement model for the upscaling and moved codeformer behind it.
Main problem is the temporal instability of see: https://forum.videohelp.com/threads/412547-VSGAN-vs-mlrt-avs-mlrt-model-filtering-temp...ty#post2714817
Point is that especially for such low resolution videos using ml based filters might help to find new ways to improve them.

The individual frames are absolutely stunning.

That's mainly due to codeformer. Problem is that it does work fine on some and horrible on other sources,.. (at this point I'm mainly playing around with it to figure out when to use it best and what pre- and after filtering makes sense)

=> just wanted to react to the "If I were to re-do the same thing today, I would perform pretty much the same operations. " in a way to kind of say 'maybe first do some testing '

Cu Selur

[johnmeyer]

Thank you for the explanation. I have some work to do in order to integrate those tools into my future projects.

=> just wanted to react to the "If I were to re-do the same thing today, I would perform pretty much the same operations. " in a way to kind of say 'maybe first do some testing '

Excellent point. I only meant that I would first try to clean up each frame spatially and then do some sort of interpolation to increase the frame rate, the same as I did back then. However, I'll have to admit that I was expecting that while I might be able to find better tools for the temporal part of the problem, I had no idea that so much more can now be done spatially. Incredible!!

[Alwyn]

I'm no expert at AVISynth, in fact I'm a complete novice, but here's my attempt applying the Framerate converter filter (to 30 frames per sec), with a bit of denoising with Neat Video. I first converted the Sorensen MOV mono to Lagarith stereo and used that in my script.

I have my flak jacket on so rip into me as much as you like. I'm sure there is much more that could be done, video quality-wise.

The smoothness is much better than the original, but with some frame jumps here and there. Acceptable or not? It's all in the eye of the beholder.

AVISource ("H:\Videohelp\DMForever98\tonight_show199702320 LAGS.avi")
Converttoyv12()
FrameRateConverter(30) 
Spline36Resize(640,480)
Prefetch(24)

[Selur]

Instead of DirectShowSource use AviSource.

[Alwyn]

Thanks Selur.

[rgr]

I'm no expert at AVISynth, in fact I'm a complete novice, but here's my attempt applying the Framerate converter filter (to 30 frames per sec), with a bit of denoising with Neat Video. I first converted the Sorensen MOV mono to Lagarith stereo and used that in my script.

I have my flak jacket on so rip into me as much as you like. I'm sure there is much more that could be done, video quality-wise.

The smoothness is much better than the original, but with some frame jumps here and there. Acceptable or not? It's all in the eye of the beholder.

AVISource ("H:\Videohelp\DMForever98\tonight_show199702320 LAGS.avi")
Converttoyv12()
FrameRateConverter(30) 
Spline36Resize(640,480)
Prefetch(24)

Use RIFE instead of FRC.

[Alwyn]

I tried a script Jagabo made for me for another video but couldn't get it to work on this one. Could you come up with a script that would work for the file?

[rgr]

Just start with just RIFE()

Important: RIFE works with 32-bit planar RGB.


ffms2("H:\Videohelp\DMForever98\tonight_show199702 320 LAGS.avi")
ConvertToPlanarRGB(matrix="Rec709") # or Rec601
Convertbits(32)
RIFE() #def. model is 5 -- can test RIFE(model=6)
Spline36Resize(640,480) # maybe before RIFE

Convertbits(8)

It's worth doing Neat first, interpolation is simply better then.
From my experience - 1. resize, 2. Neat, 3. RIFE.

[lollo]

Also remember that RIFE is AviSynth+ only:

Requirements
Vulkan compatible device
[x64]: AviSynth+ r3682 or greater (AviSynth+ 3.7.3 (test 6, r3935 can be downloaded from here)
Microsoft VisualC++ Redistributable Package 2022 (can be downloaded from here)
Supported color formats: RGBPS
models must be located in the same folder as RIFE.

http://avisynth.nl/index.php/RIFE