Software and/or AI to fix damaged videos of a kayaking trip?

[GaiaWanderer]

I took some videos while kayaking around the ships of the maritime museum in San Diego Bay, using a CamPark (GoPro knockoff). Someone told me the damage was most likely caused by the camera overheating.

The best description I have is that the videos freeze and stutter. Here is the worst one, it's 12 minutes long and constantly freezes for a second or two at a time:

https://youtu.be/JkoNNE8CoFA

Here's the MediaInfo results: https://filebin.net/cfv9fnk4cz99ktrb

I'm super bummed, because now I'm one the east coast and I'll probably never have a chance to go back with my GoPro.

So far, I've tried Flowframes, Video2x, and VideoProc Converter AI. I've only managed to make minor improvements to the least damaged ones.

TIA, hopefully someone here can point me in the right direction to possibly save these videos.

[Swedaniel]

what format are they in? does this camera have usb port to transfer files to the computer? if it use the cameras own "program" to transfer the files, have you tried to transfer the files again back to the computer with for example an other usb cable or an other usb port or check the usb port settings to make sure it does not malfunction during the transfer? the video on the camera might not be exactly the same as it will be when it arrive to the computer and during this transfer some cameras sort of compile/encode the video to the disc and not just transfer, if there is errors/flaws/malfunction to usb port or the built in software for this the video might not function well but the original video is still intact and can just transfer it again but check usb cables, settings and such. Or whatever could possibly disturb the transfer from the camera! Check so this transfer software launch proper, sometime it it is just an small window or tray icon that come avalible!

Try use usb 2.0 if the version 3.0 doesnt work, by using an other cable or an other usb port! can help with lower speed for this occation! The camera would use very much battery if it was fully encoding the videos onto the built in storage, aswell there can be an delay after finishing the recording for it need to encode the video so it might complete the task during transfer instead!

Edit: If the camera then have options to select different encoding types or such functions you might want to check it out allso once you're at it! With some cameras you might notice allso the recorded format is not the same as the one after the transfer (the encoded one) and it may ask to transfer back the encoded version and to then play this video from the cameras built in program will launch an other built in player with more video options but less editing options and from here allso this encoded video can sometimes consume more battery to play on the camera! Some camera people say allmost allways it is best to save the original file on the camera memory!

[GaiaWanderer]

what format are they in? does this camera have usb port to transfer files to the computer?...

Sorry, I should have posted the file's details. Here's the MediaInfo results: https://filebin.net/cfv9fnk4cz99ktrb

They were filmed in 2021 and now only reside on my PC. The MP4 files were copied directly from the CamPark's SD card.

[Swedaniel]

Ok! Well if it was an museum, did you use any geo tagging? Did you use any type of built in wifi to download geo taggin locations? (which parts of this video is an protected area of this museum, or around the museom or some sort of nature reservate, and this video must be unlocked via some geo tagging thing or partly encrypted/failed decoding? Tried any other video players? And im sorry i do not click external links/pictures/images/videos!

You could try play the video in VideoLAN's VLC Media Players (it is free open source software) it can play basicly anything! Ellse maybe if you have DivX Player you could try rename the video from .mp4 to .avi which tend to decode the video to mpeg-2 instead of mpeg-4!

Edit: Ellse if you could find an other media player to decode it successfully, if it even works, maybe if you're lucky you can export the video to mpeg-2 from this video player and then convert it into mp4! (Try look for setting "decode mpeg-4 videos to mpeg-2" and some players allow to export the hopefully successfully decoded mpeg-2 videos).

Allso on this website there is some sort of mp4 repair/mp4 tools avalible, maybe they can help out? I tried one repair tool for avi before that i found here (DivFix++) and it fixed my video, but it was broken index for an avi file, maybe the repair tools here for mp4's can be of help?

[Selur]

Can you share a sample of the source? Downloading a reencode from youtube doesn't make sense to me.

[rgr]

I took some videos while kayaking around the ships of the maritime museum in San Diego Bay, using a CamPark (GoPro knockoff). Someone told me the damage was most likely caused by the camera overheating.

The best description I have is that the videos freeze and stutter. Here is the worst one, it's 12 minutes long and constantly freezes for a second or two at a time:

https://youtu.be/JkoNNE8CoFA

Here's the MediaInfo results: https://filebin.net/cfv9fnk4cz99ktrb

I'm super bummed, because now I'm one the east coast and I'll probably never have a chance to go back with my GoPro.

So far, I've tried Flowframes, Video2x, and VideoProc Converter AI. I've only managed to make minor improvements to the least damaged ones.

TIA, hopefully someone here can point me in the right direction to possibly save these videos.

Maybe try the Topaz Video AI demo with repeated frame detection, but don't expect miracles. Too much movement in the video.

[GaiaWanderer]

Can you share a sample of the source? Downloading a reencode from youtube doesn't make sense to me.

I hadn't thought of that.

Previous link didn't work. Trying again to upload the original:

https://drive.google.com/file/d/1-iT2gO5gj3loS95tDZ7KL7GVTxvNHiD8/view?usp=sharing

8:17AM EST / Dec 3, 2023: I just tested the link, got a message that it's still processing.

Appreciating everyone's patience while I figure this stuff out!

[Alwyn]

The must be an issue with this board and file.io because user "Portalscreensaver" tried to host some files on file.io but we couldn't download them with a message "The transfer you requested has been deleted." Can you try Google Drive or Dropox?

[Selur]

Same here:
"Deleted

The transfer you requested has been deleted."

[GaiaWanderer]

The must be an issue with this board and file.io because user "Portalscreensaver" tried to host some files on file.io but we couldn't download them with a message "The transfer you requested has been deleted." Can you try Google Drive or Dropox?

Just replaced it with a Google Drive link. Apologies for the difficulties!

[GaiaWanderer]

Same here:
"Deleted

The transfer you requested has been deleted."

Just replaced it with a Google Drive link. Apologies for the difficulties!

[Alwyn]

File's available now.

I'm sure the AVISynth experts will be able to improve it.

[Selur]

Don't have much hope for this.
a. there are tons of duplicates
b. there are jumps
FillDrops will help somewhat, but due to the number of consecutive duplicates, the interpolation in FillDrops will definitely create visible artifacts.

Cu Selur

[GaiaWanderer]

Don't have much hope for this.
a. there are tons of duplicates
b. there are jumps
FillDrops will help somewhat, but due to the number of consecutive duplicates, the interpolation in FillDrops will definitely create visible artifacts.

Cu Selur

Thanks, I'll give it a try.

[Selur]

I used FillDuplicateFrames which is a variant of FillDrops for Vapoursynth which replaces multiple consecutive frames through interpolation.
Even by replacing duplicate frames with interpolations, the output isn't really good. Increasing the frame rate and interpolating even more might make it somewhat smoother, but there will be even more artifacts.

Cu Selur

[rgr]

With Topaz. It helped a little, but with so many jumps...

[rgr]

I used FillDuplicateFrames which is a variant of FillDrops for Vapoursynth which replaces multiple consecutive frames through interpolation.
Even by replacing duplicate frames with interpolations, the output isn't really good. Increasing the frame rate and interpolating even more might make it somewhat smoother, but there will be even more artifacts.

Cu Selur

Where can I find this script (FillDuplicate Frames) -- any home page, forum?

[Selur]

https://github.com/Selur/VapoursynthScriptsInHybrid/ has FillDuplicateFrames.py

Cu Selur

[rgr]

https://github.com/Selur/VapoursynthScriptsInHybrid/ has FillDuplicateFrames.py

Cu Selur

Yes, I found that and nothing else
And I wanted to read how it works, because it would be useful to me.
This uses RIFE for interpolation? The effect is nice.
Only for Vapoursynth?

[Selur]

Yes, that is a script I wrote for Vapoursynth, but it should be easy to port it to Avisynth.
Whether RIFE is used or not depends on the used parameters, when rife is used it will try to use https://github.com/styler00dollar/VapourSynth-RIFE-ncnn-Vulkan

Cu Selur

[_Al_]

https://github.com/Selur/VapoursynthScriptsInHybrid/ has FillDuplicateFrames.py

And I wanted to read how it works, because it would be useful to me.

Usage is in that script, your script could be, something like below, where FillDuplicateFrames.py is in script directory, site-packages or directory it is in is updated in sys.path:

import vapoursynth as vs
from vapoursynth import core
from FillDuplicateFrames import FillDuplicateFrames
source = "some_video.xxx"
clip = core.lsmas.LibavSMASHSource(source)  #or some other source plugin
fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='RIFE', rifeSceneThr=0.15)
fdf.out.set_output()

edit: I did not check that video properly, just seen it when thread started, not sure if some prep work needs to be done for that loaded clip or not sure what Selur exactly did.

[Selur]

iirc. I didn't do any additional filtering for that encode (wasn't really worth the effort from my perspective, which is why I didn't post the used script).
What FillDuplicateFrames does is it checks whether a frame differs more than thresh from its neighbor, if not it's a duplicate. As long as there are duplicates, they get collected, once a non-duplicate appears the pack of [start end]-duplicate frames get replaced with the selected method by interpolating using the adjacent non-duplicate frames. The method itself is rather straight forward, but works. (only tricky part is to collect the frames and interpolate)

[_Al_]

That script might throw off a bit a non python users using that property for output. That RIFE interpolation will be exposed to everyone now. Maybe using code consensus: clip = func(clip, arguments here) would be more friendly.
To the bottom of FillDuplicateFrames.py to add, just an idea:

def interpolate(clip, debug=True, thresh=0.001, method='RIFE', rifeSceneThr=0.15):
    fdf = FillDuplicateFrames(clip, debug=debug, thresh=thresh, method=method, rifeSceneThr=rifeSceneThr)
    return fdf.out

then using in script:

from FillDuplicateFrames import interpolate
.
.
clip = core.lsmas.LibavSMASHSource(source)  #or some other source plugin
clip = interpolate(clip, method='RIFE')
clip.set_output()