I have an old home video clip in wmv format from about 2007.
It was recorded on a random old point-n-shoot camera, and according to the metadata was converted to wmv using an ancient copy of Windows MovieMaker 2.1
The file is 320x240, supposedly 15fps.
The problem is that it plays back very jerky, even for 15fps.
If I step through the video frame by frame, I can see that there are a lot of duplicated frames, not in any clear pattern.
For example, the frame sequence could be like this:
1-2-3-4-4-6-7-8-9-10-11-11-11-14-15-16-17-18-19-19-21... etc
What I'd like to do is replace the duplicate frames with new motion-interpolated frames, while keeping the original frame rate intact, and not mangle the other "good" frames.
The mpdecimate filter on ffmpeg does a great job of detecting the duplicated frames.
As a test only, I used the filter "-vf mpdecimate,setpts=N/FRAME_RATE/TB" to output a file with the duplicates dropped completely.
This works but changes the run time, as it just skips over the duplicate frames.
Is there a way to use the minterpolate filter to only interpolate specific frames that mpdecimate identifies?
I'm familiar with ffmpeg and Adobe Premiere, but am open to try other tools to accomplish this.
First time poster here, and any help or advice is appreciated!
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 14 of 14
Thread
-
-
True duplicates can be easily replaced with motion-estimated frames. I've posted my "FillDrops()" and "FillDropsI()" (for interlaced video) over at Doom9.org many times. It works really well.
However, most video that contains duplicate frames -- especially a video that was created in the way you have described your video -- usually includes those duplicates because a frame was dropped a few frames earlier in the video. If nothing is done, this will screw up the sound sync. So, to keep the sound in sync, the capture card adds a duplicate frame. Thus, you may have a more difficult problem than the one you describe, and you'll therefore need to do the much, much tougher task of figuring out where there is a dropped frame. Once you detect that point, you insert a motion estimated frame at that point, rather than at the place where the duplicate happens. Thus, what you probably want to do is:
1. Detect the duplicate and delete it (easy).
2. Detect the dropped frame (difficult, because the software has to look for a larger visual jump than what happened on adjacent frames).
3. Insert a motion estimated frame at the location where you think you've detected a dropped frame (easy).
4. Make sure that you only add frames if there is a duplicate frame near a dropped frame (hard).
The last one is needed so you don't change the length of the video.
I tried to create a script that does this, and posted the result in this thread I started in doom9.org seven years ago:
Automatically fix dups followed (eventually) by drops
This also gives you links to the filldrops() function.Last edited by johnmeyer; 15th Dec 2017 at 13:03. Reason: typo
-
Yes, my filldrops script is based on the really old MugFunky script. He used the original MVTools, and I updated that to MVTools2; adjusted several settings; and adapted it to also work with interlaced video.
-
I'm new to AVISynth and scripts to apologies if I've messed this up.
I installed AviSynth 2.6.0. 32-Bit.
I have latest ffmpeg 32-Bit. (Had 64 originally, but trying to 32-bit to match "bitness" of avisynth.
I installed the mvtools2.dll plugin.
I saved the script as magic.avs, updating it for my source wmv and correct path to the mvtools2.dll plugin.
Running "ffmpeg -i magic.avs" results in this error:
[avisynth @ 074f46c0] Script error: there is no function named "mt_merge"
(magic.avs, line 47)
magic.avs: Unknown error occurred
I tried googling this but didn't find a simple answer and was a bit overwhelmed.
Any ideas? -
Please give the complete AviSynth script and any error message you get when opening the script in Virtual Dub. Which might be the same as the ffmpeg error message.
-
You need masktools2. http://avisynth.nl/index.php/MaskTools2
But, filldrops/filldrops2 don't work right for multiple duplicates in a row. You'll need to call them twice for frames repeated 3 times, or three times for frames repeated 4 times, etc. Even then, the motion won't be quite right.Last edited by jagabo; 15th Dec 2017 at 18:17.
-
If you have LOTS of multiple duplicates in a row, the video probably cannot be rescued.
-
I didn't even notice that - that there might be multiple dupes in a row. But there was only one place in his example where there were 2 dupes in a row, so it still might work reasonably well.
What's the answer then? Pull out all the duplicate frames with multidecimate or dedup followed by interpolating to whatever framerate he wants? Which probably won't work very well either.
But then, a 320x240 15fps WMV source with a bunch of duplicate frames must look like crap to begin with. Especially if, as johnmeyer supposes, each dupe frame means a missing unique frame elsewhere. -
Most of the videos I've seen that have this problem are from streaming sources, or from an older capture card. Both of these often have problems keeping up with the incoming video and, when they can't keep up, they drop a frame. This creates a temporal gap, and with something like a smooth camera pan, these gaps are painfully obvious. Then, to keep the audio from going out of sync, the streaming software or capture app will duplicate a frame. The dupe is created at a later moment in time, but the number of frames between the drop and the duplicate are seldom the same. Thus, unfortunately, you cannot simply look for duplicate frames, something that is relatively easy using one of the YDifference functions, count forward a set number of frames, and interpolate a new frame there.
The script I came up with, that I linked to above, used some clever (well, they seemed that way to me) techniques invented by a guy named Didée over at doom9.org. I adapted his ideas and came up with a function that does what I describe. However, because the drops and dups are often so irregular, my approach required looking at a whole bunch of frames at once. I think it was something like 20-30 frames in a cycle. As a result, the script is pretty slow.
The good thing about the script is that when it makes a mistake and mis-identifies a dropped frame, the motion estimation is good enough that you don't usually notice the spurious addition. -
For example, the frame sequence could be like this:
1-2-3-4-4-6-7-8-9-10-11-11-11-14-15-16-17-18-19-19-21... etc
Another clue is WMM . Those frames are probably duplicates added by WMM from a lower base framerate to make up the new 15fps framerate . In avisynth-ese it would be analgous to changefps(15) from say a 12fps video or some lower fps video
The distinction is important, because drops from a true 15 fps sequence would be in the correct time exactly. If it's like the way he indicated, if you were to interpolate those (ie. the missing 5,12,13,20) , it would be smooth. But a changefps-like manipulation in WMM or video editor would be a lower fps to higher fps conversion by duplicating frames in approx the correct location. Frames are not "evenly spaced" in time. If you were to interpolate those, it would still be choppy . In the 2nd case it would be better to delete duplicates using dedup/multidecimate or some other duplicate removal then reinterpolate / retime completely using something like mvtools2/smoothfps2/interframe/framerateconverter... etc... but with likely ugly artifacting results, but at least the timing would be "evenly spaced"Last edited by poisondeathray; 15th Dec 2017 at 22:51.
-
If you want to go through the video frame by frame and locate the longer strings of duplicate and missing frames you can use ReplaceFramesMC() to replace multiple missing frames with motion interpolated frames (overwriting duplicates that were inserted in their place), and InsertFramesMC() to insert one or more motion interpolated frames.
https://forum.videohelp.com/threads/352741-Frame-interpolation/page2#post2226119
And if you're really ambitious, after generating a clean(ish) 15 fps video, you can double the frame rate to 30 fps with motion interpolation with Interframe2, DoubleFPS2, or some other such filters. Or even 60 fps. -
Hi.
I have some jerky video vs lot of dup frame and some drop one: sorce (3,5mb)
I tried to fix it whis johnmeyer "magic" script but got much more mess: result (3,5mb)
Here is script. Where I was mistaken?
PHP Code:LoadPlugin("C:\soft\[url=https://www.videohelp.com/software/MeGUI]MeGui[/url]\tools\avisynth_plugin\mvtools2.dll")
#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.8
showdot = false # true for troubleshooting; otherwise, false
#SetMTMode(5,4)
LoadPlugin("C:\soft\MeGui\tools\avisynth_plugin\masktools2.dll")
LoadPlugin("C:\soft\MeGui\tools\avisynth_plugin\TIVTC.dll")
LoadPlugin("C:\soft\MeGui\tools\lsmash\LSMASHSource.dll")
global source=LWLibavVideoSource("E:\2004.04.14 (1).mkv").ConvertToYV12
#SetMTMode(2)
global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )
super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2)
bvec = MAnalyse(super, overlap=4, isb = true, search=4, dct=5)
fvec = MAnalyse(super, overlap=4, isb = false, search=4, dct=5)
double = source.MFlowFps(super, bvec, fvec, num=60, den=1, blend=false)
#Remove comment from ShowMetrics, and change "return final" to "return test" to look at metrics in order to determine proper JumpThresh
#test=ShowMetrics(source)
#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)
#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))
#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)
interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)
#Decimate
RequestLinear
final=tdecimate(display=false,mode=1,cycleR=10,cycle=20) # Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)
#---------------------
#Alternate two-pass approach to decimation
#Pass 1
#RequestLinear(debug=false)
#final=tdecimate(display=false,mode=4,output="e:\metrics.txt")
#Pass 2 (remember to un-comment "requestlinear")
#RequestLinear
#final=tdecimate(display=false,mode=2,rate=30,input="e:\metrics.txt",maxndl=20)
#---------------------
return final
#return stackvertical(source,final)
#----------------
#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average
function ShowMetrics (clip c)
{
fixed=source.ScriptClip("Subtitle(String(
\ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) )
\ )/4) /
\ (YDifferenceFromPrevious(source) + 0.01)
\ ))")
return fixed
}
#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average
function GenerateMask (clip c)
{
MyMask=c.ScriptClip("""
\ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.2 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) )
\ )/4) /
\ (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh
\ ? WhiteFrame : BlackFrame """)
return MyMask
}
Similar Threads
-
Remove duplicate frames from 30FPS video
By midts in forum Video ConversionReplies: 7Last Post: 7th Apr 2017, 08:48 -
Getting rid of duplicate frames in a 30p video sourced from 24p
By alexpigment in forum Video ConversionReplies: 5Last Post: 22nd Feb 2016, 10:04 -
Removing duplicate frames
By Colek in forum Video ConversionReplies: 7Last Post: 9th Oct 2015, 12:40 -
Script to detect duplicate frames?
By kieranvyas in forum EditingReplies: 30Last Post: 22nd Aug 2015, 20:01 -
What if source has and duplicate frames and is interlaced and more
By rhaz in forum DVD RippingReplies: 17Last Post: 15th Jun 2014, 06:17