repair Videos with duplicates and dropped frames

[micha.el]

Hi
I have a lot of older Videos with duplicates and dropped frames. They are re-encoded and I don't have the original files anymore.
On my PC Monitor its fine but when I watch them on my TV the motion is sometimes very jerky.
I searched the Internet and found these threads:

https://forum.videohelp.com/threads/386186-Fix-duplicate-frames-on-old-digicam-video
https://forum.doom9.org/showthread.php?t=161758
https://forum.doom9.org/showthread.php?t=174104

and I think that one of these scrpts

https://forum.doom9.org/showthread.php?p=1510813#post1510813
https://forum.doom9.org/showthread.php?p=1789584#post1789584

could help me fix the problem.

But I don't really understand the scripts, or scripts in general, and tried copying and running in Staxrip.
That only gave me a lot of error messages.
So I'm hoping someone could help me make a script that works with Staxrip or fix this problem another way .

I have attached a 12 seconds clip that shows the problem.

Thanks

[pm-s]

tdecimate can help

[micha.el]

As far as I read, tdecimate is only intended to remove duplicates.
The duplicates aren't the real Problem, they are the byproduct from the drops I think.
The drops make the Video jerky.
It's like instead of frames 1 2 3 4 5 6 7 8 9 the video has 1 2 2 3 5 6 6 7 9...
So just deleting the duplicates wouldn't make it smooth.
The scripts I mentioned seem to be able to find the drops and "re-create" the missing frames via interpolation.

I guess I don't have a clue about scripting etc.
Until now I have only used programs like avidemux, handbrake or staxrip.
Staxrip has a lot of default filters that I understand, but I'm not sure how I could get tdecimate to work.

[pm-s]

Tdecimate works on avisynth. You render those avisynth scripts with virtualdub2.

[johnmeyer]

If you have the situation described in the thread I created and to which you linked in the first post, then TDecimate is the wrong tool because it does nothing to fix the drops. Unfortunately, when you have both dups and drops, the dups are easy to remove and fix, but the drops are very difficult. That script I created does work reasonably well, although if someone wants to take a crack and improving the drop detection (which looks for unusually large temporal differences), it could be made to work almost perfectly.

[jagabo]

TDecimate will not help that video. User johnmeyer has a filter called filldrops that can fix capture issues where a duplicate is inserted for a dropped frame -- what you have. The duplicate is replaced with a motion interpolate frame. It's not perfect (and it only works for single frame drops) but it helps your video a lot. A sample is attached. In this sample I had it mark the replaced frames with "interpolated" so it would be obvious which frames were replaced.

LWLibavVideoSource("test.mkv") 
FillDropsAlt()

[Selur]

FillDropsAlt, is that the same as filldrops:

function filldrops (clip c)
{
  super=MSuper(c,pel=2)
  vfe=manalyse(super,truemotion=true,isb=false,delta=1)
  vbe=manalyse(super,truemotion=true,isb=true,delta=1)
  filldrops = mflowinter(c,super,vbe,vfe,time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
  return fixed
}

from: https://forum.doom9.org/showthread.php?p=1775184#post1775184
or is that another script?

Cu Selur

[johnmeyer]

Here is the script that I created and posted over in doom9.org in 2011. It will handle captures where the drop-dup pattern is not consistent. This is important because many of these situations arise when capturing streaming video over a poor bandwidth connection, and also from faulty capture cards. In both cases the drops, usually followed a few frames later by a dup, often don't happen for awhile and then come in "waves." The beauty of this script is that it will leave your video alone until it starts finding dups and drops.

# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
#
# Create interpolated frames a 2x original frame rate using MVTools2
# Detect jumps 
# Create white mask at each jump point; black mask for all other frames
# Repeat each frame of original video and use mask to "choose" between original video, or motion estimated video
# Decimate exactly 50% to get back to original frame rate. 
# This decimation removes the dup frames from the original video and also the dups created by repeating each frame of original video
# However, at each point where motion-estimated frame was inserted, no decimation occurs. Thus, if dups=drops, and the drop happens 
# within < "cycle" (TDecimate parameter) of the dup, the dup will be removed and the drop will be filled. 
# If no drops or dups occur within "cycle," then no motion estimation happens, and decimation merely gets back to original, 
# unchanged video.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.8
showdot = false # true for troubleshooting; otherwise, false

#SetMTMode(5,4)
global source=AVISource( "E:\CIF Track\3200-Complete (edited).avi" ).ConvertToYV12
#SetMTMode(2)

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2) 
bvec  = MAnalyse(super, overlap=4, isb = true, search=4, dct=5) 
fvec  = MAnalyse(super, overlap=4, isb = false, search=4, dct=5) 
double = source.MFlowFps(super, bvec, fvec, num=60, den=1, blend=false)

#Remove comment from ShowMetrics, and change "return final" to "return test" to look at metrics in order to determine proper JumpThresh
#test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)

interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate
RequestLinear
final=tdecimate(display=false,mode=1,cycleR=10,cycle=20)  # Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)

#---------------------
#Alternate two-pass approach to decimation 

#Pass 1
#RequestLinear(debug=false)
#final=tdecimate(display=false,mode=4,output="e:\metrics.txt")

#Pass 2 (remember to un-comment "requestlinear")
#RequestLinear
#final=tdecimate(display=false,mode=2,rate=30,input="e:\metrics.txt",maxndl=20)
#---------------------

return final
#return stackvertical(source,final)

#----------------
#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.2 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01)
    \ ))")
  return fixed
}



#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.2 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.2 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh 
    \ ? WhiteFrame : BlackFrame """)
  return MyMask
}

[jagabo]

FillDropsAlt, is that the same as filldrops:

function filldrops (clip c)
{
  super=MSuper(c,pel=2)
  vfe=manalyse(super,truemotion=true,isb=false,delta=1)
  vbe=manalyse(super,truemotion=true,isb=true,delta=1)
  filldrops = mflowinter(c,super,vbe,vfe,time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
  return fixed
}

from: https://forum.doom9.org/showthread.php?p=1775184#post1775184
or is that another script?

Cu Selur

It looks like my filldropsAlt() is the same as your filldrops() except for the frame difference threshold.

function filldropsAlt (clip c)
{
  super=MSuper(c,pel=2)
  vfe=manalyse(super,truemotion=true,isb=false,delta =1)
  vbe=manalyse(super,truemotion=true,isb=true,delta= 1)
  filldrops = mflowinter(c,super,vbe,vfe,time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.5")
  return fixed
}

I have an older version of filldrops that uses mvtools rather than mvtools2. So I labeled the newer one Alt.

[Selur]

@jabago: okay, that's basically the version I posted with another threshold (instead 0.1, 0.5 is used).

[jagabo]

@jabago: okay, that's basically the version I posted with another threshold (instead 0.1, 0.5 is used).

Yes. I had to set the threshold to 3.0 to get all the dup frames in the OP's video (compression artifacts). But then it replaced many unique frames.

[Selur]

btw. here is a modified version which takes 'thresh' as additional parameter:

function filldrops(clip c, float "thresh")
{
  thresh = default(thresh, 0.1)
  
  super=MSuper(c,pel=2)
  vfe=MAnalyse(super,truemotion=true,isb=false,delta =1)
  vbe=MAnalyse(super,truemotion=true,isb=true,delta= 1)
  filldrops = MFlowInter(c,super,vbe,vfe,time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", String(thresh))
  return fixed
}

(I hate it to modify parameters inside functions. )

Cu Selur

[JKyle]

@micha.el,

See here to try ExactDedup as a custom filter in StaxRip.

@Selur and @jagabo,

Thanks for the script.
I'm thinking of incorporating FillDropsAlt (Selur's mod) in StaxRip.
Plz tell me if it's OK with you guys.

[Selur]

no complain from me,..

[JKyle]

Take the following steps to use FillDropsAlt (Selur's mod) as a custom filter in StaxRip:

1. Copy the following script code (a slightly modified Selur's mod) and save it as FillDropsAlt.avsi (text encoding: UTF-8).

function FillDropsAlt(clip c, float "thresh")
{
  thresh = Default(thresh, 0.1)

  super = MSuper(c, pel=2)
  vfe = MAnalyse(super, truemotion=true, isb=false, delta=1)
  vbe = MAnalyse(super, truemotion=true, isb=true, delta=1)
  filldrops = MFlowInter(c, super, vbe, vfe, time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", String(thresh))
  return fixed
}

2. Click on Tools > Folders > Settings in StaxRip, and in Explorer, browse down to Plugins > AviSynth, create a folder named Scripts and put FillDropsAlt.avsi in Scripts folder.

3. In StaxRip main window, click on AVS Filters > Profiles.... In the Edit pane, put the following FillDropsAlt filter profile under the [Restoration] category. (Alphabetical order recommended.)

DeDup | FillDropsAlt =
    LoadPlugin("%app:mvtools2%")
    Import("%settings_dir%Plugins\AviSynth\Scripts\FillDropsAlt.avsi")
    FillDropsAlt(thresh=$enter_text:Enter the Luma Difference Threshold in decimal (Default: 0.1)$)

(There need to be empty lines above and below.)

4. You can apply FillDropsAlt in the AVS filter menu Restoration > DeDup > FillDropsAlt after loading a file.

[micha.el]

Big thanks to JKyle, Selur and jagabo
Especially for the foolproof guide to set up and use ExactDedup and FillDropsAlt as a custom filter in Staxrip.
After some converting it seems that FillDrops does really help in smoothing the videos.

What exactly does the "Luma Difference Threshold" mean?
What would be achieved by changing it from the default?

I also tried the script from johnmeyer in Post 8 to see how it works with my videos, but I don't understand most lines in the script.
I changed the 'loadplugin' and 'global source' lines but all I got was a message with a lot of errors.
Not sure if there is a way to make this script work with Staxrip. I would appreciate if someone could help me, if it's possible.
I attached the error message if that helps

And again thanks a lot for all the help

[jagabo]

What exactly does the "Luma Difference Threshold" mean?

In my post and selur's it's the number at the end of this line:

fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.5")

What that line does is select one of two videos, filldrops or the second c, depending on the analysis of the first video, c. The analysis is specfied by the "YDifferenceFromPrevious()", "lessthan", "0.5" arguments. In this case the current frame is compared to the previous frame. If the average absolute difference in brightness of corresponding pixels is less than 0.5 (Y units) then filldrops is used, otherwise c (ie, that frame isn't changed). Exact duplicate frames would have a average difference of 0. But since the video has compression artifacts duplicate frames aren't exact duplicates. So you need a threshold value to decide when the differences are large enough not to be considered a duplicate. If you used this version of the function you would just change the 0.5 to some other value.

In JKyle's variation he has turned that threshold value into a variable that specified when you call the function:

fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", String(thresh))

If you specify a value that value is used. If you don't specify a value it uses a default of 0.1:

thresh = Default(thresh, 0.1)

You then call the function using whatever threshold you want:

FillDrops(alt, 0.5) # threshold = 0.5

The benefit of doing it this way is that you can have one defined filldropsAlt() function that doesn't have to be changed from script to script, you just call it with different values as necessary.

What would be achieved by changing it from the default?

As noted earlier, it lets you specify how different a frame has to be (from the previous frame) before it's considered not to be a duplicate, and therefore not replaced with a motion interpolated frame. If your threshold is too small some duplicate will not be replaced. If it's too large many frames which aren't duplicates (they just have very small motions) will be replaced.

[micha.el]

thanks jagabo

Another question
I've got a Video that has only dropped frames (jumps). It is 25 fps and I think it was originally 29.97 fps and the drops are probably because of the frame reduction.
The jump seems to be every 5th frame, but sometimes there is no (visible) jump.
And the jump seems to be shifting. In the VirtualDub timeline the jump for example is at frames 113,118,123,128... and later it is at 214,219,224...

Is there a filter that is able to find these jumps and create (a) interpolated frame(s)?

FillDrops didn't really help here since there are no duplicates.

I found this: https://forum.videohelp.com/threads/352741-Frame-interpolation#post2222196
but since the jumps in my video are irregular I don't think that would work.

Or maybeI first convert to 29.97 and then try FillDrops?

[jagabo]

I've got a Video that has only dropped frames (jumps). It is 25 fps and I think it was originally 29.97 fps and the drops are probably because of the frame reduction.
The jump seems to be every 5th frame, but sometimes there is no (visible) jump.
And the jump seems to be shifting. In the VirtualDub timeline the jump for example is at frames 113,118,123,128... and later it is at 214,219,224...

Is there a filter that is able to find these jumps and create (a) interpolated frame(s)?

I've never worked out a fully automated way of doing that.

There's a semi-automated ways to approach it: First make five clips where each clip has an interpolated frame added at a different position. Then manually go through the videos and use ReplaceFramesSimple() to select the correct clip for each segment. For short videos that's not too much work. But if there's hundreds of pattern changes over a two hour movie it's quite tedious. I've done something like this a few times myself.

http://avisynth.nl/index.php/RemapFrames

A possible way to fully automate it would be to look for frames where the difference between successive frames is significantly larger than the difference of the frames around it. Ie, a bigger motion will usually result in a bigger luma difference. Here are five consecutive frames with the YDifferenceFromPrevious is printed on each frame (from a video with this exact problem):


[Attachment 59812 - Click to enlarge]

The middle frame has a larger motion than the other four. And it's YDifferenceFromPrevious is correspondingly higher: 11.9 vs. an average of 7.6 for other four. So a motion interpolated frame would be inserted there. I have a vague notion of how to do this within the constructs of AviSynth but have never worked out all the details. And it probably won't be perfect.

[micha.el]

@jagabo

The semi-automated way seems to complicated for me, and yes the video is about 124 min long.
The other way sounds interesting.

Is there a simple way to write the YDifferenceFromPrevious on each frame so I could look if there is a pattern in my video?

[jagabo]

Is there a simple way to write the YDifferenceFromPrevious on each frame so I could look if there is a pattern in my video?

Something like:

LSmashVideoSource("video.mp4") 
ScriptClip(last, "Subtitle(String(YDifferenceFromPrevious))")

[johnmeyer]

I just tried out my old script and, after a little tuning, it seemed to do a pretty good job. Here is the resulting clip:

Drops and Jumps Removed

Here is the script I used. If I was doing this for real, I'd spend more time tuning the parameters because it fails to do the right thing when there is almost no movement (e.g., at the beginning of the clip before the ball is hiked).

# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
# Further modification on June 8, 2013
#
# Create interpolated frames a 2x original frame rate using MVTools2
# Detect jumps 
# Create white mask at each jump point; black mask for all other frames
# Repeat each frame of original video and use mask to "choose" between original video, or motion estimated video
# Decimate exactly 50% to get back to original frame rate. 
# This decimation removes the dup frames from the original video and also the dups created by repeating each frame of original video
# However, at each point where motion-estimated frame was inserted, no decimation occurs. Thus, if dups=drops, and the drop happens 
# within < "cycle" (TDecimate parameter) of the dup, the dup will be removed and the drop will be filled. 
# If no drops or dups occur within "cycle," then no motion estimation happens, and decimation merely gets back to original, 
# unchanged video.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-26.dll") 

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.88
showdot = false # true for troubleshooting; otherwise, false

global source=AVISource( "E:\fs.avi" ).ConvertToYV12.killaudio()
global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2) 
bvec  = MAnalyse(super, overlap=0, blksize=16, isb = true, search=4, dct=0) 
fvec  = MAnalyse(super, overlap=0, blksize=16, isb = false, search=4, dct=0) 
double = source.MFlowFps(super, bvec, fvec, num=120000, den=1001, blend=false)

#Remove comment from ShowMetrics, and change "return final" to "return test" to look at metrics in order to determine proper JumpThresh
#test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)
interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate
RequestLinear
decimated = tdecimate(display=false,mode=1,cycleR=10,cycle=20)  # Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)

# some repeats are still left after decimation, so fix those as well
final = filldrops(decimated)
return final

#return test
#return stackhorizontal(source,final)

#----------------
#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01)
    \ ))")
  return fixed
}



#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh 
    \ ? WhiteFrame : BlackFrame """)
  return MyMask
}


function filldrops (clip c)
{
  super=MSuper(c,pel=2)
  vfe=manalyse(super,truemotion=true,isb=false,delta=1)
  vbe=manalyse(super,truemotion=true,isb=true,delta=1)
  filldrops = mflowinter(c,super,vbe,vfe,time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.4")
  return fixed
}

[JKyle]

I transformed @jonnmeyer's script into a filter function named DeJump so that it can be used for more general cases.
(Forgive me my poor naming skill. )

As explained in the script, my modifications are as follows:

• parametrized input clip, jumpthresh, and thresh (for FillDrops)
• parametrized num and den options for double via MFlowFps to take input clip properties
• removed ShowMetrics function needed for test
• removed FillDrops function to make use of the externalized function script FillDrops.avsi (mod by Selur)
• unglobalized and internalized some variables to be used in GenerateMask: jumpthresh, blackframe, whiteframe

The script is as follows and you can download it from here in my repository as well.

#########################################################################################
#
# DeJump.avsi
#
# mod 2021-07-12 by JKyle
#
# An AviSynth filter that smooths out jumps and drops
#
# - Based on script created by Didée
# - Modified by John Meyer on June 29, 2011
# - Further modification by John Meyer on June 8, 2013
# https://forum.videohelp.com/threads/402416-repair-Videos-with-duplicates-and-dropped-frames#post2625132
#
# - Transformed into DeJump filter function by JKyle on July 12, 2021
#    - parametrized input clip, `jumpthresh`, and `thresh` (for `FillDrops`)
#    - parametrized `num` and `den` options for `double` via `MFlowFps` to take input clip properties
#    - removed `ShowMetrics` function needed for test
#    - removed `FillDrops` function to make use of the externalized function script `FillDrops.avsi` (mod by Selur)
#    - unglobalized and internalized some variables to be used in `GenerateMask`: `jumpthresh`, `blackframe`, `whiteframe`
#
### Requirements ###
#-------------------
# MvTools2 (https://github.com/pinterf/mvtools)
# MaskTools2 (https://github.com/pinterf/masktools)
# TIVTC (https://github.com/pinterf/TIVTC)
# FillDrops mod 2021-07-10 by Selur (https://github.com/JJKylee/Filter-Scripts/blob/main/AviSynth/FillDrops.avsi)
#
### Arguments ###
#-----------------
# jumpthresh (default=0.88) - Threshold for detecting jumps
#    Increase to catch more jumps.
#    Should always be less than 1.0
#
# thresh (default=0.4) - Luma Difference Threshold for FillDrops
#
# showdot (default=false) - Troubleshooting
#    true for troubleshooting; otherwise, false
#
### How this filter works ###
#----------------------------
# Create interpolated frames a 2x original frame rate using MVTools2
# Detect jumps 
# Create white mask at each jump point; black mask for all other frames
# Repeat each frame of original video and use mask to "choose" between original video, or motion estimated video
# Decimate exactly 50% to get back to original frame rate. 
# This decimation removes the dup frames from the original video and also the dups created by repeating each frame of original video
# However, at each point where motion-estimated frame was inserted, no decimation occurs. Thus, if dups=drops, and the drop happens 
# within < "cycle" (TDecimate parameter) of the dup, the dup will be removed and the drop will be filled. 
# If no drops or dups occur within "cycle," then no motion estimation happens, and decimation merely gets back to original, 
# unchanged video.
#
#########################################################################################


function DeJump (clip c, float "jumpthresh", float "thresh", bool "showdot")
{

    jumpthresh  = Default(jumpthresh, 0.88)
    thresh      = Default(thresh, 0.4)
    showdot     = Default(showdot, false)

    source = c.ConvertToYV12.KillAudio()
    newnum = c.FrameRateNumerator * 2 # Added for double (JKyle)
    newden = c.FrameRateDenominator # Added for double (JKyle)
    blackframe = BlankClip(source, color=$000000)
    # removed internalized `whiteframe` (JKyle)

    super = showdot ? source.Subtitle("***").MSuper(pel=2) : source.MSuper(pel=2)
    bvec  = MAnalyse(super, overlap=0, blksize=16, isb=true, search=4, dct=0)
    fvec  = MAnalyse(super, overlap=0, blksize=16, isb=false, search=4, dct=0)
    double = source.MFlowFps(super, bvec, fvec, num=newnum, den=newden, blend=false)

    #Generate a white or black frame, depending on frame difference
    bwmask = GenerateMask(source, jumpthresh)

    #Generate the 2x framerate mask needed to choose the motion-estimated frames
    themask = Interleave(blackframe, Trim(bwmask, 1, 0))

    #Merge double framerate from original with motion-esimated frames, but only where there are jumps
    #(i.e., original frames are used except at jump points)
    Interleave(source, source).mt_merge(double, themask, luma=true, U=3, V=3)

    #Decimate
    RequestLinear
    decimated = TDecimate(display=false, mode=1, cycleR=10, cycle=20)  #Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)

    # some repeats are still left after decimation, so fix those as well
    final = FillDrops(decimated, thresh)
    return final

}


#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

# `jumpthresh` argument added for internal use (JKyle)

function GenerateMask (clip c, float jumpthresh)
{

    blackframe = BlankClip(c, color=$000000) # internalized for GenerateMask (JKyle)
    whiteframe = BlankClip(c, color=$FFFFFF) # internalized for GenerateMask (JKyle)
    mymask = c.ScriptClip("""
    \ (( (YDifferenceFromPrevious(SelectEvery(c, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(SelectEvery(c, 1, 3))  :
    \       YDifferenceFromPrevious(SelectEvery(c, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(SelectEvery(c, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(SelectEvery(c, 1, 2))  :
    \     YDifferenceFromPrevious(SelectEvery(c, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(SelectEvery(c, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(SelectEvery(c, 1, -2))  :
    \     YDifferenceFromPrevious(SelectEvery(c, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(SelectEvery(c, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(SelectEvery(c, 1, -3))  :
    \     YDifferenceFromPrevious(SelectEvery(c, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(c) + 0.01) <= jumpthresh 
    \ ? whiteframe : blackframe """)
    return mymask

}

And FillDrops.avsi (Selur's mod) is also available from here in my repository.
(I renamed the filter back to FillDrops from FillDropsAlt for a maintenance purpose.)

Again its code is as follows:

#########################################################################################
#
# FillDrops.avsi
#
# An AviSynth filter to detect and replace duplicate frames
# with a motion-interpolation of the adjacent frames.
#
# mod 2021-07-10 by Selur
# 
# - based on Mug Funky's FillDrops
# - FillDrops by johnmeyer on Doom9's forum
# https://forum.doom9.org/showthread.php?p=1775184#post1775184
# - mod by Selur to parametrize Luma Difference Threshold
# https://forum.videohelp.com/threads/402416-repair-Videos-with-duplicates-and-dropped-frames#post2624936
#
### Requirements ###
#-------------------
# MvTools2 (https://github.com/pinterf/mvtools)
#
### Arguments ###
#-----------------
# thresh (default=0.1) - Luma Difference Threshold
#    Lets you specify how different a frame has to be (from the previous frame)
#    before it's considered not to be a duplicate, and therefore not replaced with a motion interpolated frame.
#    If the threshold is too small, some duplicate will not be replaced.
#    If it's too large, many frames which aren't duplicates (just having very small motions) will be replaced.
#    (explanation by jagabo on VideoHelp)
#
### Changelog ###
#---------------
# Changed requirements: from MvTools to MvTools2 (johnmeyer)
# Parametrized Luma Difference Threshold (Selur)
#
#########################################################################################

function FillDrops(clip c, float "thresh")
{
  thresh = Default(thresh, 0.1)

  super = MSuper(c, pel=2)
  vfe = MAnalyse(super, truemotion=true, isb=false, delta=1)
  vbe = MAnalyse(super, truemotion=true, isb=true, delta=1)
  filldrops = MFlowInter(c, super, vbe, vfe, time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", String(thresh))
  return fixed
}

In order to use DeJump in StaxRip, just put DeJump.avsi and FillDrops.avsi in <StaxRip-Folder>\Settings\Plugins\AviSynth\Scripts and put the following DeJump filter profile in AVS Filters... > Profiles in the main window under the [Restoration] category.

DeJump =
    LoadPlugin("%app:mvtools2%")
    LoadPlugin("%app:masktools2%")
    LoadPlugin("%app:TIVTC%")
    Import("%settings_dir%Plugins\AviSynth\Scripts\FillDrops.avsi")
    Import("%settings_dir%Plugins\AviSynth\Scripts\DeJump.avsi")
    DeJump(jumpthresh=$enter_text:Enter the Threshold for detecting Jumps in decimal (Default: 0.88)$, thresh=$enter_text:Enter the Luma Difference Threshold for FillDrops in decimal (Default: 0.4)$)

(Empty lines above and below)

I tested and confirmed that it works very well.

[micha.el]

Thanks

@johnmeyer your clip is exactly what I hoped it would be. And at least in my case, if there is no or only slow movement than there are no annoying (visible) jumps.

@JKyle Wow, without me knowing how it would have to look like or how to achieve this, that is absolutely what I wanted when I first found johnmeyer's script.
It would have taken me ages to figure all that out by myself, if I would have been able to... eventually

I'm really grateful to everybody helping in this thread.

[Selur]

@JKyle: nice work.

[JKyle]

Looking further into the source code of DeJump, I've found that we have no reason to give up HBD(high bit depth) support since all the dependencies support HBD.

So I removed ConvertToYV12 to support HBD and instead put a warning on input clip pixel types to avoid RGB or interleaved pixel types like YUY2, etc.

Here's the revised version along with some minor changes.

I've confirmed that it works well with HBD inputs.

[Selur]

Nice catch!

[micha.el]

Is there a way to increase the speed of Staxrip using the DeJump filter?
My PC has only around 30% cpu load.
I tried Set Threats (with a small increase) and Prefetch (didn't help at all).

[Selur]

a. In theory one could adjust FillDrops to use SVP instead of MVTools which might be faster (not sure).

b. changing:

  vfe = MAnalyse(super, truemotion=true, isb=false, delta=1)
  vbe = MAnalyse(super, truemotion=true, isb=true, delta=1)
  filldrops = MFlowInter(c, super, vbe, vfe, time=50)

to

vf=c.mvanalyse(truemotion=true,pel=2,isb=false,delta=1, idx=1)
vb=c.mvanalyse(truemotion=true,pel=2,isb=true,delta=1, idx=1)
global filldrops_d = c.mvflowinter(vb,vf,time=50, idx=1)

should reduce memory usage and gain speed.
Similar should work for DeJump.
I thought about adjusting DeJump & Filldrops to take additional (optional) index parameters (idx1, idx2), but I was too lazy, especially since it is not totally without risk.

In case other MVTools-based operations are running in the same script, eventually use another positive number for idx ... it must be unique for each instance where MVTools are working on a given instance of input.

see https://forum.doom9.org/showthread.php?p=753787#post753787 for more details about filldrops

Cu Selur

Ps.: those improvements will not help with threading.

[johnmeyer]

@JKyle - If you ever revisit your excellent re-write of my script, you might also want to parameterize the Cycle/CycleR numbers. For this script, Cycle must be exactly 2x CycleR. Using larger values for Cycle can help when the drops and dups are widely scattered. However, the larger the values, the slower the script will run.