Jumping-Hesitating Video

[Alwyn]

I got this off Facebook. Thanks to your guidance, I know how to fix the pause by decimating, but how could the jump be fixed?

The pattern seems to be 8 frames to the jump, then 11 to the pause, and repeat.

Also, how could this have been created? Would it be a Facebook encoding issue or would it have been in the original file (which I assume is from a phone)?

Thanks for any input.

[Alwyn]

The jumps are between frames:

7-8
25-26
42-43
60-61
77-78
and so on.

[lollo]

but how could the jump be fixed?

AviSynth RIFE or MVtools2 or SVPflow

[Selur]

https://forum.videohelp.com/threads/407293-ReplaceFrameX-InsertFrameX might be interesting to insert frames.

[realspeed]

surely If you made the video in the first place you could do it in an edting suit. there will be a copywrite on the video if it is not one of yours

[Alwyn]

you could do it in an edting suit

It is obvious from my post that I didn't make it.

Out of interest, how would you do it in an editing suite?

there will be a copywrite on the video if it is not one of yours

You know where I got it from. What are the copyright rules on that?

I suppose you are also posting your copyright concerns on the dozens of topics here on VH which are actively facilitating breaking of encrypted files?

@Lollo and @Selur, thanks, I'll try!

[realspeed]

Alwyn
DOn't get so up tight on a good piece of advice. Copywrite is owned by the person who made the video. When I had my company, before retirement, my manageress actually sued someone for exaxtly the same thing. It cost the person several hundreds of pounds in compensation. Everyone on here should be aware before stealing others art work.

[Alwyn]

I suggest you stop flying off the handle. You have no idea what I am doing with this bit of (very uninteresting) video.

Now, back on topic, how would you do this in your editor?

[Selur]

for a blind approach: RIFE(4.6 model)+SRestore(30) might produce smooth content

[Alwyn]

Thanks Selur, I'm going to have a go at Staxrip initially, from this topic:

https://forum.videohelp.com/threads/402416-repair-Videos-with-duplicates-and-dropped-frames/page2

[jagabo]

The jumps are between frames:

7-8
25-26
42-43
60-61
77-78
and so on.

There are also duplicates at semi-constant intervals:

0,1
19,20
38,39
57,58
76,77
95, 96
114,115
133,134
151,152

So you'll need to remove the duplicates as well as interpolate the missing frames.

[Selur]

Did a quick 'blind' encode with FillDrops+RIFE+sRestore in Vapoursynth just to see with what one would end up,...

[jagabo]

Manually with DeleteFrames() and RIFE(), original frames on left, fixed on right:

[poisondeathray]

What is the fast moving object, right to left, near the top of the frame, above the plane around 13-14 seconds. Is it a bird, or a plane, or...

[Alwyn]

Thanks Selur, still a bit of uneven movement.

Jagabo, I'm trying to get RIFE running but have got this failure:


[Attachment 72788 - Click to enlarge]

What does the 3668 refer to, because I'm already running A+ 3.7.2 (although I do see 3.7.3 is now been released).

PDR, one of their late mates, I suspect! They are really "rifeing" along, judging by the first frame in which they appear.

[poisondeathray]

r3668 is the version number

If you preview a 1 line script

version()

It will tell you the version you are using. 3.7.2 is too old

[poisondeathray]

The size of the object in comparison to the plane would suggest it was farther away. But it's speed is much faster than the visible plane

[Alwyn]

It will tell you the version you are using. 3.7.2 is too old

Geez, you have to be quick; 3.7.3 has only been out for 2 weeks. I'll install it now. It's a bit confusing when there's parallel "versions" and "releases".

[Alwyn]

Before I do update, is 3.7.3 going to blow up the 3.7.2 installation of my plugins folders?

[poisondeathray]

It will tell you the version you are using. 3.7.2 is too old

Geez, you have to be quick; 3.7.3 has only been out for 2 weeks. I'll install it now. It's a bit confusing when there's parallel "versions" and "releases".

No, there are versions in beta testing for months

https://forum.doom9.org/showthread.php?p=1909698#post1909698

[poisondeathray]

Before I do update, is 3.7.3 going to blow up the 3.7.2 installation of my plugins folders?

Not the way beta testers update. You just swap the avisynth.dll . All the registry information, plugin folders remain intact , same as your current install

There is a "files only" download on the official 3.7.3 release, that contains the avisynth.dll

But some people don't feel comfortable fiddling with windows system folders. Or if it's a 1st time installl / new system, you need the installer to setup the registry

[jagabo]

Update to 3.7.3. I have:


[Attachment 72789 - Click to enlarge]

Oops, too late.

[jagabo]

The script I used:

function ReplaceFramesRIFE(clip Source, int N, int "X")
{
  # N is number of the 1st frame in Source that needs replacing. 
  # X is total number of frames to replace
  # e.g. ReplaceFramesRIFE(101, 5) would replace 101,102,103,104,105 , by using 100 and 106 as reference points for interpolation
  #
  # requires YV12, YV16, or YV24 input, outputs same as input

  X = default(x, 1)

  Trim(Source, N-1, 1)+Trim(Source, N+X, 1)
  z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
  Rife(gpu_thread=1, model=9, factor_num=X+1, factor_den=1, sc=false)
  z_ConvertFormat(pixel_type=Source.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l") 

  AssumeFPS(Source.FrameRateNumerator, Source.FrameRateDenominator)
  Trim(1,X)
Subtitle("RIFE")

  Source.trim(0,N-1) ++ last ++ Source.trim(N+X,0)
}

function InsertFramesRIFE(clip Source, int N, int "X")
{
  # N is number of the 1st frame in Source that needs replacing. 
  # X is total number of frames to insert
  # e.g. ReplaceFramesRIFE(101, 5) would insert 5 frames, by using 100 and 101 as reference points for interpolation.
  #
  # requires YV12, YV16, or YV24 input, outputs same as input

  X = default(x, 1)

  Trim(Source, N-1, 1)+Trim(Source, N, 1)
  z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
  Rife(gpu_thread=1, model=9, factor_num=X+1, factor_den=1, sc=false)
  z_ConvertFormat(pixel_type=Source.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l") 

  AssumeFPS(Source.FrameRateNumerator, Source.FrameRateDenominator)
  Trim(1,X)
Subtitle("RIFE")

  Source.trim(0,N-1) ++ last ++ Source.trim(N,0)
}


# dups:  1 20 39 58 77  96 115 134  152 171 190 
# jumps: 9 28 46 65 83 102 120 139? 158 176 195


LWLibavVideoSource("F35.mp4", cache=false, prefer_hw=2) 
Trim(0, 133)
Crop(320,0,-320,-0)
src = last

DeleteFrame(0)
InsertFramesRIFE(8)
DeleteFrame(20)
InsertFramesRIFE(27)
DeleteFrame(39)
InsertFramesRIFE(45)
DeleteFrame(58)
InsertFramesRIFE(64)
DeleteFrame(77)
InsertFramesRIFE(82)
DeleteFrame(96)
InsertFramesRIFE(101)
DeleteFrame(115)
InsertFramesRIFE(119)

StackHorizontal(src, last)

[Alwyn]

No, there are versions in beta testing for months

I'm not in the habit of just willy-niily installing every beta I find.

Thanks for the tip on avisynth.dll. Yet another quirk in the AVISynth quagmire.

Can you confirm the avisynth.dll from the 3.7.3+ package goes into System32, not syswow64.

Answer, I think so. My AVISynth 3.7.3 is now at r4003!

Thanks for the script, Jagabo, it's looking like it's above my paygrade.

[jagabo]

Can you confirm the avisynth.dll from the 3.7.3+ package goes into System32, not syswow64?

64 bit dll goes in system32, 32 bit dll goes in syswow64.

[Alwyn]

Done, I'm now at r4003.

[jagabo]

Thanks for the script, Jagabo, it's looking like it's above my paygrade.

It's not difficult, just tedious for a long video. First I went over the first 133 frames and noted all the duplicate frames and missing frames. Then I used DeleteFrames() to delete a duplicate and InsertFramesRIFE to insert a missing frame. Then repeated that for each dup/missing pair.

I stopped at frame 133 because there was an odd out-of-place frame soon after that. And this was just a proof of concept.

It may also be possible to more fully automate the process.

[Alwyn]

Cheers Jagabo.

[poisondeathray]

How long is the full video ?

I played with the "automatic" scripts for dupes/drops

John Meyer's dupes and drops script or smoothskip. John's script is more versatile and configurable, and you can modify it to use RIFE for better quality interpolation. Normally it's very good for typical drops and compensatory inserted duplicates

However this is not a "normal" case:

1) The camera power zooms are really large jumps , and the first one has a reverse frame - they will be registered as false positive "drops" (large jumps in motion), then you'd miss the real missing frames nearby . You'd have to exclude those frames, but that changes the cycle length and skews the calculation

2) One section has about 90 consec. frames without dupes or drops, so that messes up the cycle length too - that will force dropping some good frames, becoming jerky . The regular pattern was about 1 dupe every 19


Tip : If you go frame by frame deleting frames, start at the end and work in reverse, otherwise you change the frame numbers that you noted earlier

[Alwyn]

How long is the full video ?

17 seconds. The clip I posted was the clip in toto.