video with alot of horizontal jitter

[spiritgumm]

Have a video on dvdr with alot of horizontal jitter (not always the whole frame, sometimes just parts of it). I tried Depan in Avisynth, and Deshaker in VDubmod, but no luck. Attached sample. Any suggestions?
I believe it is hard-telecined, 29.97 fps.
dungeon.m2v

[poisondeathray]

you can try dejitter plugin and tweak the settings, but you're not going to be able to fix this very easily automatically , short of some manual work (e.g. in photoshop, after effects)

[spiritgumm]

Judging from the before/after pictures on the Dejitter page, I can think of other videos I could use it on, but it didn't help when applied to this video. Thanks anyways.

[lordsmurf]

That can't be fixed. It's just a crap conversion.
Go back to better source.

[spiritgumm]

unfortunately with rare videos it's hard to find good sources, which is why I'm here.

[lordsmurf]

Most videos aren't really that rare. It's just an issue of people not knowing where to look.
What is this? I bet I could find it.

Attached is what I could do in software real quick. There may be more things possible, if I spent some time at it, but it's still never going to be as clean as better source.

[poisondeathray]

If you can't find a better source, one option is to replace the bad frames with nearby dupes or interpolated frames. There are some plugins in avisynth that can do this (one is called "bad frames", there are some others). But if you have long strings of consecutive bad frames, it's not going to be very pretty

[spiritgumm]

If you can't find a better source, one option is to replace the bad frames with nearby dupes or interpolated frames. There are some plugins in avisynth that can do this (one is called "bad frames", there are some others). But if you have long strings of consecutive bad frames, it's not going to be very pretty

I thought about this option, but as you say, it won't be pretty - not that it's pretty now. I might test it, but seems like I'll be trading one jerky video for another.

Lordsmurf, thanks for the sample re-do. I dont know if it's much better than what I got (I deleted my tests already). Did you use some other plugin?
I also tried someone's mock-TBC script but I wasn't able to get it working.
The video is a Playhouse 90 episode called "The Dungeon." The video is bad the first 15 minutes, but settles down to just an occasional jerk once a minute. You might be able to find another copy, but it might be just as bad (and if on dvd, way overpriced).

[poisondeathray]

If you can't find a better source, one option is to replace the bad frames with nearby dupes or interpolated frames. There are some plugins in avisynth that can do this (one is called "bad frames", there are some others). But if you have long strings of consecutive bad frames, it's not going to be very pretty

I thought about this option, but as you say, it won't be pretty - not that it's pretty now. I might test it, but seems like I'll be trading one jerky video for another.

Lordsmurf, thanks for the sample re-do. I dont know if it's much better than what I got (I deleted my tests already). Did you use some other plugin?
I also tried someone's mock-TBC script but I wasn't able to get it working.
The video is a Playhouse 90 episode called "The Dungeon." The video is bad the first 15 minutes, but settles down to just an occasional jerk once a minute. You might be able to find another copy, but it might be just as bad (and if on dvd, way overpriced).

Motion interpolated frames generate in-between frames from surrounding frames (as opposed to just blending, or repeating dupes). Different software can give better or worse results. Avisynth is free so at least it's worth a try. The "side effect" of motion interpolation is prediction errors - you get weird looking non natural looking artifacts. The end result depends on how bad it is and how much motion there is. Long strings of consecutive bad frames will give you worse results because there is not as much "good" information to derive new frames from.

So if you have mostly single jerks (past the 15 minutes mark), your results should be fairly good. Even if you blend the inbetween frame, at regular playback it wont be as noticable - I would argue it would be better than what you have now, at least for the single jerks. Your single jerks are heavily displaced and very noticable. So even a dupe that is in "regular position" will look not as disruptive. A blend will look smoother during playback than a dupe.

This particular example I did with nuke, but you should be able to achieve similar results with avisynth with mvtools. MFlowInter usually does a good job with replacing single frames. I added a zip file comparing the original 14,15,16 frame number sequence (15 is obviously the "bad" frame) , and the replacement generated by MVtools2

[spiritgumm]

wow, that sample looked great. I couldn't find any reference to "nuke" (except a member name). What ware is it?
I'll give MVflowinter a shot when I get back in couple days.

[poisondeathray]

Nuke is compositing and visual effects software , like after effects, except it's nodal based, not layer based . I didn't do any compositing in that example, it was straight interpolation only - you could improve upon it by removing dirt spots and compositing the bad sections of frames , for example

I'll give MVflowinter a shot when I get back in couple days.

Be careful, MVFlowInter is from the older MVTools , MFlowInter is the newer function from MVTools2

I would still look for a cleaner source if possible. While much of the interpolation is automatic, the identification of "bad" frames is still manually done - software isn't good enough to figure out what is "good frame" vs. what is "bad frame".

[spiritgumm]

I downloaded MVtools2, but I'm afraid you're overestimating my ability to decipher how to set the parameters.

There was one example script on the MVtools site which mentions MFLowinter - "To recreate bad frames by interpolation with MFlowInter:" but that script didn't stop the jitter. And I'm assuming it was suppose to recreate bad frames into good frames, not just "recreate bad frames." So...any advice?

[poisondeathray]

Yes you replace the bad frames. But remember earlier I said you still have to identify which frames are "good" , and which are "bad"

Use the examples they give in the documentation and replace the frame numbers

In the zip example above, where frame 15 was replaced with mflowinter , this was the script I used (I randomly picked 15 as a bad frame, that had "good" frames (13,16) on either side for this example) :

mpeg2source("dungeon.d2v")
tfm
tdecimate
vinverse

#bad frame 15
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2)
forward_vectors = MAnalyse(super, isb = false, delta=2)
inter = MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
trim(0,14) ++ inter.trim(14,-1) ++trim(16,0)

trim(0,14) ++ inter.trim(14,-1) ++trim(16,0)
This is saying keep frames 0-14, replace 15, keep frames 16 to end . (Of course for the real thing you would have to do that for each of the bad frames , not just 1 bad frame) . So you would have to compose inter.trim() segements for each bad frame , and append the non affected segments "good frames" - as you can see it starts to get very tedious, even if you already have a list of bad frame #'s . This is why I would prefer to do this with tools that use a GUI , but the example was just to show it's possible with free tools such as avisynth .

Maybe one of the avisynth gurus can write a wrapper function for you , that simplifies the script generation. You could ask at doom9 for example. It might just require you input the bad frame#'s

And if you recall, earlier I said there are severe limitations - if you have long strings of bad frames for example, you might have to do multiple passes, replacing the end frames of the bad run. AFAIK mflowinter couldn't interpolate past the the adjacent frames (other software can often reconstruct frames from data farther away, but that type of software is usually quite expensive).

The other severe limitation is motion prediction errors. Your sample doesn't have much motion, so the interpolation can be good if there are 2 adjacent clean frames . Try a sequence with an action movie with explosions, and this approach sucks, even with the other more expensive tools. The only way you can improve the estimation in those cases is to use foreground and background mattes to help the prediction.

[spiritgumm]

Thanks for explaining. I'm assuming from the rest of the script that it's not just substituting one frame for another but creating a hybrid, right? Or is it not motion interpolation as you described above?
The first 15 minutes has several strings of bad frames. What would a replaced string look like - a blurry still image? And I'd have to eyeball every frame of the movie to catch all the bad frames (not that I haven't done that before).
Sounds like the subbed-in frame(s) retains original audio, so that needs to be addressed separately.
I dont post on doom because of their rules requiring you to have video copyrights (even though they don't even know what the video is).

I wonder if lordsmurf looked for a better copy...

[poisondeathray]

Yes, it's interpolating a new "in between" frame . This is in contrast to the other 2 methods : replacing with a duplicate of a nearby frame or using a blend (a blend could be anywhere from a 50/50 split, or 20/80 split etc....) . The trim segments without the "inter" are untouched from the original because they are "good" frames

I mentioned the possible "side effects" or "cons" of motion interpolation above. But I would argue that even a simple blend - but in non displaced position - would look better than what you have now , because the bad frames are so displaced (mostly horizontally) and very annoying to watch . I mentioned one of the plugins earlier, called "badframes". There are a few other related ones if you search
http://avisynth.org.ru/badframes/badframes.html

Yes, there will be no sync issues because it's replacing an exisiting frame (you're not adding new frames or deleting frames - so the frame count is not changed. It's interpolating from the adjacent frames, and replacing the "bad" frame) . The audio is untouched. You're just swapping bad frames with motion interpolated ones. It's the same idea as swapping out with a blend or dupe.

For strings of bad frames, I don't think this avisynth would work very well, or you would have to do multiple passes (e.g. replace the ends, then replace the ends, then replace the ends....etc..) , because I think this plugin only works on adjacent frames and you can't specify 1 side or the other. I think if you tried it with 1 pass on a long string, you would have a blurry frame, displaced, and with errors, ie. probably worse than what you started with. But for single bad frames with limited motion, it can do a decent job.

A better copy would be ideal = way less work.

PS . You can phrase your question in a generic fashion . It doesn't have to be for specific material. It can be for general learning purposes. e.g. Asking how to write or modify a function based on mflowinter such that it accepts a list of frame numbers and appends segments.