Help stabilizing screen shaking animation?

[killerteengohan]

I am having a problem with a Blu-Ray of mine and I have never needed to fix this before, so I am kind of lost on what to do. The entire screen jerks around quite a bit to the point that it is a huge distraction and annoying. It's not just the rare occasional jerk you might see on older animation either, this happens a lot through the entire movie. The DVD I have does not do this, but the Blu-Ray does it, and it does it a lot.

I have a couple sample videos.

https://mega.nz/#!R54XQAAZ!lEYF9Nt76WGLY62iVZE3Qjx3ImG1g7G4nylTTntOs8k
https://mega.nz/#!IopxQIbT!N9NeKumGAMWlO8BLKDSj8rXC3y0lFQ77dpiD_EGbNcs


How can I reduce this or stop it from happening? any help would be greatly appreciated. Thanks!!

[jagabo]

Stab() with a moderate range helps a little. But some the frames are distorted too. You can get rid of some of the distorted frames using Dup() with a high threshold. But that will lose some of the small mouth/eye/other movements that you want to keep. Maybe you can work out something with mt_motion() and a threshold that only uses the Dup() frames with larger movements/distortions. Or, since most of the distortions occur around scene changes, only use a strong Dup() within ~4 frames of a scene change.

[pandy]

Perhaps https://www.epifocal.net/blog/video-stabilization-with-ffmpeg

[killerteengohan]

Stab() with a moderate range helps a little. But some the frames are distorted too. You can get rid of some of the distorted frames using Dup() with a high threshold. But that will lose some of the small mouth/eye/other movements that you want to keep. Maybe you can work out something with mt_motion() and a threshold that only uses the Dup() frames with larger movements/distortions. Or, since most of the distortions occur around scene changes, only use a strong Dup() within ~4 frames of a scene change.

I have never used either of those before. I will give stab a try when I get the time to see if it helps and get back to you with the results or any questions.

What is the distortion your referring to when you say some of the frames are distorted?

[jagabo]

The film frame was moving as it was being digitized.



There's no way to align the two frames. Dup() converts near duplicate frames to duplicates. With a high threshold it will see those two frames as similar enough to replace one with the other. But with a threshold that high frames with true small motions are replaced too -- like frames where just a character's mouth is moving, or eyes are blinking.

[killerteengohan]

I did notice a much bigger screen movement at that particular part of the scene before that distortion happens when I watched the clip. The large jumps like that one aren't very often and if its on a rare occasion, I can deal with that. That distortion that's in the source doesn't bother me as much as when the entire screen moves around. I didn't even notice it until you pointed it out because the screen movement took my eyes from that and I focused on the entire screen moving around. Its all those tiny full screen jumps and character movements with it driving me nuts. As long as those can be reduced or removed with as little side effects as possible or ruining good frames, I will be happy.

[killerteengohan]

Jagabo, I have no function for dup and I cannot find the plugin for the life of me. I keep getting sent to a neuron2.net website where I cannot find it at all and it looks like a fake website to me.

Could you possibly attach or share the .dll I need with me please?

[jagabo]

Dup can be found here: http://rationalqm.us/dup/dupnew.html
An update from someone else: http://forum.doom9.org/showpost.php?p=1747207&postcount=17 (dup1.dll can be found in the Release folder).

I have not been able to find a 64 bit version.

[killerteengohan]

Dup can be found here: http://rationalqm.us/dup/dupnew.html
An update from someone else: http://forum.doom9.org/showpost.php?p=1747207&postcount=17 (dup1.dll can be found in the Release folder).

I have not been able to find a 64 bit version.

Thanks, I appreciate it! I don't use 64-bit version so it's all good.

[killerteengohan]

I gave stab a try and it seemed to work pretty well at reducing the screen shaking being used at it's default settings. I am very satisfied with how much the screen movements are reduced. The only downside I am noticing so far is background movement or some moving objects seem to become a bit jumpy, or bounce around some in multiple directions.

Here is a clear example in this 23 second comparison clip.

Wthout Stab()
https://mega.nz/#!o4ojGKrI!MIym84ThxeFJ6D8UshV0L00PxvfQGvZgZ4CB2lGKqnA

With Stab()
https://mega.nz/#!os5xQCSB!HPs0_huZnxZ0EwK_Gex6dQHQy5g_KamINFzCFaMsD10


The foreground objects seem to be fine, but moving background objects like the clouds for example are what I was talking about. Look closely how smooth the cloud movement is in the original, then compare with the version that has Stab used on it. While not god awful, it is still there.

Is there a parameter I could adjust to keep it's movement more like the original source or at least lessen the noticeability of this? I tried mirror=15 but that made it a lot worse and it jumped quite a bit more. Dup seemed to make it worse as well.

[jagabo]

Stab is locking onto the character motion in that shot because the character takes up a large portion of the frame and he's brighter, sharper, and higher contrast. So when the character makes small motions the background clouds and stars bounce. This is often a problem with stabilizers. The only way I see of avoiding this is to add code to Stab to make it base its stabilization on only a portion of the frame. Then special case shots like this.

Also, at times stabilization can make slow pans jerky as it locks motion for several frames, then suddenly makes a large move, locks again, etc.

[killerteengohan]

I suppose I can work on that solution a little later. That aside for the moment, the only other thing I can think of that I would like help with addressing is the black borders that stab causes around the outer rim of the video. I suppose I could just crop 2 off all sides, but I believe I heard you can actually fill that with pixels from the original source and make it appear almost seemless. If this is true how would I go about doing that? I would like to avoid the cropping if that's possible.

[jagabo]

Using mirror=15 should get rid of the black borders and shouldn't have any effect on the deshaking. Make sure you crop any existing borders before Stab, then add them back later if necessary.

[killerteengohan]

How do you crop 2 off the border all around the video and then insert them back on afterwards?? I figured once cropped off, they were gone.

[killerteengohan]

Mirror=15 caused the background object movements I mentioned earlier to become much more erratic and noticeable. They seem to move around a lot more.

I never did find out how to crop borders off and put them back on afterwards.

[manono]

You're talking about black borders, right? Then:

Before:
Crop(2,2,-2,-2)

After:
AddBorders(2,2,2,2)

But this is so obvious that I must be missing something. As for working with borders to make them look better, you might try BorderControl. It works nicely with 2-pixel borders. More than that and it becomes progressively messier.

[killerteengohan]

You're talking about black borders, right? Then:

Before:
Crop(2,2,-2,-2)

After:
AddBorders(2,2,2,2)

But this is so obvious that I must be missing something. As for working with borders to make them look better, you might try BorderControl. It works nicely with 2-pixel borders. More than that and it becomes progressively messier.

Actually I was talking about borders that were exactly what was cropped off. As in still has the original video in it. I know how to add black borders back on, but thanks for that idea.

I thought he was saying crop 2 video off all 4 sides or somehow make a copy of them, then after stab() did its job and put those black borders on, crop those black borders off and put the original video back on it. I guess I may have misunderstood.

Can I make a second script of the original video uncropped and somehow overlay it over the cropped and stab() version to get the original outer pixels back?

[jagabo]

I thought he was saying crop 2 video off all 4 sides or somehow make a copy of them, then after stab() did its job and put those black borders on, crop those black borders off and put the original video back on it. I guess I may have misunderstood.

I was saying you need to crop black borders before Stab(mirror=15) so that the active picture is reflected at the edges, not the black borders. Then, if you need to restore the frame size (like when making a DVD) you can add black borders (or whatever you want) back -- as Manono indicated.

Can I make a second script of the original video uncropped and somehow overlay it over the cropped and stab() version to get the original outer pixels back?

Of course. Then you'll have moving borders with a still picture in the middle.

[killerteengohan]

Can I make a second script of the original video uncropped and somehow overlay it over the cropped and stab() version to get the original outer pixels back?

Of course. Then you'll have moving borders with a still picture in the middle.

At 2 pixels, I may not notice the movement much seeing as how I would be staring at the center image mostly. I would like to try it just to see it. How would an overlay like that be done in avisynth so only those outer cropped pixels are placed over top of the stabilized video?

[jagabo]

At 2 pixels, I may not notice the movement much seeing as how I would be staring at the center image mostly.

They why would you care about reflected borders or static black borders? Both of which would be less distracting than moving borders.

I would like to try it just to see it. How would an overlay like that be done in avisynth so only those outer cropped pixels are placed over top of the stabilized video?

Overlay().

[killerteengohan]

At 2 pixels, I may not notice the movement much seeing as how I would be staring at the center image mostly.

They why would you care about reflected borders or static black borders? Both of which would be less distracting than moving borders.

I would like to try it just to see it. How would an overlay like that be done in avisynth so only those outer cropped pixels are placed over top of the stabilized video?

Overlay().

Static black drives me nuts lol, I have always cropped black bars off the video. I have OCD and my reasons. I wouldn't be asking if I didn't want to try and learn and see how it looks when I have never tried something before.


I get that I would be using Overlay() but not sure how to write it so it uses only part of a second source.

So I make it like this?

A=last
B=Import("original.avs")
Overlay(A,B)

That looks like it would put the second source video over top of the first if that's being used right, but it would also cover the destabilization of the first with untouched video from the second. How do I make it so the second overlays only the outer 2 pixels border all the way around it and doesn't cover the destabilized video in the inner part?

[jagabo]

src = last
MAX_DESHAKE = 2

Stab(range=4, dxmax=MAX_DESHAKE, dymax=MAX_DESHAKE, mirror=15)
Crop(MAX_DESHAKE,MAX_DESHAKE,-MAX_DESHAKE,-MAX_DESHAKE)
Overlay(src, last, x=MAX_DESHAKE, y=MAX_DESHAKE)

[killerteengohan]

Does anyone know why stab appears to be making random frames turn green and or get a green tint/hue all over random frames it outputs?? I just noticed this and its flickering from normal colors to green tinted colors as it plays. It stabilizes great, but that is annoying.

[poisondeathray]

Does anyone know why stab appears to be making random frames turn green and or get a green tint/hue all over random frames it outputs?? I just noticed this and its flickering from normal colors to green tinted colors as it plays. It stabilizes great, but that is annoying.

This was a depan bug. Fixed a long time ago. Update your depan

[killerteengohan]

Does anyone know why stab appears to be making random frames turn green and or get a green tint/hue all over random frames it outputs?? I just noticed this and its flickering from normal colors to green tinted colors as it plays. It stabilizes great, but that is annoying.

This was a depan bug. Fixed a long time ago. Update your depan

Thanks, I will go see if the latest version fixes it for me.

Do you happen to know why it looks as if stab is adding artifacts to the video? It looks like it is sharpening it and adding grain or blocks on the video making them much worse than before.

Zoom in on these 2 images to 200% - 300% and flip through them. It will be very obvious what I am referring to.
https://mega.nz/#!EtJ3EYaZ!P3RBg7G8kpeBANzwhIQXqhHZrGIToyL8Se84KkaPLbY


I also have these 2 video clips. One without and one with stab to show what I am talking about. The moving blocks or artifacts are more noticeable.
https://mega.nz/#!V1RDEAJQ!l65YCOtSKO9-94cNNGnxOZ8WHICClmx0gHIrc2KF2wc

View it in fullscreen starting at the 12 second mark and look at the left collar on his green jacket around his neck. It flickers brightly for some reason and does not in the version without stab. That flickering is very annoying and or distracting.

[killerteengohan]

Actually updating depan fixed both the green tint issue and the flickering I mentioned with those 2 video samples. Thanks for bringing that up!

It still looks like it is sharpening the image and adding the grain/artifacts like in the images, but the green tint and bright flickering has stopped.

Whats up with the artifacts?

[jagabo]

Stab uses Depan. Depan works with subpixel accuracy by default. That essentially means the image is upscaled, deshaken, then downscaled. That is done with a bicubic filter by default. Hence you get a little sharpening. You could switch to bilinear but the you'll probably complain it blurs the video a little. Or disable sub-pixel accuracy but you'll be left with more shaking.
Maybe blend two Stabs together -- one with bicubic, the other with bilinear.

function StabMod (clip clp, int "range", int "dxmax", int "dymax", int "mirror", int "subpixel")
{
    range = default(range, 1)
    dxmax = default(dxmax, 4)
    dymax = default(dymax, 4)
    mirror = default(mirror, 0)
    subpixel = default(subpixel, 2)

    temp = clp.TemporalSoften(7,255,255,25,2)
    inter = Interleave(temp.Repair(clp.TemporalSoften(1,255,255,25,2)),clp)
    mdata = DePanEstimate(inter,range=range,trust=0,dxmax=dxmax,dymax=dymax)
    DePan(inter,subpixel=subpixel, data=mdata,offset=-1, mirror=mirror )
    SelectEvery(2,0)
}


# Blend bilinear and bicubic
# Merge(StabMod(subpixel=2), StabMod(subpixel=1))

[killerteengohan]

Well I am trying to use stab on selected frames only. It works fine until I call it about 18+ times and then it always gives me an error. I have several trims to use it on and I certainly need to be able to call it more than 18 times.


[Attachment 46642 - Click to enlarge]


It seems to be this line in the stab avsi giving me all the trouble.

mdata = DePanEstimate(inter,range=range,pixaspect=PAR,trus t=0,dxmax=dxmax,dymax=dymax,zoommax=zoom)
and also this one (line 17) when the error changes after I exit and reopen MeGui
DePan(inter,data=mdata,offset=-1, mirror=mirror )


I tried stab2 as well and its giving me the same issue. Why cant I cant I get this to work?

[jagabo]

Well I am trying to use stab on selected frames only.

How exactly are you doing that? Maybe you're running out of memory. What about calling it only once and using ReplaceFramesSimple()?

[killerteengohan]

Well I am trying to use stab on selected frames only.

How exactly are you doing that? Maybe you're running out of memory. What about calling it only once and using ReplaceFramesSimple()?

I am doing it like this.

I am selecting all the frames I want to stabilize and the frames I do not want to stabilize, then I am doing the rest of the filtering once that's finished.


__film = last
__t0 = __film.trim(0, 834)stab()
__t1 = __film.trim(835, 3341)
__t2 = __film.trim(3342, 8461)stab()
__t3 = __film.trim(8462, 11551)
__t4 = __film.trim(11552, 21088)stab()
__t5 = __film.trim(21089, 21741)
__t6 = __film.trim(21742, 23979)stab()
__t7 = __film.trim(23980, 24111)
__t8 = __film.trim(24112, 25172)stab()
__t9 = __film.trim(25173, 25412)
__t10 = __film.trim(25413, 25462)stab()
__t11 = __film.trim(25463, 25653)
__t12 = __film.trim(25654, 26050)stab()
__t13 = __film.trim(26051, 26133)
__t14 = __film.trim(26134, 26407)stab()
__t15 = __film.trim(26408, 26456)
__t16 = __film.trim(26457, 26532)stab()
__t17 = __film.trim(26533, 26653)
__t18 = __film.trim(26654, 27762)stab()
__t19 = __film.trim(27763, 27798)
__t20 = __film.trim(27799, 27834)stab()
__t21 = __film.trim(27835, 27941)
__t22 = __film.trim(27942, 28099)stab()
__t23 = __film.trim(28100, 28226)
__t24 = __film.trim(28227, 29227)stab()
__t25 = __film.trim(29228, 29371)
__t26 = __film.trim(29372, 32821)stab()
__t27 = __film.trim(32822, 33040)
__t28 = __film.trim(33041, 33243)stab()
__t29 = __film.trim(33244, 34013)
__t30 = __film.trim(34014, 34061)stab()
__t31 = __film.trim(34062, 35594)
__t32 = __film.trim(35595, 35635)stab()
__t33 = __film.trim(35636, 36209)
__t34 = __film.trim(36210, 36262)stab()
__t35 = __film.trim(36263, 39633)
__t36 = __film.trim(39634, 42645)stab()
__t37 = __film.trim(42646, 42714)
__t38 = __film.trim(42715, 42810)stab()
__t39 = __film.trim(42811, 42859)
__t40 = __film.trim(42860, 46662)stab()
__t41 = __film.trim(46663, 46704)
__t42 = __film.trim(46705, 48919)stab()
__t43 = __film.trim(48920, 49033)
__t44 = __film.trim(49034, 52121)stab()
__t45 = __film.trim(52122, 52153)
__t46 = __film.trim(52154, 52209)stab()
__t47 = __film.trim(52210, 52437)
__t48 = __film.trim(52438, 52528)stab()
__t49 = __film.trim(52529, 52857)
__t50 = __film.trim(52858, 64550)stab()
__t51 = __film.trim(64551, 65394)
__t52 = __film.trim(65395, 70411)stab()
__t53 = __film.trim(70412, 70435)
__t54 = __film.trim(70436, 70976)stab()
__t55 = __film.trim(70977, 71012)
__t56 = __film.trim(71013, 71099)stab()
__t57 = __film.trim(71100, 71726)
__t58 = __film.trim(71727, 73124)stab()
__t59 = __film.trim(73125, 73672)
__t60 = __film.trim(73673, 74478)stab()
__t61 = __film.trim(74479, 75940)
__t62 = __film.trim(75941, 76211)stab()
__t63 = __film.trim(76212, 77012)
__t64 = __film.trim(77013, 77093)stab()
__t65 = __film.trim(77094, 77248)
__t66 = __film.trim(77249, 77985)stab()
__t67 = __film.trim(77986, 79429)
__t68 = __film.trim(79430, 80106)stab()
__t69 = __film.trim(80107, 81472)
__t70 = __film.trim(81473, 83160)stab()
__t71 = __film.trim(83161, 84061)
__t72 = __film.trim(84062, 85238)stab()
__t73 = __film.trim(85239, 86012)
__t74 = __film.trim(86013, 86068)stab()
__t75 = __film.trim(86069, 87186)
__t76 = __film.trim(87187, 87238)stab()
__t77 = __film.trim(87239, 87273)
__t78 = __film.trim(87274, 87369)stab()
__t79 = __film.trim(87370, 88226)
__t80 = __film.trim(88227, 93483)stab()
__t81 = __film.trim(93484, 95023)
__t82 = __film.trim(95024, 112195)stab()
__t83 = __film.trim(112196, 113076)
__t84 = __film.trim(113077, 113450)stab()
__t85 = __film.trim(113451, 114094)
__t86 = __film.trim(114095, 114190)
__t87 = __film.trim(114191, 114520)stab()
__t88 = __film.trim(114521, 114852)
__t89 = __film.trim(114853, 116105)stab()
__t90 = __film.trim(116106, 116373)
__t91 = __film.trim(116374, 118152)stab()
__t92 = __film.trim(118153, 118197)
__t93 = __film.trim(118198, 120194)stab()
__t94 = __film.trim(120195, 126080)
__t95 = __film.trim(126081, 126722)stab()
__t96 = __film.trim(126723, 127184)
__t97 = __film.trim(127185, 129430)stab()
__t98 = __film.trim(129431, 129545)
__t99 = __film.trim(129546, 131930)stab()
__t100 = __film.trim(131931, 131972)
__t101 = __film.trim(131973, 132091)stab()
__t102 = __film.trim(132092, 132757)
__t103 = __film.trim(132758, 132871)stab()
__t104 = __film.trim(132872, 132972)
__t105 = __film.trim(132973, 133332)stab()
__t106 = __film.trim(133333, 133460)
__t107 = __film.trim(133461, 134041)stab()
__t108 = __film.trim(134042, 135219)
__t109 = __film.trim(135220, 142195)stab()
__t110 = __film.trim(142196, 150153)
__t0 ++ __t1 ++ __t2 ++ __t3 ++ __t4 ++ __t5 ++ __t6 ++ __t7 ++ __t8 ++ __t9 ++ __t10 ++ __t11 ++ __t12 ++ __t13 ++ __t14 ++ __t15 ++ __t16 ++ __t17 ++ __t18 ++ __t19 ++ __t20 ++ __t21 ++ __t22 ++ __t23 ++ __t24 ++ __t25 ++ __t26 ++ __t27 ++ __t28 ++ __t29 ++ __t30 ++ __t31 ++ __t32 ++ __t33 ++ __t34 ++ __t35 ++ __t36 ++ __t37 ++ __t38 ++ __t39 ++ __t40 ++ __t41 ++ __t42 ++ __t43 ++ __t44 ++ __t45 ++ __t46 ++ __t47 ++ __t48 ++ __t49 ++ __t50 ++ __t51 ++ __t52 ++ __t53 ++ __t54 ++ __t55 ++ __t56 ++ __t57 ++ __t58 ++ __t59 ++ __t60 ++ __t61 ++ __t62 ++ __t63 ++ __t64 ++ __t65 ++ __t66 ++ __t67 ++ __t68 ++ __t69 ++ __t70 ++ __t71 ++ __t72 ++ __t73 ++ __t74 ++ __t75 ++ __t76 ++ __t77 ++ __t78 ++ __t79 ++ __t80 ++ __t81 ++ __t82 ++ __t83 ++ __t84 ++ __t85 ++ __t86 ++ __t87 ++ __t88 ++ __t89 ++ __t90 ++ __t91 ++ __t92 ++ __t93 ++ __t94 ++ __t95 ++ __t96 ++ __t97 ++ __t98 ++ __t99 ++ __t100 ++ __t101 ++ __t102 ++ __t103 ++ __t104 ++ __t105 ++ __t106 ++ __t107 ++ __t108 ++ __t109 ++ __t110