Stabilization Tools Pack v2.1

[Dogway]

Deshaker is the one that tries to smooth global motion for hand held shake, not fix rock-steady a sucession of frames, you are using the wrong tool for the job. That's why Stab and Deshaker are so different. IIRC (please correct me if I'm wrong) plugins in NLE or standalone tools commonly have both algos as parameters so you switch one or another on a scene-to-scene basis, painful. I did fix a DV trip video last xmas and used Deshaker succesfully, its goal is to reduce the smaller nervous shake that gets your head dizzy. Probably there could be a stabilizer that tries to use both depending on global motion, but this could easily be prone to issues.

Most deshakers aim for the first "smooth motion" that deals with hand held video footage like those from DV cams, or more recently smart phones (embedded deshakers). Stab() belongs to the less common case where you want to stick the frame in place through the whole scene, dealing with subpixel precision, it is important to understand this difference. In this thread I only talk about the latter case, I'm curious to know if some plugins like Nuke's F_Steadiness or AE's Warp Stabilizer can help in this regard. So I'm not sure if this is the issue you have with your source, can you post a sample?

My sample is mostly fine at source (check rocks at bottom-right), after stab() the rocks now shake (subpixel shake), this is a depan false-positive fooled by the character trembling, it happens more severely with people clapping and so on. To address this I tried to used a mask (revealing only 1/8 frame borders) since action is typically at the center, but Depan caught the mask as global motion, even blurring the mask didn't help... I just thought I can try to remove the black portion to take it out of the equation and see if works.

For autoadjust check the official thread, but I fear the XP version is very old now, for things like these I felt forced to switch to Win7 for serious work :(

[lordsmurf]

This thread is deja vu. I've had this conversation before at VH.

I'm not using the wrong tool for anything. Clearly they have different specific uses, but both are still stabilization methods. Stab() is just corrections within a shorter range. But stab() can also be augmented (via DePanEstimate) to be as wide as you want to go. Stab() also has a side effect function to decimate frames, which can be useful for some sources.

Deshaker is too inaccurate at worst, too weak at best. That's been my experience for years. I often think those that say "you're doing it wrong" just have lower expectations or less-worse sources. So, for them, maybe it does work fine. Anytime I see before/after samples of Deshaker on Youtube, it either still looks like crap, or was a minimal correction that would have been handled fine with other/better methods.

Honestly, the sample footage errors are so tiny as to be unnoticeable. I'm worried about far more severe errors that make a difference between watchable and unwatchable.

[Dogway]

In my opinion the smooth motion type of deshakers have been discussed to dead, this thread instead is not deja vu since I talk about the often ignored static scene stabilization, this is critical specially on animation, if you never work with that I can understand why you don't find it useful. This is why I dedicated the first 2 lines of OP to make this very thing clear. You haven't supplied any sample for me to test either so it's all talk by now.

To reinforce what I am talking about, I went and checked AE's Warp Stabilizer documentation. On the "result" setting you can choose between "smooth motion" (Deshaker look), or "No motion" (Stab() look). Honestly I haven't delved any more on Depan to see if it can do "Smooth Motion" deshaking because Deshaker by Gunnar Thalin is very good and can be used with avisynth as well. What we are lacking is more "no motion" type of deshakers for avisynth.

As for the sample errors, they might pass unnoticeable but when upscaled to fullscreen they are not unnoticeable any more. As I said this is subpixel precision work, it's the same as the 8-bit or 16-bit filtering topic, some people have stuck to old avisynth practices and still think filtering in 16-bit is useless and unnoticeable, other jumped on the high bitdepth bandwagon and are content with their workflow, you can't blame neither.

[johnmeyer]

Mercalli may be what I want. But I've also read bad reviews on it, and the samples were clearly cherry-picked. However, that was some years ago, so I'll give it another go. I'm going to download the demo and put some ugly footage to the test. (I don't really consider $119 to $299 expensive. That's doable.)

Mercalli is definitely the Cadillac of the stabilization products. Download the trial and see what you think. You really don't have to cherry pick the clips. It works pretty well on just about anything. Of course ANY product or technology will fail under certain circumstances, and reviewing a product or technology merely on the places where it fails doesn't help much.

If you do decide to trial it, make sure to try out the different "cameras," which is their interface to the different stabilization algorithms. They do different things if you tell it that the camera is in motion (like shooting from a car) than if you are trying to hold the camera steady on a distant object (like the bird video). If you don't make that adjustment, it may make the wrong assumptions and do the wrong thing.

Can you answer these?
- What's the important difference between the plugins and standalones?

The standalone is the way to go if you are serious about getting the best stabilization. Of course using a standalone means a more complicated workflow, because you have to save to an intermediate, and then bring that back into your NLE. With Mercalli, all the best features are only in the standalone. I've been using the Vegas plugin, and I am fed up with the limitations and am going to purchase the Mercalli V4 standalone as soon as time and budget permit.

Do the plugins work with Premiere CS3+. I can't find that info. Honestly, I don't really care for NLE plugins. I prefer standalones.

I think so, but since I don't use that program I cannot say for sure. However, since you prefer standalones, you can avoid all the limitations and hassles of the plugin.

The jello-vision removal of the V4 is interesting.

I agree. Most of my cameras are relatively free of this problem, with my main camera (an old Sony FX1) not having it at all because its sensor predates the switch to CMOS. Their earlier attempts at this helped a little bit, but were not in the same league as the stabilization.

Stabilization mat not be as bad as I'm suggesting, but it's also not utopian as you're suggesting.

Well, I don't know how to address that because, as you rightly point out, it doesn't work each time and every time, and it is not perfection. The two issues that keep it from being true perfection are the border fixup and the blur from slow shutter speed. If you know you are going to have to stabilize and set the shutter speed to even 1/125, that avoids the blur problem completely. The border issue is still a problem with the version I have, but see my comments in the next paragraph about the new features in V4.

However, I really need to emphasize that neither Jerry or I, or any of the other people who have posted Mercalli results on YouTube and Vimeo have cherry picked our footage. The product really is very, very good. Go back and play that side-by-side shot of the bird.

More to the point, I just finished editing a four camera wedding that I shot, all by myself, ten days ago. Needless to say, keeping four cameras running involved a lot of compromises, and the one I hand-held while I ran around to different vantage points often had to be stabilized. The "rock steady" setting in Mercalli (which is what Jerry used with his extreme telephoto hand-held footage) really does produce near perfect results. However, if the shot moves too much, then to avoid borders, the algorithm has to zoom in more. This, of course, makes the image go soft, since you are losing pixels from the zoom. The new version (V4) of Mercalli supposedly has much better border fixup than previous version. This is one thing Gunther Thalin pioneered with his Deshaker. It was far from perfect, but in some cases did allow you to avoid losing resolution from the zoom needed to mask the black borders. If the new Mercalli has "solved" this problem, then we are actually getting pretty close to the perfection that you want.

Out of all the new features in Mercalli, this is the one that will make the biggest difference to me and, I suspect, to most other users.

Now, if you really want to talk about miracle restoration software, let's have a discussion about iZotope RX4 Advanced. It costs a month's salary, but it truly can perform miracles, and it can do it on just about anything -- no cherry picking at all. If you have problem audio, I can fix it with this tool. It is truly FM.

[poisondeathray]

You're correct dogway - there are different classes and types of trackers and stabilizers. They are used for different scenarios. I hope you don't mind because it's moving away from your original thread topic

Deshaker and Mercalli fall in the hand held category. Although Mercalli touts "a rock stable mode" it's not what people in VFX call "rock stable" . There is a big difference. Rock stable is required for things like compositing - you need sub/pixel perfect accuracy. Deshaker and Mercalli are useless for this, they are bascially ease of use, 1 or 2 click wonders - they usually do a good job for smoothing out hand held footage on typical shots

One big problem is severe deviations will skew their results - e.g. lets say a car zooms by in the scene or someone waves their hand and it takes up a large portion of the screen. The results will sort of follow along. There is no way to exclude or include areas to track . User intervention is required for that - that's why host FX programs like After Effects, NukeX are used for those types of shots. You attempted a similar solution with a mask in avisynth. In your example clip, even small motion skewed the results of stab. There is no way an algorithm can automatically determine what is foreground or background or what to exclude with any accuracy - human eyes are required.

You wan't to minimze the amount of user guidance, so there is no reason why it has to be only one or the other; you can combine avisynth , with deshaker, with mercalli etc..You might limit some "complex" shots for use in mocha or nukex or AE .

The other broad classes are point vs planar trackers (those distinguish by tracking methodology) and matchmoving or 3d trackers which are used for 3d compositing . Each has specific scenarios or types of shots that they work well for, and each has weaknesses as well.

The additional problem with animation and film gate weave, is often it's accompanied with frame warping. It's not just x,y translation and zoom .

Mercalli V4 does a good job of improving rolling shutter - far better than any other tool so far - but the examples you see around are cherry picked. I upgraded a few months ago. IMO it's still the best for generic hand held shot smoothing



Adding the ability to include masks / inclusion/exclusion would be a good improvement for stab/etc... If you can find a way to add that it would greatly improve the usefulness in some situations. I have no idea how to implement that in code...

[johnmeyer]

(I saw jmac on one of those posts. I wish jmac would develop some useful filters, as he has lots of useful ideas. He was the closest I ever saw towards a software TBC actually happening.)

I followed that thread closely, with fingers crossed, hoping he would succeed. Unfortunately, I never saw even one clip that ever looked better after going through the script. He had the right idea, but just couldn't quite pull it off.

I think the amazing RT_Stats library, developed by StainlessS, might provide the tools needed to complete his idea, and it could be done as a script, since most of the plumbing is already in RT_Stats. I already mentioned the project where I was able to remove noise bars from a video taken of a CRT screen, but I helped someone else who was trying to remove the weird horizontal offsets you get when you incorrectly re-size interlaced video. If you skim through this thread:

repair bad deinterlacing

you will see that it involves some of the same ideas, namely the ability to detect horizontal offsets between scan lines, and then attempt to line them back up. We actually produced a result that was quite a bit better than the original.

If you don't have time to read that whole thread, just look at the image in the first post, and then look at the various improvements posted by StainlessS in this post:

Bad Deinterlacing Improvements

Not perfect, but remarkably better. Both of us ran out of steam when the OP didn't stay involved.

[johnmeyer]

Although Mercalli touts "a rock stable mode" it's not what people in VFX call "rock stable" . There is a big difference. Rock stable is required for things like compositing - you need sub/pixel perfect accuracy. Deshaker and Mercalli are useless for this, ...

You are definitely correct. That is something I haven't yet had the need to do. Interesting problem.

One big problem is severe deviations will skew their results - e.g. lets say a car zooms by in the scene or someone waves their hand and it takes up a large portion of the screen. The results will sort of follow along. There is no way to exclude or include areas to track . User intervention is required for that.

Yes, I wish there were an interface that would permit this. There is a similar problem when using motion estimation for creating tween frames: when a distant object emerges behind a foreground object, the algorithm creates weird morphs. Deshaker does have a rectangular mask that can tell the algorithm to ignore motion inside our outside of any arbitrary, non-rotated rectangle, but since it cannot be keyframed, it is of only limited usefulness (although I have actually used it to solve a few problems, and it did work).

The additional problem with animation and film gate weave, is often it's accompanied with frame warping. It's not just x,y translation and zoom.

I do a lot of work with film and have not seen frame warping problem when removing gate weave. However, I certainly have seen it on other footage.

[Dogway]

poisondeathray: Yes, I tried to fix the false-positives with a center mask, and now I might try without the mask (only the borders rearranged or so). Anyway a plugin must be in need here, the concept I think shouldn't be very difficult, I think Stab() currently works fine enough except for the false-positives, what we should do is run another analysis pass on top to discard these scenes. The logic should be that every part of certain (thr) high frequency details must be "%" different to neighbour frames to be considered jitter. That is, if high frequency details are exact through the frames, even if it's just a portion then there is lack of jitter (as jitter is global), and no stabilization should be performed. This only a proof of concept and maybe I can try to do something with masktools later on, currently haven't much time.

I have no experience on Mercalli "rock stable mode" or AE's Warp Stabilizer "No motion". From the Bird's example johnmeyer linked it looks very very stable, but maybe I'm being fooled by the huge difference against the original. If indeed those don't work it's a bit of a let down. I used Nuke in my CG days but haven't tested F_Steadiness plugin, do you think this is what can achieve VFX grade "rock stable" results? I really would like to know. I read about Mocha 3D tracking (that should be damn perfect) but... tracking each scene of a one hour source as a hobby? Sometimes I'm not sure where I must put the limit.

On frame warping I have been tackling the issue for some time now. What works best IMO is layering different temporal denoisers on the highest frequencies, sometimes with masking and whatnot, but it's not perfect as anything is.

[lordsmurf]

The two issues that keep it from being true perfection are the border fixup and the blur from slow shutter speed.

Not really. Those are minor nuisances at most. What I see a lot of is false positives and false negatives. It misses motion, or it screws up the stabilizing.

For example: I think I've found a flaw in DePanStabilize and Avisynth -- one not documented anyway. jagabo and I will be looking into it in private for now. It's still early, and it's still just a hypothesis based off some observations. Stuff like this is too common when it comes to video stabilization. It's so touchy and problem-prone.

However, I really need to emphasize that neither Jerry or I, or any of the other people who have posted Mercalli results on YouTube and Vimeo have cherry picked our footage. The product really is very, very good. Go back and play that side-by-side shot of the bird.

Ah, those are yours/his. I did not know that. I thought those were official videos from that company.

I just finished editing a four camera wedding that I shot

Editing, restoring AND shooting? You really are a glutton for punishment.

However, if the shot moves too much, then to avoid borders, the algorithm has to zoom in more.

I'm not worried about borders. Simple stabilizing and matting will appease most anybody, even me. Those extra dozen or so pixel on the edges mean little compared to the chewy gooey center of the image where the action is. I know some need/want the edges, but that's the least of my worries.

Now, if you really want to talk about miracle restoration software, let's have a discussion about iZotope RX4 Advanced. It costs a month's salary, but it truly can perform miracles, and it can do it on just about anything -- no cherry picking at all. If you have problem audio, I can fix it with this tool. It is truly FM.

I actually still have an old version installed (v1), from a project long ago that needed it. But I never used it after that. I'm quite pleased with other methods using a mix of SoundForge, Audacity, and Goldwave. Pricing isn't much different. That one is complicated, and I didn't get very far with it. I remember you mentioned this software a while back.

There is no way to exclude or include areas to track

Yeah, it just screws up the scene. What I end up doing is stabilizing as much as possible, with a mix of programs, and then splitting it up and frankensteining back together again in Premiere or FCP, all while trying to use as much restored footage as possible.

Heavy NR is the same way. You run it once for "most" the video to be fine. Then you run it 1-2 more times with lower NR settings. Cut and cobble to get the finished product.

"Hollywood" goes scene-by-scene to start with, but individuals and small studios (even medium-sized ones) don't have the budget for that level of work. On a "good, better, best" scale, that's better.

What I'm wanting to do is make that process faster, requiring less man hours. Pipe dream, I know, but sometimes dreams come true!

[poisondeathray]

I have no experience on Mercalli "rock stable mode" or AE's Warp Stabilizer "No motion". From the Bird's example johnmeyer linked it looks very very stable, but maybe I'm being fooled by the huge difference against the original. If indeed those don't work it's a bit of a let down. I used Nuke in my CG days but haven't tested F_Steadiness plugin, do you think this is what can achieve VFX grade "rock stable" results? I really would like to know. I read about Mocha 3D tracking (that should be damn perfect) but... tracking each scene of a one hour source as a hobby? Sometimes I'm not sure where I must put the limit.

It might look "stable" enough watching it - don't get me wrong - it's certainly a fantastic improvement and pleasing to watch especially compared to "before" . But it's wobbly and unusable in VFX terms. You need "rock stable" several magnitude higher do things such as compositing. If you applied a patch there, it would "slip off". If you added something to the scene or a repair patch to "fix" some defect, it wouldn't "stay put". Ignore the moving weeds/grasses for the moment and get a general sense of the entire scene. There is rotation if you look at the difference between the start and end frame (that's the easiest to see) but there is translation between frames as well. "Rock stable" would be framed exactly the same between the start and end (and everything in between).

F_steadiness isn't used very much or very good, it's a leftover from Tinderbox days. Mocha's 3D tracking actually isn't that good (there are far better 3D trackers) , but it's a great 2D planar tracker. Warp stabilizer doesn't really produce "rock stable" results either, not without user input. None of the other methods are 1 click, they all require at least some user input. They take much longer to do - that's why you don't use them for general use, you only use them for specific shots and situations. Or maybe if you have a team and the budget to do it .

BTW, my definition of "cherry pick" is that the clips you see online demonstrate only "very good results" not the bad or mediocre ones. By definition that is cherry picking - only the "good cherries" are shown. You're seeing bias creep in. A more neutral review would also show the clips and situations where it fails, or where it doesn't work quite as well. I don't mean imply that John and the others are intentionally misleading - perhaps they just didn't take a large enough sample size. A completely new person will get the wrong idea of how it really performs expecting everything, everytime to be really good. Don't get me wrong - I have mercalli v4 and hands down, it's the best for rolling shutter so far (that list includes plugins specific for rolling shutter) . Really a stellar improvement. The results are great in general for hand held footage overall too, and very adjustable/tweakable (e.g. more zoom, less smooth etc..). But it doesn't perform that well some situations and is not "VFX rock stable" (it's not meant to be).

[Dogway]

I never did (much) compositing so I don't know but then what would you suggest for rock stable solutions? We have probably named most of the professional tools out there, even if I won't bother I wonder how it compares to avisynth tools. Currently we have stab() and stabilise() (this one has a great concept but was underdeveloped).
On rolling shutter I heard good things on the Nuke's plugin but as I said I haven't run a benchmark or anything, I used it on critical scenes on a past project.

[poisondeathray]

I never did (much) compositing so I don't know but then what would you suggest for rock stable solutions? We have probably named most of the professional tools out there, even if I won't bother I wonder how it compares to avisynth tools. Currently we have stab() and stabilise() (this one has a great concept but was underdeveloped).

There is not necessarily a single "best" solution . Different shot characteristics require different approaches. On some shots a point tracker like AE's internal one might work well, but others, mocha's tracker . For example a noisy shot might "confuse" the point trackers. Or there might not be suitable planes to track for mocha.

Sometimes you have to "massage" the footage then reapply to the original footage. Eg. if noise is interfering with the track, you might track a denoised version then apply that to the original. There are bunch of other tricks you might use, it depends on the specific shot

But my favorite overall for VFX stable 2D tracking would be mocha

On rolling shutter I heard good things on the Nuke's plugin but as I said I haven't run a benchmark or anything, I used it on critical scenes on a past project.

It's good for improving linear skew (leaning buildings, trees etc...), but does almost nothing for "jello". Mercalli produces a significant improvement with jello as well (under most circumstances)

[johnmeyer]

Stabilizing for compositing is a completely different animal than stabilizing to make shaky footage watchable. Heck, I own seven tripods, and only one would likely hold a camera steady enough to do match shots, and then only under ideal studio conditions. Outdoors, all it takes is a little wind to make the tripod vibrate and that is going to be too much for what you folks want to do. This is even truer when you start using extreme telephotos. If you looked at any of the other wildlife photos on that site I linked to, you'll find that he has some videos taken with his DSLR using several of those absurdly long telephotos that the paparazzi use to photograph movie stars on remote ocean beachfronts. Those require all sorts of help to get stable results.

I have concluded that you folks are talking about perfection that is way beyond anything most of us consider. I'm just trying to get something that looks professional and will make a client happy and, for that, Deshaker, Depanstabilize, and most of all Mercalli, are really helpful tools.

[lordsmurf]

A more neutral review would also show the clips and situations where it fails, or where it doesn't work quite as well. I don't mean imply that John and the others are intentionally misleading - perhaps they just didn't take a large enough sample size. A completely new person will get the wrong idea of how it really performs expecting everything, everytime to be really good. Don't get me wrong - I have mercalli v4 and hands down, it's the best for rolling shutter so far (that list includes plugins specific for rolling shutter) . Really a stellar improvement. The results are great in general for hand held footage overall too, and very adjustable/tweakable (e.g. more zoom, less smooth etc..). But it doesn't perform that well some situations and is not "VFX rock stable" (it's not meant to be).

I may take up your challenge, and show some of the uglier sides of stabilizing video. This is going to take some time, but I think it's worth it. Too many samples are cherry picked. Many more are just crappy samples from novices. It'll be good to show how different filters cause different outcomes, as well as how much real work is involved with true restoration (NLE!).

With Mercalli, I'm seeing what you're seeing. It's not perfect, just different. I think I'm going to grab this, since it's not expensive. Mercalli fixes some scenes where my DePanStabilize failed. It'll just provide yet another filter version to dump into an NLE and chop up. I vaguely researching this some years ago, before all my medial issues. I've been having deja vu with this conversation.

So, if anything, this has added more work, not saved it. But the end results are going to be better.

[poisondeathray]

I have concluded that you folks are talking about perfection that is way beyond anything most of us consider. I'm just trying to get something that looks professional and will make a client happy and, for that, Deshaker, Depanstabilize, and most of all Mercalli, are really helpful tools.

Perfection? hell no ! They have their share of problems too. Sometimes you need to use a bunch of tricks and jump through hoops to get the other techniques working. Sometimes a shot is unsolvable.

I agree that they are helpful tools to have, for general usage of smoothing out shots. - I've already mentioned that several times

I'm just pointing out that Mercalli is using the term "rock stable" very loosely. It's a special term used in the VFX industry reserved for something else entirely. I'm not trying to pick on that specific clip, I can show you other Mercalli or deshaker processed clips that drift, even on controlled indoor shots.

But it's important to distinguish that there are different types of stabilization - different tools suitable for different situations. It's important to be clear on what they can or cannot do, or what they are supposed to be used for. It's about using the right tool for the job, or at least about knowing about the different options or approaches

There are a bunch of tracking, stabilization and restoration technqiues that require on the rock stable category. Or sometimes you need to fix a certain segment or problem area that deshaker or mercalli had problems with. There are those common prototypical scenarios that trackers in that class always have problems with. Anyone that has used them more than causally will know what I'm talking about. Do you just ignore them or say "good enough ?"

You almost make it sound like it's obscure or something nobody uses. AE isn't exactly rare software - it comes with the Adobe subscription, and MochaAE comes with AE. Blender is free and has a few trackers, including a planar tracker. It's a great compositor too. Nuke released a free version. People use these types of approaches with compositing, roto/masks, tracking , patch repairs all time in restoration work. Roto is tedious work - But having a rock stable track and stabilization will cut the amount of work down significantly.

[poisondeathray]

With Mercalli, I'm seeing what you're seeing. It's not perfect, just different.

No, it's not "perfect" - but it's still the best of the bunch in that class, IMO . In terms of stabilization it's not "light years" better, but it does have the edge in most shots . As mentioned earlier it has the same problems as deshaker or any of the others when encountering some difficult scenarios - there is no "automatic" fix for those types of issues

The rolling shutter improvement is what sold it for me. That is significantly better than the others. Very few dealt with wobble very well, until now (linear skew maybe, but the jello/wobble wasn't addressed very well) . It' s still not perfect, but I haven't seen any better in that regard

[johnmeyer]

You almost make it sound like it's obscure or something nobody uses. AE isn't exactly rare software - it comes with the Adobe subscription, and MochaAE comes with AE. Blender is free and has a few trackers, including a planar tracker. It's a great compositor too. Nuke released a free version. People use these types of approaches with compositing, roto/masks, tracking , patch repairs all time in restoration work. Roto is tedious work - But having a rock stable track and stabilization will cut the amount of work down significantly.

So are you saying that Blender, or MochaAE, or something else works better, and can truly achieve "rock stable" with the same clips on which Mercalli fails? If so, that is news to me. I wasn't aware that either of these programs could achieve better results with the clips I demonstrated.

I'd like to see some results!!

I have shown results, have linked to the guide I wrote, and have referenced the extensive scripts I have written to automate the "deshaking" process. All I hear in return is that sometimes motion stabilization fails, or that it doesn't stabilize down to the sub-pixel level.

Tell me something I don't already know.

Having written that Deshaker guide and spent several days writing and improving those automation scripts, and having spent dozens of hours stabilizing clips with Mercalli (and MotinPerfect and Steadyhand, and Twixtor), I am well aware of their limitations, and even included many of those limitations in the Deshaker guide itself, should you care to read it.

I don't understand any of these posts. It is as though any attempt I make to point out a feature or advantage that one tool might have over another is met with a claim that I am hyping the product or making claims of perfection that don't exist. Good grief, give me a break! Of course none of them are perfect, and of course they fail on some clips. I would guess that I give up on at least 25% of the clips that I attempt to stabilize. I give up and either use another shot or just live with the unwanted motion because the stabilizer version looks worse due to the artifacts that have been introduced such as blurring, warping, too much zoom, bad edge fill patterns, rolling shutter artifacts, etc.

There, I just listed a whole bunch of ways these programs can fail. Feel better?

Any restoration tool fails to work on some material, and this is no different.

In the end, what point are either of you trying to make? FWIW, I don't have this problem over in doom9.org where I've made several hundred posts over the past 7-8 years. Sure, they'll tell me that my script is written wrong, or that there is a better way to do it, but then they'll offer some code, or give a link to a better result, or try to help me. No one here is posting results or providing information on what technique or product will produce better results. All I get is comments like "The problem with "stabilizing" filters (Avisynth or otherwise) is that none of them really stabilize at all, beyond very minor issues you mostly see on VHS tapes. As far as in-picture content is concerned, it does almost nothing." I added the emphasis there, but that statement just simply isn't true, and the examples I posted, like the bird video (which, once again, was NOT cherry picked, and was done by a user, not by the company), clearly show results that most people would call remarkable, stunning, and amazing.

I am a relative newcomer in this forum, but this exchange doesn't make me want to hang around any more. See ya over at doom9.org.

[Dogway]

johnmeyer, let's chill out all a bit, I ran away from doom9 due to their repeatedly hostility to me. I have been fine and good here but truth is VH is more a chatty place whereas doom9 is more an action place (more akin to our character). I have a hunch that lordsmurf wanted simply to bomb my thread for some reason (mind the tone in his first post), so far he hasn't supplied a sample either.

About the discordance of speech between poisondeathray and me with you is because we are talking about a very different type of stabilization that you might have not usually dealt with.
Your goal is to reduce shaky motion (even if not at subpixel precision), normally from hand held cameras. And we, or rather I, am trying to fix the subtle jitter or shakiness that is created from the the telecine process, film passing through the film gate. Specially on 2D animation, where there are a lot of static scenes, this type of subpixel stabilization algorithm is what should be used most of the time because "reducing motion" is not good enough if it's not "fixing motion" (or "rock stable" you name it).
I tried to bring this discussion in doom9 in the past, but animation is not held in high regard over there, by the knowledgeable people at least. We all know our share on the topic, let's just listen and learn.

[poisondeathray]

You almost make it sound like it's obscure or something nobody uses. AE isn't exactly rare software - it comes with the Adobe subscription, and MochaAE comes with AE. Blender is free and has a few trackers, including a planar tracker. It's a great compositor too. Nuke released a free version. People use these types of approaches with compositing, roto/masks, tracking , patch repairs all time in restoration work. Roto is tedious work - But having a rock stable track and stabilization will cut the amount of work down significantly.

So are you saying that Blender, or MochaAE, or something else works better, and can truly achieve "rock stable" with the same clips on which Mercalli fails? If so, that is news to me. I wasn't aware that either of these programs could achieve better results with the clips I demonstrated.

I'd like to see some results!!

I have shown results, have linked to the guide I wrote, and have referenced the extensive scripts I have written to automate the "deshaking" process. All I hear in return is that sometimes motion stabilization fails, or that it doesn't stabilize down to the sub-pixel level.

Tell me something I don't already know.

Having written that Deshaker guide and spent several days writing and improving those automation scripts, and having spent dozens of hours stabilizing clips with Mercalli (and MotinPerfect and Steadyhand, and Twixtor), I am well aware of their limitations, and even included many of those limitations in the Deshaker guide itself, should you care to read it.

I don't understand any of these posts. It is as though any attempt I make to point out a feature or advantage that one tool might have over another is met with a claim that I am hyping the product or making claims of perfection that don't exist. Good grief, give me a break! Of course none of them are perfect, and of course they fail on some clips. I would guess that I give up on at least 25% of the clips that I attempt to stabilize. I give up and either use another shot or just live with the unwanted motion because the stabilizer version looks worse due to the artifacts that have been introduced such as blurring, warping, too much zoom, bad edge fill patterns, rolling shutter artifacts, etc.

There, I just listed a whole bunch of ways these programs can fail. Feel better?

Any restoration tool fails to work on some material, and this is no different.

In the end, what point are either of you trying to make? FWIW, I don't have this problem over in doom9.org where I've made several hundred posts over the past 7-8 years. Sure, they'll tell me that my script is written wrong, or that there is a better way to do it, but then they'll offer some code, or give a link to a better result, or try to help me. No one here is posting results or providing information on what technique or product will produce better results. All I get is comments like "The problem with "stabilizing" filters (Avisynth or otherwise) is that none of them really stabilize at all, beyond very minor issues you mostly see on VHS tapes. As far as in-picture content is concerned, it does almost nothing." I added the emphasis there, but that statement just simply isn't true, and the examples I posted, like the bird video (which, once again, was NOT cherry picked, and was done by a user, not by the company), clearly show results that most people would call remarkable, stunning, and amazing.

I am a relative newcomer in this forum, but this exchange doesn't make me want to hang around any more. See ya over at doom9.org.

WOW . Totally relax out john . You seem to be taking things personally.

We're just evaluating different tools. If a tool didn't work for Smurf, it's not YOUR fault. You are not the "tool" - at least I hope you're not

I was just discussing with dogway that there are different types of tools . They have strengths and weaknesses. There are certain situations where some tools are more suitable for. More tools in the tool belt. More options. More ways to help improve results. Complementary tools . That's all.

We are discussing ways to improve results - at least dogway and I were - isn't that a good thing ? Maybe to brainstorm or steal ideas from other software to help improve some avisynth functions. Dogway has already made some improvements to stab with stab2, so I was hoping to keep float ideas to keep the ball rolling . For example, deshaker has a log file. Maybe someone can code up a GUI that allows for user input to modify the values over certain range , that would allow you to override or have more control over problem areas



I can't tell if you're being sarcastic or not or if someone hijacked your account

In that one clip I already said the results were good, just not "rock stable". There is a difference. I don't have that specific source clip so I cant tell you if another method will yield "better" results on that clip. And you don't need "rock stable" in every situation. But for some situations and restoration scenarios (e.g. patch repairs, composite repairs) it's critical . If you don't do that type of work, then please keep an open mind, because other people do use those types of workflows.

I'll say it again, deshaker and mercalli are not in the class of "rock stable" trackers - and they aren't meant to be. That's a good thing because their intended use is different. Nothing wrong with that. To achieve rock stable, the tools used are much slower, require user input, process by scene. You don't use something like mocha to stabilize everything - too slow, too much work. You don't use a screwdriver to hammer a nail in. Wrong tool . Pros / cons

BUT on on other clips and situations, for sure you can improve the results. "auto" anything will have problems - you know that. Even if you could have some user input / user override in mercalli you could improve the results. Even if you can define exclusion areas . All those other methods we were discussing have some type of user input or guidance. If you could salvage some footage and discard only 15% instead of the 25% you discarded wouldn't that be a good thing ?

[johnmeyer]

Poison ...

Yes, a bit of over-reaction. The reason, however, is that it seems like every time I have tried in this thread to have the conversation you suggest we should be having -- exploring alternatives, and developing better solutions -- I keep getting responses that I am making false and misleading claims about the efficacy of various tools, and perhaps even rigging my examples. This included my link to Jerry's bird stabilization example which someone seemed to think was created by someone at Mercalli, and that it was faked, or could only be achieved by "cherry picking" the video.

Those claims are completely wrong. I faked nothing; Jerry doesn't work for Mercalli; and he didn't cherry pick that video clip.

I am always looking for a better tool, and a way to "sharpen" my existing set of tools. Motion stabilization, just like noise reduction or any other restoration technique, will never work in all situations; can not be set to "auto" and then achieve good results; and must be tuned to each individual clip. In addition, each stabilization tool has its own strengths and weaknesses, and its own idiosyncrasies. You have already said those things, and I completely agree.

Finally, the reason I reacted the way I did is that, while others may disagree, I think I have some small claim at being an expert in this field, and have already presented my qualifications for that statement. While I certainly don't expect those qualifications to make me exempt from criticism, I do expect that it would exempt me from claims that I faked my results (I "cherry picked" the video used) or that all the technology that I've spent weeks of my life understanding, using, and documenting, "does almost nothing."

[poisondeathray]

Poison ...

Yes, a bit of over-reaction. The reason, however, is that it seems like every time I have tried in this thread to have the conversation you suggest we should be having -- exploring alternatives, and developing better solutions -- I keep getting responses that I am making false and misleading claims about the efficacy of various tools, and perhaps even rigging my examples. This included my link to Jerry's bird stabilization example which someone seemed to think was created by someone at Mercalli, and that it was faked, or could only be achieved by "cherry picking" the video.

Those claims are completely wrong. I faked nothing; Jerry doesn't work for Mercalli; and he didn't cherry pick that video clip.

I am always looking for a better tool, and a way to "sharpen" my existing set of tools. Motion stabilization, just like noise reduction or any other restoration technique, will never work in all situations; can not be set to "auto" and then achieve good results; and must be tuned to each individual clip. In addition, each stabilization tool has its own strengths and weaknesses, and its own idiosyncrasies. You have already said those things, and I completely agree.

Finally, the reason I reacted the way I did is that, while others may disagree, I think I have some small claim at being an expert in this field, and have already presented my qualifications for that statement. While I certainly don't expect those qualifications to make me exempt from criticism, I do expect that it would exempt me from claims that I faked my results (I "cherry picked" the video used) or that all the technology that I've spent weeks of my life understanding, using, and documenting, "does almost nothing."

Wow.. Nobody is criticising you or your work. Nobody is saying you faked anything. I think you are reading into it too much, or jumping to conclusions. Maybe Smurf pushed the wrong buttons on you and got you all worked up. Take a breath and re-read what is being said. If something is misinterpreted that I wrote - I apologize and hope we can get past this ... I think it's a big misunderstanding !

When I first wrote "cherry pick" in post 35, it was before I read your preceding post (look at the close posting times 5:56, 6:01). It's not a direct reply to your post, otherwise I would have quoted it as I am here. No where is your name even mentioned. Notice I started with "dogway..." When I went back and read your post the next day , I thought wow...So I can see how it could be misconstrued to look like a direct response to your post. So frantically I wrote something clarify what I meant by "cherry pick" in post #40, you might have missed it, the 3rd and last paragraph "BTW...". It might have been a poor choice of words, that's why I wanted to clarify and posted the edit so there was no misunderstanding

I don't for a second believe you intentionally cherry picked (maybe prodad does, they have incentive to) . All I'm saying is a balanced impartial review would show both pros and cons. They would show the whole picture (excuse the pun ) . When you read something like a camera or car review - you want to learn about strengths and weaknesses - at least I would. If you read the post I'm saying the typical results posted online are skewed - they show only strong positive results. Nobody remarks about "so-so" results. There is bias there. So people looking at these might come to the wrong conclusions, or at least not have the whole picture. I'm qualified to say this because I've been using it for a few months. You do get fantastic results on some clips. Not so much on others.

Me - I get worked up over the term "rock stable". It's used inappropriately by prodad . It's a very special term reserved for something else. I'll try to post some examples and more description about the differences when I have time. I've posted examples demonstrating the differences before, but they are scattered around . I'll try to round a few them up

Anyways, I hope that clarifies things and hope everyone can move on

Cheers

[xekon]

I have a question about the stab2() function. working with some Film anime that uses cel animation, there is a small amount of jitter I believe is the term, where the picture moves only a very small amount. I see that dxmax and dymax is the max amount of deviation in pixels.

does this function stabilize the image to the nearest pixel, or will it stabilize to a half or even quarter pixel? or is the image always shifted in whole pixel increments?

I am also wondering if I can limit dxmax and dymax to half a pixel? 0.5? the function only seems to accept an integer... I don't want to stabilize the image too much, because there are scenes where the picture intentionally moves, like during earthquakes or falling trees.

[lordsmurf]

I have a hunch that lordsmurf wanted simply to bomb my thread for some reason (mind the tone in his first post), so far he hasn't supplied a sample either.

I'm not really sure what you mean here. I found this thread while looking for some more advanced stabilization methods. I routinely visit different sites and forums, to see what's new. I've been doing that for about 15-20 years now.

At first, stab2() didn't work. You mentioned XP was the issue. Problem solved.

Samples are on my to-do list. I mentioned this a few posts back.

I don't understand any of these posts. It is as though any attempt I make to point out a feature or advantage that one tool might have over another is met with a claim that I am hyping the product or making claims of perfection that don't exist.

I found this to be an extremely productive conversation. Because of you, I now have Mercalli.

I do think that you're being a bit too forgiving, in terms of the abilities of each method. It's a bit too much hyperbole for me. But I can understand your enthusiasm, as you've been around a long time. I'll easily admit that what we have now is amazing compared to what we had each just 5 years ago, nevermind the 15-20 that I've been around.

Don't go and hide at doom9. Your opinions/info is valued here as well.

Me - I get worked up over the term "rock stable". It's used inappropriately by prodad . It's a very special term reserved for something else.

I don't blame you one bit. I get annoyed by the layman use of the term "jitter" (meaning up/down vertical motion noise). Jitter is a reserved term that is related to the timing. That's a pet peeve of mine.

A similar concept is what led to the stupid myth that VCD was "VHS quality", due to "resolution" being used interchangeably with "lines of resolution". The 240 lines and the 352x240 resolution was therefore "the same" to video dullards net-wide. Still, to this day, people consider VCD to be "VHS quality" (or VHS to be "VCD quality") when in reality the VCD was exactly 50% of the actual quality of the analog tape -- both interframe and intraframe.

So yeah, call them to the carpet for this.

[Dogway]

xekon: It uses subpixel precision, actually shifts of 1px are rather strong.
I don't recommend you to use very low dxmax or dymax, test around but maybe lower than 4 isn't recommended, it's nice to have some leeway since what exactly dxmax stands by is not fully documented. Anyway big fixes (higher than 2px) will be replaced with original if you enable FixFalPos=true.

Don't worry on using higher integers than what you aim to, if the algorithm is prone to fail, it will fail regardless. For example I just tested on the sample I linked on the previous page, there using dxmax/dymax of 1 doesn't improved the already failed attempt at stabilization. Upscaling to double the size and running it again (effectively behaving as if dxmax/dymax was 0.5) didn't make any change either. If you have some problematic areas after using FixFalPos=true simply trim out the scenes.

[poisondeathray]

*Warning: long post


Well I'm an idiot - Mercalli calls it "rock steady" not "rock stable". Both John and Dogway used "rock steady" as Mercalli does. There is a difference. "Rock steady" is fine - it does not imply "no motion" or "locked off" as if it were shot on a tripod. It's not a reserved term. Everything is ok now, at least for me. Whew!

But the point is still the same - no camera motion, vs. some motion. For stabilization, deshaker/mercalli isn't in the "rock stable" category. They tend to drift over time. They can achieve those results sometimes, but they aren't meant to. The other category locks into a reference frame - everything refers back to the "poster" frame. That's why they only do it scene by scene and can be a PITA. They aren't meant to be used for general use.

"Rock stable tracking" is a broad term. It doesn't necessarily have to include any stabilization (you can track but use it for other purposes), but stabilization has to include tracking. It just means the method of tracking or analyzing the target (whatever the target might be) is close to perfect, no visible slippage. "Rock stable tracking" implies it would have been suitable for compositing.

"Rock stable stabilization" means "as if" the camera was fixed and locked off on a tripod. No camera motion. All the frames in the sequence align up. That's why the bird clip isn't considered "rock stable." (John, if you want to upload the original clip, I can look and see if it's possible.) Obviously this term doesn't apply to some shots when there is supposed to be camera motion - e.g. maybe you have a dolly /tracking shot, or crane, or flythrough with a drone. If you have rock stable tracking data, you have the option to make it less stable if you want to (e.g. maybe you want to reintroduce some natural camera motion, or reintroduce camera rotation from a fixed point like a tripod; maybe for a pan you want to eliminate y-axis movement only, etc...). "Rock stable stabilization" also implies it would have been suitable for certain types of compositing.

The point of this post isn't to put deshaker or mercalli in a bad light. I've already described what they can do (and can't do, and it would be wonderful if they could add more control). The point of this post is to be able to use other tools to supplement your workflow, maybe show some alternative solutions for fairly common problem areas that deshaker/mercalli and others in that class have with rock stable (no camera motion) stabilization, as if it were shot on a tripod.

Tracking and stabilization go hand and hand, but for the purposes of discussion, let's ignore for now the other uses for "rock stable" tracking for restoration and other scenarios - and just look at "stabilizing" the general overall picture.

Even if nobody finds it useful, at minimum this can serve as a listing for some test clips for farther development of avisynth depan methods or possible deshaker mods.

These are "cherry picked" on purpose because the are prototypical / representative examples where deshaker or mercalli might have problems with. There are enough examples online where you see mercalli breeze through showing good results. These are real situations, from older posts where people had problems - that alone means its not a random sample - people don't post at videohelp if everything is working fine, only problems are posted.


1) This was mentioned earlier but a common scenario that skews deshaker or mercalli results - the large object moving through the scene. Maybe some kid partially runs in front of the camera. Everyone that has used those will know what I'm talking about. Sometimes it might not be enough to completely ruin the shot, but it still affects it and skews the results over a section. You can exclude it from the calculations using the other applications with a holdout mask. Keyframable, ability to do add multiple masks, full control. You can even motion track them so you reduce the amount of manual work. Essentially you're defining what to track and what not to track. I shouldn't have to post an example for this one, this one is clear cut. This is what I mean by user intervention , and manual, not auto - in terms of the actual tracking. This is why the other class of tools can get rock stable results

Now there is an "Edius" version of mercalli instead of the standalone . It supposedly has additional functionality like but I don't know if it's "enough" to address this type of situation. It has something like overlays, but I don't know if you can edit the tracking data in the host application. Someone that uses Edius and Mercalli version please chime in here

Interactive overlay charts for uncovering "problem zones" in the video as an aid for manual adjustment as part of maximum video optimisation

2) Think of maybe an indoor shoot, a multi cam shoot. Maybe a wedding or interview piece. Someone nudges a camera, or maybe there is some vibration or loud music, maybe some stomping or dancing - slight ground motion that makes picture unstable. Maybe you're using a cheap tripod on one of the cameras. Deshaker/mercalli might be able to get close, but it usually won't be rock stable . You can usually salvage it with one of the other methods, or at least get better results .

Here is an example of a faulty tripod.
https://forum.videohelp.com/threads/351502-CS6-Premiere-Pro-Warp-Stabilizer-Does-not-work-for-me-yet

This poster was complaining about warp stabilizer. It was supposed to be "rock stable" because it's a tripod shot.

You can read the discussion and short explanation about foreground/background , "rock stable" tracking, situations where a stabilizer might "fail", what types of shots predispose them to problems, and when you might use a planar tracker etc..

The test source clip is in post #4
https://forum.videohelp.com/threads/351502-CS6-Premiere-Pro-Warp-Stabilizer-Does-not-wo...=1#post2205048

Mercalli is mentioned there too (I used V2 , but never upgraded to V3, because it didn't seem to be much better than V2. But V4 - the rolling shutter improvement alone is worth it) .

But it looks like I didn't upload the mocha processed clip in that thread , so I will upload that below with mercalli V4 below with full stabilization (rock steady) .

2.1_mocha_default.mp4
2.2_mercalli_v4_rock-steady.mp4

The mocha clip is using default track settings, no special adjustments, nothing tweaked, just plain zoom for the borders (no border fill options enabled). Both are denoised with smdegrain to keep the filesize down. There is about 1/2 pixel flutter, most noticable at the top border. But it was already in the source clip, and partly due to the noise characteristics (the OP didn't upload the original, original). You could improve it more with other processing is apply temporal stabilizer to the background (use a mask around the foreground singer). Because of the depth of field, you won't be "damaging" that many details anyway.

The key is knowing what to track, and what usually works or doesn't work. It's something you gain with experience. All this entails here is you click and draw a shape or two on the surface of a plane which defines it as a "flat surface". If you have no flat surfaces, you can't use mocha, you would use another method. In this case I drew 2 boxes (8 clicks) around the two white dots on the pillars in the top left corner. Then you push track. The speed of tracking depending on the options set. This example tracked faster than deshaker, but slightly slower than mercalli, mainly because the search area is very small (those two dots) . Usually it's much slower, because there is a lot more going on, larger search area, more complexity. The pro version has stabilization module built right in (you can export directly), but the AE version requires another step, opening it up in AE, copy /pasting then exporting.


3) Another fairly common one is somebody accidentally leaves camera on autozoom. Surely you've had a neighbor, or friend asking if you can fix it for them . This type of motion will "fool" most stabilization algorithms.

Here is a typical autozoom clip from another thread
https://forum.videohelp.com/showthread.php?t=342525

The original is sample.mpg
https://forum.videohelp.com/threads/342525-how-to-make-a-video-stickin-to-a-certain-vie...=1#post2133930

Original is interlaced. "3.1_autozoom_qtgmc_lagarith.avi" has been bob deinterlaced with QTGMC and uses lagarith comression - so that everyone is on the same starting page and we avoid any interlace treatment inconsistencies . So treat that as the "original", but feel free to play with the original, original. Ignore the aspect ratio issue for the lossless intermediate (1:1)

3.1_autozoom_qtgmc_lagarith.avi
3.2_mocha.mp4
3.3_Mercalli_rocksteady.mp4
3.4_Mercalli_universalcam.mp4

You can try to tweak it all day with mercalli, but you'll never get anything useful. The underlying problem is there is no way to tell it what to track, what not to track

Mocha isn't as straight forward in this one. I used 2 larger "boxes" too, on the left and right curtains. But as his right hand moves up, that will dislodge/interfere with the right side, so a holdout matte is used to "block" it - basically you're telling mocha to ignore his hand. Same thing with this clip - you could improve it by using a temporal smoother with foreground mask.


I don't want to derail dogway's thread. It's supposed to be about the avisynth stabilization tools pack. But I can post some more examples, different scenarios, "rock stable" sometimes has additional benefits in the stabilization scenario - it usually implies less zooming in (or black borders , or cropping). On some types of shots , it also means you can "patch" together the background peripheral edge pixels from other frames into a larger "canvas" i.e. essentially an edge filling method. Sort of the same idea as deshaker's past/future frames, but without as much blur or mismatch, because the tracking is rock stable and pixels match up. You can't do that if your track is drifting - the frames don't align up properly.

If you read my posts, they almost always say something like "it depends on the shot characteristics". It might sound like a cop out, but it really really does. I can't emphasize it enough - how you treat a clip , what tools and techniques you might use, what and where to track, all depend on how it's framed, camera motion, foreground object motion, background object motion, specific type of motion etc...

Can you improve every shot significantly? No - as mentioned earlier some shots are not "fixable". And sometimes the improvement is so miniscule it might not be worth it. The sole purpose of tracking software is to get accurate tracks. They have the tools to specify what is being tracked or not tracked. It shouldn't be a that much of a stretch of the imagination that you could improve at least some deshaker or mercalli processed shots

[poisondeathray]

On frame warping I have been tackling the issue for some time now. What works best IMO is layering different temporal denoisers on the highest frequencies, sometimes with masking and whatnot, but it's not perfect as anything is.

This is what I use as well, with layers, masks, roto etc..

You can get fairly stable in terms of x,y with one of the trackers like AE's point tracker, but you have to apply something to get rid of the warping . There is no easy fix for this. At least on animated/ simple cartoon sources, you have more room to go overboard with temporal smoothing. With live action source warping, there is too much loss of detail

There are fixes for warping in other programs such as dedicated restoration suites, but it very user intensive. "Warping" in general is one of the most difficult problems.

[xekon]

I don't blame you one bit. I get annoyed by the layman use of the term "jitter" (meaning up/down vertical motion noise). Jitter is a reserved term that is related to the timing. That's a pet peeve of mine.

LOL! well don't do me any favors or anything by letting me know the correct term, I just played the video back on a couple episodes and 95% of the corrections that Stab is making are verticle shift, so im going to assume I did use the correct term, but there are also a small amount of frames where it makes a correction horizontally as well, im going to assume on those frames its because the frame was not correctly aligned in the film.

So I am guessing my issues are mostly jitter, but also something else? another term i don't know.

[Dogway]

poisondeathray: Yesterday I was watching a video of Mercalli IIRC, it had very astounding examples on fixing jello/wobble and rolling shutter. I think it uses a broad brute force, like mapping a 2D/3D grid on plane and fix according so probably it can work for the anime type of wobble (maybe film gate wobble at scene change too?).

My biggest concern is automatic lock-on/rock-steady stabilization. We have tons of automatic global motion deshakers, but no no-motion stabilisers... only stab()? I simply want a better (in all terms quality and commodity) automatic stabiliser. Correct me if I'm wrong but all the standalone or NLE solutions of rock-steady stabilization require scene base filtering, is it right?

[poisondeathray]

poisondeathray: Yesterday I was watching a video of Mercalli IIRC, it had very astounding examples on fixing jello/wobble and rolling shutter. I think it uses a broad brute force, like mapping a 2D/3D grid on plane and fix according so probably it can work for the anime type of wobble (maybe film gate wobble at scene change too?).

Yes it's incredible on most clips for jello - at least compared to other solutions before. I don't know the exact mechanism it uses, but it's worth looking into more detail, to see if we can apply similar technique maybe in avisynth or other programs. It does take much longer to process when you have the rolling shutter options turned up, so it probably does use some type of brute force

I'm liking your train of thought . I'll play with it and see if it can help out

I know you've posted those types of clips before (at doom9, 10, maybe some here), but do you have a couple others handy ? You posted one earlier in this thread, but a few more test samples gathered up in one place (or links to them) to test some theories/approaches would be useful

My biggest concern is automatic lock-on/rock-steady stabilization. We have tons of automatic global motion deshakers, but no no-motion stabilisers... only stab()? I simply want a better (in all terms quality and commodity) automatic stabiliser. Correct me if I'm wrong but all the standalone or NLE solutions of rock-steady stabilization require scene base filtering, is it right?

Can you clarify what you mean by "scene base filtering" ?

NLE solutions are typically not VFX rock stable - they typically fall under the smoothing motion category

If your question is about VFX rock stable (stable, not steady ) . They all do it scene by scene, require some user input to define inclusion / exclusion areas - so they are not "automatic" in that sense. Once the tracking is setup and underway, it's "automatic"

[Dogway]

Do you mean that there is no other filter like stab()? (scene change aware automatic rock stable filter), not even near? Well, that's a pity... this justifies all the talk we are having here due to lack of solutions.

I don't know if you can tell me but johnmeyer said over the DBSC thread that he used to import scene change markers to vegas. I ignore if the same can be done on these other stabilization solutions to automate rock stable filtering, or if in the other hand it's just a shorcut to setup manual filtering by scene. On a related note I found yesterday a useful tool by Chikuzen for avspmod to import scene change as bookmarks, it uses x264. Cool stuff.
Also found useful macros by tp7 to deal with ReplaceFramesSimple or ClipClop replacement strings, here the article, and here the updated macros.

What do you need, false-positive bound trembling, jitter, (film gate) wobble, etc? Give me a day to collect them all, it can be of huge help to see how they all react to these seemingly impossible to fix artifacts.