Rolling Shutter and Deshaking

[blewyn]

I have a hand-held shot taken with iphone 3GS where I walked along a beach towards a small boat coming in to land.

As I walk towards the boat, the video shakes. I have tried Deshaker, After Effects Warp and Mercalli to stabilise the video. The best results so far are with Deshaker at 60% rolling shutter, however there is still a noticeable 'jello' in the deshaked footage, causing the beach to appear to wobble and the boat mast to whip left and right. I guess the software has trouble compensating for both rolling shutter and fast movement due to my footsteps.

Any ideas as to how I might better stabilise this shot ? I have a couple of other movies shot on the 3GS so I'm keen to get it right.

Is there a program (an affordable one) that focuses solely on rolling shutter removal, the footage from which could then be processed for deshaking ?

[blewyn]

Further to the above, I'm getting horrendous rolling shutter effects on all my iphone 3gs videos when I try to deshake them.

I'm using 55% rolling shutter. Any ideas ? (the footage is 640x480 30.0fps)

[budwzr]

I'm not the resident expert on this, but the jello effect comes from zooming too much to make the stabilization settings you're asking for.

Stabilization is essentially moving the frame up and down, left right, and in and out, to try and keep certain points steady. So you have to avoid cranking up one variable too much.

Myself, I use my NLE to roughly stabilize footage first, to lessen the load.

[blewyn]

I'm not the resident expert on this, but the jello effect comes from zooming too much to make the stabilization settings you're asking for.

There's no zoom. The jello effect comes from the rolling shutter. When you have rapid movement the stabiliser program can't figure out if the changes between each frame are due to actual movement or rolling shutter - especially given that the correction works on a per-frame basis. I think a proper RS correction algorithm needs to work on an intra-frame basis useing shape recognition or something.

Most of my iphone 3gs footage is unuasble after stabilisation....wobbles like crazy. Might have to resign myself to making a few shakycam videos.....

[lordsmurf]

The only way I've seen so far is with the Nuke Foundry plugin Rolling Shutter.
http://www.thefoundry.co.uk/products/rollingshutter/

It can only be bought from a partner: http://www.thefoundry.co.uk/partners/search.php
Prices vary. It's $420 here: http://www.av3software.com/products/rolling-shutter-ae-node-locked

And then you already need to own either Nuke or After Affects.

For $99, there's also this: http://www.newbluefx.com/video-essentials-iii.html
I've not seen or tested that one, however.

[jagabo]

With Gunnar Thalin's Deshaker be sure to enable the rolling shutter option before running the first pass.

[pippas]

I think you'll find that it's only skew, or flashbanding, that can be dealt with by specialised 'rolling shutter' software.
The 3rd common effect - 'jello' - seems to be (understandably!) largely untreatable.

It's also much more of a problem with phone cameras, and some of the early CMOS HD camcorders, like the Canon HV 30, which use pretty slow scan times.

Pain in the neck, CMOS sensors, for anything that involves rapid movement !!.....

[blewyn]

Pippas, I suspect you're right, the algorithm for it simply hasn't been figured out yet. Very tricky because you would need to be able to separate changes in the video due to movement from changes due to the rolling shutter effect due to the movement. I guess some sort of shape recognition will be necessary.

[blewyn]

With Gunnar Thalin's Deshaker be sure to enable the rolling shutter option before running the first pass.

Yep. I don't have my 3GS with me at the mo so I couldn't do the rolling shutter measurements, so I just did a whole set of 12 runs with different percentages. The closest ones - far as I can tell - are 55 and 60%.