Video stabilizing problems (software stabilisation)

[Bencuri]

I have tried to stabilize some of my video files that I liked but lacked good stabilization by the camcorder. I used Mercalli 2.0 stand alone stabilizer software. Mostly the results were very satisfying, but there were some videos where the program was not successfull in 100%.

I uploaded an example: the original video and the Mercalli software output, too. My problem is that, in this software, the most important setting is the "pan shot smoothing" bar. It makes most of the work. When I set it low, the image is still shaky. Around 20, the shakiness reduces in case of my videos to a level that is okay for me. However in case of this particular video below (unlike in my other videos), simoultaneously, the picture becomes a bit wobbly here and there. It is not too wobbly, but you can notice it. Can this wobble be eliminated somehow? I also included a screenshot here about how I set the software.

Maybe this software is not enough for this video, and I should use one that is more advanced. Like Virtual Dub Deshaker. If that is the solution, what parameters do I have to care about to get rid of the wobbles?

[poisondeathray]

I However in case of this particular video below (unlike in my other videos), simoultaneously, the picture becomes a bit wobbly here and there. It is not too wobbly, but you can notice it. Can this wobble be eliminated somehow?

The short answer is no , at least not very well

This is CMOS wobble or "jello" or rolling shutter artifacts . Basically it affects all CMOS cameras with slow scan rates. Use those search terms (google "rolling shutter artifacts") if you want more info , it's well documented and there is lots of information readily available

There are partial repairs that can be done in mercalli and other products (e.g. after effects warp stabilizer, foundry rolling shutter plugin) , but none of them do a great job . These products do a better job with linear skew, but poorly on the wobble you get from vibrations or wiggling the camera. The basic reason is, it's impossible to calculate the motion vectors because different parts of the frame are scanned at different points in time (and by that time, the camera has already moved) . For the same reason, linear skew can be corrected fairly well (e.g. during panning, and objects seem to "bend") - because it can be caluculated

[mammo1789]

I don't understand why the original video flickers like crazy is it my pc or what

[poisondeathray]

I don't understand why the original video flickers like crazy is it my pc or what

Do you mean on patterns , like the rooftop , tiles etc... ? That's aliasing

This camera has a low quality OLPF (and even if it has a decent OLPF, the image would be even softer, it's too soft already. )

FH1 is an older model. Panasonic bought Sanyo's camcorder division a few years back. Newer models like TM900 , X900 produce much cleaner footage, less aliasing with higher effective resolution. Details are crisp not "muddy" and "mushy" like this

[Bencuri]

I don't understand why the original video flickers like crazy is it my pc or what

Do you mean on patterns , like the rooftop , tiles etc... ? That's aliasing

This camera has a low quality OLPF (and even if it has a decent OLPF, the image would be even softer, it's too soft already. )

FH1 is an older model. Panasonic bought Sanyo's camcorder division a few years back. Newer models like TM900 , X900 produce much cleaner footage, less aliasing with higher effective resolution. Details are crisp not "muddy" and "mushy" like this

Actually this is subjective, but I don't find that aliasing that dangerous. I also have the HD1000, the FH1 has much less aliasing problem. I also have a JVC GZ-HM300, a Samsung HMX H200, both has more aliasing problem than the FH1. Sanyo definately improved the line compared to the 2007 models with the FH1. And the image sharpness also become better. The HD1000 videos cannot be enjoyed too much on the computer, they are a bit blurry, except for marco recordings. The FH1 is way much better on the computer, I enjoy the colors just as much as on TV, and the image is way much sharper, too, compared to the HD1000. So maybe the Panasonic is still sharper (I saw some samples), but this Sanyo is not that much bad, either. Personally I feel regerding colors, Sanyo is unbeatable. I have a Canon HF M406, too, the colors are not this nice. On the FH1 videos, the sky has nicer colors, for example. On the Panasonic SD90, the sky is not as nice to me, either, it looked similar to the Canon, kind of boring. The Canon has more moderate colors, compared to the Sanyo. Hs kind of a general Home Video feel. If you set it vivid, it will be to warm. When I saw Panasonic samples, that had more interesting colors, but the video seemed a bit sterile to me. In the FH1 videos, I like that they are balanced between a moderate and vivid setting, has good detail. If you record a video of an orchestra, you can see the notes on the sheet music from the distance. This is not bad, regarding sharpness. Considering the techinal side, Sanyo is not advanced, but colors are very nice on this camcorder. I rarely find camcorders with such inspiring colors. They are usually too moderate or sterile. One exception is the JVC GZ-HM1, I would love to own that.

[Bencuri]

I don't understand why the original video flickers like crazy is it my pc or what

Maybe you have some problem with the playback. I have such a problem with the Sanyo HD1000 videos on one of my laptops. They are flickering. But on TV, not. I don't know why.

Maybe you have the problem because the video is 60 fps. No?

[Bencuri]

I However in case of this particular video below (unlike in my other videos), simoultaneously, the picture becomes a bit wobbly here and there. It is not too wobbly, but you can notice it. Can this wobble be eliminated somehow?

The short answer is no , at least not very well

This is CMOS wobble or "jello" or rolling shutter artifacts . Basically it affects all CMOS cameras with slow scan rates. Use those search terms (google "rolling shutter artifacts") if you want more info , it's well documented and there is lots of information readily available

There are partial repairs that can be done in mercalli and other products (e.g. after effects warp stabilizer, foundry rolling shutter plugin) , but none of them do a great job . These products do a better job with linear skew, but poorly on the wobble you get from vibrations or wiggling the camera. The basic reason is, it's impossible to calculate the motion vectors because different parts of the frame are scanned at different points in time (and by that time, the camera has already moved) . For the same reason, linear skew can be corrected fairly well (e.g. during panning, and objects seem to "bend") - because it can be caluculated

Maybe I don't have all the knowledge needed, but are you sure you didn't misunderstood something? I don't say this to attack you, I just really want to make sure, because I am now confused. This wobble I write about is not present in the original video. It only comes alive when I set the stabilizing effect higher than 10-15 on the slider. My usual setting is 20, at 20, the wobble appears in this video, in other videos, not. Isn't it possible that this is not a rolling sutter problem in this case (as the original video is okay, just shaky), but that the stabilizer can only corrigate the shakiness in a way that it fixes the image, but due to the correction the image starts to wobble? So the wobbling comes from the work of the stabiliser, and not because of the rolling shutter? The images on the frames are convergated to each other, and from the difference between them, the result is the wobble. Hmm? Explain this to me if I am not right.

[poisondeathray]

Actually this is subjective, but I don't find that aliasing that dangerous. I also have the HD1000, the FH1 has much less aliasing problem. I also have a JVC GZ-HM300, a Samsung HMX H200, both has more aliasing problem than the FH1. Sanyo definately improved the line compared to the 2007 models with the FH1. And the image sharpness also become better. The HD1000 videos cannot be enjoyed too much on the computer, they are a bit blurry, except for marco recordings. The FH1 is way much better on the computer, I enjoy the colors just as much as on TV, and the image is way much sharper, too, compared to the HD1000. So maybe the Panasonic is still sharper (I was some samples), but this Sanyo is not that much bad, either. Personally I feel regerding colors, Sanyo is unbeatable. I have a Canon HF M406, too, the colors are not this nice. On the FH1 videos, the sky has nicer colors, for example. On the Panasonic SD90, the sky is not as nice to me, either, it looked similar to the Canon, kind of boring. The Canon has more moderate colors, compared to the Sanyo. Hs kind of a general Home Video feel. If you set it vivid, it will be to warm. When I saw Panasonic samples, that had more interesting colors, but the video seemed a bit sterile to me. In the FH1 videos, I like that they are balanced between a moderate and vivid setting, has good detail. If you record a video of an orchestra, you can see the notes on the sheet music from the distance. This is not bad, regarding sharpness. Considering the techinal side, Sanyo is not advanced, but colors are very nice on this camcorder. I rarely find camcorders with such inspiring colors. They are usually too moderate or sterile. One exception is the JVC GZ-HM1, I would love to own that.

RE: Colors - Sure it's subjective. But the main point is you can adjust this in post. You can grade it however you want with any camera when you are editing the footage . You can make it more saturated, fix the midtones, whatever you want, etc...

But something like low details, mushy footage - you can't fix that except get another camera . This isn't 1080p footage. The effective resolution is much lower. You can scale this footage down to 720p and back up to 1080p without noticing any quality loss - because there are no fine details to lose. But on "real" 1080p footage, you will notice quality loss, fine details missing. Colors should be the least of your worries IMO

[poisondeathray]

Maybe I don't have all the knowledge needed, but are you sure you didn't misunderstood something? I don't say this to attack you, I just really want to make sure, because I am now confused. This wobble I write about is not present in the original video. It only comes alive when I set the stabilizing effect higher than 10-15 on the slider. My usual setting is 20, at 20, the wobble appears in this video, in other videos, not. Isn't it possible that this is not a rolling sutter problem in this case (as the original video is okay, just shaky), but that the stabilizer can only corrigate the shakiness in a way that it fixes the image, but due to the correction the image starts to wobble? So the wobbling comes from the work of the stabiliser, and not because of the rolling shutter? The images on the frames are convergated to each other, and from the difference between them, the result is the wobble. Hmm? Explain this to me if I am not right.

The "jello" is already in the original footage . It's just made worse by the software. Watch it again, or look at it frame by frame - you will notice objects wiggling like jelly in the original footage

First you should read up on what rolling shutter is. It's crucial to your understanding. The short explanation is different parts of the frame are scanned at different moments of time. So by the time the middle part is scanned, the camera has already moved. That's why you get skewed objects

When you use a software stabilizer it tries to make the footage more steady. The problem it can only fix the whole frame (move up, down , left, right of a single frame) , not parts of a frame. The rolling shutter artifacts affect portions of the frame. So when it tries to apply corrective action the end result can actually look worse - sometimes it looks like objects vibrate or "phase" in and out , this isn't shown well in your example, but when you process more footage you will see it.

A "real" solution would detect and adjust different parts of a frame. This is almost impossible to do "automatically" with any accuracy, because not only do you have to know scan rates, you have to re-calucate different velocities and motion vectors for different parts of the frame (e.g. and object

[Bencuri]

And what about if I would do a Rolling Shutter correction with a specialised software first, and feed the Mercally stabiliser with that corrected output? Could that make a difference? Only a few of the videos have this issue I described, and even in the example case the problem is just starting to appear, the whole wobble problem is still on the initial, minimal level. What do you think?

[poisondeathray]

Here is an animated gif cropped from the original footage - do you see the objects warping? Look at the pole. The top half moves differently than the bottom half. But it's a single solid object !! It should move together . This is the basis for rolling shutter problems





Stablizers like deshaker, mercalli - they track feature points . They track motion vectors of various objects travelling in sucessive before/after frames. They can get confused if an object moves in left, but suddenly moves the opposite direction, but other objects move in another direction and magnitude. How can the same single solid object move with different velocities ? So the corrective action sometimes makes things look worse - which motion vector should it pay attention to? Which direction is the frame moving in (in relation to the surrounding frames)? It's hard to say because the individual motion vecotrs are all over the place , they are not in accordance with each other. When you factor in zoom (as in this shot), it messes up things even more

[poisondeathray]

And what about if I would do a Rolling Shutter correction with a specialised software first, and feed the Mercally stabiliser with that corrected output? Could that make a difference? Only a few of the videos have this issue I described, and even in the example case the problem is just starting to appear, the whole wobble problem is still on the initial, minimal level. What do you think?

It's impossible to use a generic correction, because it would depend on a specific shot characteristics

If you fix it in this scene with certain settings, those same settings will cause problems in other scenes.

You have to adjust it by scene (and within a scene) , because if the pan/zoom and jiggle characteristics are different, the repair (over)compensation will be different

Again, all the available rolling shutter software only fixes linear skew, not jello very well. The reason being, "jello" from vibrations has multiple directions and velocity motion vectors (different parts of the frame move in different direction and speed) , but linear skew only has 1 set of vectors, and if you know the rolling shutter % of your specific camera (it can be determined and measured with simple tests) it can be corrected for. If your problems were just skew, you could fix it (quite well too)


So we're back to the original answer: "The short answer is no"

[Bencuri]

Okay it seems then I will have to get used to this recent Mercalli output. Actually it is not that disturbing. Other softwares do a worse job. Actually I realized this wobble by watching this video 10-15 times.

[poisondeathray]

Okay it seems then I will have to get used to this recent Mercalli output. Actually it is not that disturbing. Other softwares do a worse job. Actually I realized this wobble by watching this video 10-15 times.

Yes, often the stablization is worse than the original. It's not just Mercalli, all stabilizers have the same problems. But the real problem is the lower scan rate of certain CMOS sensors. Models with faster scan rates mean less jello, less skew . (Or if you had a global shutter, there would be none)

You can adjust the settings, and might get slightly better results, but you cannot use the same settings for the entire thing. You would have to adjust them not only between scenes, but within a scene (whenever there is handheld movement or vibrations you might need custom settings) . When you do compound movements (e.g. coupled with zoom) it's even worse

[Bencuri]

Actually this is subjective, but I don't find that aliasing that dangerous. I also have the HD1000, the FH1 has much less aliasing problem. I also have a JVC GZ-HM300, a Samsung HMX H200, both has more aliasing problem than the FH1. Sanyo definately improved the line compared to the 2007 models with the FH1. And the image sharpness also become better. The HD1000 videos cannot be enjoyed too much on the computer, they are a bit blurry, except for marco recordings. The FH1 is way much better on the computer, I enjoy the colors just as much as on TV, and the image is way much sharper, too, compared to the HD1000. So maybe the Panasonic is still sharper (I was some samples), but this Sanyo is not that much bad, either. Personally I feel regerding colors, Sanyo is unbeatable. I have a Canon HF M406, too, the colors are not this nice. On the FH1 videos, the sky has nicer colors, for example. On the Panasonic SD90, the sky is not as nice to me, either, it looked similar to the Canon, kind of boring. The Canon has more moderate colors, compared to the Sanyo. Hs kind of a general Home Video feel. If you set it vivid, it will be to warm. When I saw Panasonic samples, that had more interesting colors, but the video seemed a bit sterile to me. In the FH1 videos, I like that they are balanced between a moderate and vivid setting, has good detail. If you record a video of an orchestra, you can see the notes on the sheet music from the distance. This is not bad, regarding sharpness. Considering the techinal side, Sanyo is not advanced, but colors are very nice on this camcorder. I rarely find camcorders with such inspiring colors. They are usually too moderate or sterile. One exception is the JVC GZ-HM1, I would love to own that.

RE: Colors - Sure it's subjective. But the main point is you can adjust this in post. You can grade it however you want with any camera when you are editing the footage . You can make it more saturated, fix the midtones, whatever you want, etc...

But something like low details, mushy footage - you can't fix that except get another camera . This isn't 1080p footage. The effective resolution is much lower. You can scale this footage down to 720p and back up to 1080p without noticing any quality loss - because there are no fine details to lose. But on "real" 1080p footage, you will notice quality loss, fine details missing. Colors should be the least of your worries IMO

I thought a lot about buying a DSLR and modifying the colors afterwards. But the problem is that I have no clue about the advanced editing of colors in a video or a photo. I open the panels, there are tons of graphs, a lots of features, and I don't have a clue what they mean. For years I haven't been inspired by the Cameras or Camcorders I had. I saw photos by others here and there that I liked , but I never know by which machine or effect they were made. When I purchased the Sanyo HD1000 and watched the first clip, I felt for the first time I found what I wanted. When I cut photos out of the videos, and modify them a bit with basic things, I get right just that look I always wanted. It became so easy. I have no clue how to get that look with another machine that records a video or make a photo that is different in colors. This is the problem.

[poisondeathray]

I thought a lot about buying a DSLR and modifying the colors afterwards. But the problem is that I have no clue about the advanced editing of colors in a video or a photo. I open the panels, there are tons of graphs, a lots of features, and I don't have a clue what they mean. For years I haven't been inspired by the Cameras or Camcorders I had. I saw photos by others here and there that I liked , but I never know by which machine or effect they were made. When I purchased the Sanyo HD1000 and watched the first clip, I felt for the first time I found what I wanted. When I cut photos out of the videos, and modify them a bit with basic things, I get right just that look I always wanted. It became so easy. I have no clue how to get that look with another machine that records a video or make a photo that is different in colors. This is the problem.

It's quite easy to learn the basics. You don't need advanced knowledge or programs. You can find the same filters on your TV for example (e.g. saturation, contrast, etc...). All editing software has these basic color correction features


There are a more complex things like secondary color correction, masks & rotowork, but if you just like the "colors" approximately similar to what this camera puts out, it's quite simple. If you need it exactly perfectly matching another camera (e.g., for multicamera shoot), that's another story - that's more advanced