Deshaker problem

[2Bdecided]

Deshaker is designed primarily for camcorder videos. stab() is designed primarily for film gate weave.

In fact stab() uses very similar (though probably not as advanced) mechanisms as Deshaker, but carefully limited over the kind of range you find in film transfers. You can enforce those same limits in Deshaker, but I've never seen the point when stab() is single pass and much quicker.

VirtualDUB, which uses Deshaker, is free. As is AVIsynth, in which you run stab().

Cheers,
David.

[timsky]

Putting the clip through Deshaker twice with the same settings gets it almost perfect! Here it is. Is there any disadvantage in putting it through twice?

[Mephesto]

Here's the best I could do with both Stab() and Deshaker. Stab had to be fine-tuned on certain frames where it failed to alleviate the violent shake (frames 95-97).

You decide which looks better. Deshaker slowly zooms in but Stab is less advanced especially on this footage that has morphing so it couldn't tell for sure which direction it was panning.

[timsky]

To me the deshaker one looks best because it lacks that jerk. Both are much better than I can achieve with just one pass. What settings did you use?

[Mephesto]

My stab output doesn't have jerk but it does have wobbling only noticeable when you increase the framerate to 60, it doesn't look at frames in the long term like Deshaker. What I wish I could get deshaker to do is have 100% smoothness with the zoom like it does with panning. Your deshaked output didn't have long-term zoom but it still had smoothed shaking. My settings:

[poisondeathray]

Mephesto - you should tell him if you applied any pre processing (denoising, cropping, resizing), or did you only do that post deshaker ?

What happens if you uncheckmark "detect zoom" for the analysis pass ?

Or you might be able to manipulate the log file zoom values to at least reduce the amount of zoom

[Mephesto]

Mephesto - you should tell him if you applied any pre processing (denoising, cropping, resizing), or did you only do that post deshaker ?

Right, cropped 240 pixels from both sides, denoised, resized to 960x720 and ran deshaker on that. I don't have the memory to stabilize the full resolution.

What happens if you uncheckmark "detect zoom" for the analysis pass ?

Not sure, I doubt very much.

Or you might be able to manipulate the log file zoom values to at least reduce the amount of zoom

Here's the thing, it has to zoom anyway to hide the borders. The reason it zooms noticeably by the time you get to frame 240 is because around frame 95 the frame jerks down about 4 pixels which is approximately how much frame 240 is zoomed compared to frame 1 in every direction. Because the zoom smoothness has been set to infinite, it gradually zooms from 1440x1080 to 1430x1070 in a linear sweep across the 240 frames.

What I wish is that it zooms as much as it needs to right away and stay like that the whole time. The borders are a mess anyway.

[poisondeathray]

Or you might be able to manipulate the log file zoom values to at least reduce the amount of zoom

Here's the thing, it has to zoom anyway to hide the borders. The reason it zooms noticeably by the time you get to frame 240 is because around frame 95 the frame jerks down about 4 pixels which is approximately how much frame 240 is zoomed compared to frame 1 in every direction. Because the zoom smoothness has been set to infinite, it gradually zooms from 1440x1080 to 1430x1070 in a linear sweep across the 240 frames.

What I wish is that it zooms as much as it needs to right away and stay like that the whole time. The borders are a mess anyway.

Yes, the gradual zoom isn't too bad to watch . It might be problematic if he's stabilizing to do compositing restoration work , but he said it's fine for his purposes

If you want to set a fixed zoom right away, why not leave borders, then manually zoom (scale up & crop) by your fixed amount ?

I don't know if the rest of his film has those large deviations, but you can try frame interpolation plugins in avisynth to interpolate over those few "bad" frames

[Mephesto]

I tried doing this without 'detect' zoom' and there was no gradual zoom this time. However, jerkiness is slightly more noticeable now.

Because not all objects on the screen shake in the same direction (warping), one of the ways deshaker reduces that so well is because some of that warp is recognized as slight zooming or rotating. Slightly adjusting zoom or rotation in addition to x-y pans makes fixing warped footage like this more powerful. This is why I always leave 'detect zoom' and 'detect rotation' on even when there isn't any. It helps.

If you want to set a fixed zoom right away, why not leave borders, then manually zoom (scale up & crop) by your fixed amount ?

It will still zoom, but will start less cropped, I tried that.

[timsky]

I tried to run it on your settings but it crashes every time using Deshaker in VideoPad. I downloaded VirtualDub but it wont accept 1920x1080 MP4 files, and Premiere Pro CS4 wont export in AVI 1920x1080 p24

[poisondeathray]

I tried to run it on your settings but it crashes every time using Deshaker in VideoPad. I downloaded VirtualDub but it wont accept 1920x1080 MP4 files, and Premiere Pro CS4 wont export in AVI 1920x1080 p24

You can import MP4 into vdub with the ffinputdriver plugin, or directshow input driver , or avisynth script

PP CS4 can certainly export in 1920x1080p24 through AME

But what is your original format you got from the lab ?

[timsky]

I will look at those plugins later. Just to report that using the default settings in Deshaker except with -1 in horiz pan, vert pan, zoom and rotate produces an almost perfect stabilization, so it is obviously all those -1 settings which are so important.

[poisondeathray]

For reference, this what I mean by "rock stable" motion tracking with mocha (there is a "lite" version that comes with after effects). Because the motion data is precise, you can fix borders more precisely (so don't have to zoom in as much and make picture softer, or change field of view as much), remove various film defects more effectively, clean up light leaks more easily, composite/patch things correctly instead of having them fall apart.

1st one is stabilized with fixed zoom , and denoised
2nd one is the same motion tracking, but degrained, border fill instead of zoom, some fixes like patching a few problem areas (easier because it's rock stable) , regrained

If you compare with mercalli or deshaker results, you will notice jitters and zooming that shouldn't be there . Mercalli falls in the same category as deshaker - both aren't meant to give "rock stable" results - just to smooth out handheld camera motion . Even though the Mercalli has a "rock stable" option setting - the results aren't what I'd call "stable" enough for VFX compositing / tracked patch restoration work . Stab is better for slight jitters and film gate weave in my experience than deshaker, but does not give "rock stable" results either. Small pixel deviations mean your patches will fall apart and won't "stick" to your composite.

Some of the trees and buildings are "pumping" or "rippling" in Mephesto's example ; I think they might be from denoising defects, not deshaker artifacts, because they are clustered in localized patches . But sometimes it can indicate zoom fluctuation issues (compensation for zooming in/out , or on other shots motion blur differences between frames , causing increase/decrease blurring as the objects stay in the same position)

[timsky]

Both look good to me. Another mystery: my unprocessed clip is 39.1MB, but the clip when stabilised is only 4.88MB. Why?

[poisondeathray]

Another mystery: my unprocessed clip is 39.1MB, but the clip when stabilised is only 4.88MB. Why?

Mostly it depends on the export settings used (what bitrate or quantizer), codec used . If you use a higher bitrate or lower quantizer, the quality loss will be less , filesize larger. You can even use a lossless codec that will be many times larger than your compressed original . Many people use lossless codecs for intermediates, so they don't uncessarily lose quality multiple times in the workflow . Lossless codecs are typically about 1/2 the size of their uncompressed counterparts .

Grain, noise, movement, sharpness / detail all require more bitrate for a certain level of quality because of the way encoding video data works . So if everything else is equal, a stabilized clip will require less bitrate than unstablized clip . A denoised/degrained clip will require less bitrate than a noisy/grainy one

That's partly why I asked what your original format was.

[timsky]

Well the original format was Quicktime, but that's not the point. The clip is in MP4 1920x1080 p24, but even when I put the clip in deshaker then export it immediately with no stabilizing, it is reduced from 39MB to 4.9MB. Why is it doing this?

[poisondeathray]

Well the original format was Quicktime, but that's not the point. The clip is in MP4 1920x1080 p24, but even when I put the clip in deshaker then export it immediately with no stabilizing, it is reduced from 39MB to 4.9MB. Why is it doing this?

Does the post above make any sense to you? Or shall I try to explain it differently ?

What were your export settings ? What front end are you using ? vdub ?

Ultimately,

Filesize = Bitrate * Running Time

That is a universal equation. So either you're using a lower bitrate, or maybe something crashed and the running time isn't the same (maybe it rendered fewer frames)

[timsky]

In the export settings I changed the Quality Rate Factor from the default figure of 23 to 1 (highest quality). This has changed the file size of this clip from 10MB to 190MB! (the original was 19MB). So I now have no idea how I should set it.

[poisondeathray]

In the export settings I changed the Quality Rate Factor from the default figure of 23 to 1 (highest quality). This has changed the file size of this clip from 10MB to 190MB! (the original was 19MB). So I now have no idea how I should set it.

It depends how you're going to use it . That usually determines what you would use, what settings you use, what format /codec you use

For example , is this intended as a final delivery format e.g. DVD, Blu-ray, DCP, Web ? Devices Ipad/Iphone ?

Is it intended for farther editing ? In another program?



Also , what codec are you using ?

If you don't know what codec is being used, use mediainfo, view=>text, copy & paste the results of the export back here

[timsky]

The film is basicaly finished, but I intend to put the whole film through deshaker, then remove the pan shots and any others which have not come out well after deshaking with the rock steady settings, and redo them with other settings and re-insert them. The final version will be transferred to Blu-ray disc and shown to friends etc, and needs to have maximum quality. The codec of the film is MP4 and I don't see any reason to change that?

[poisondeathray]

MP4 is a "container" .

Containers can hold many different types of audio & video compression , accessory streams

Blu-ray has restrictions on what you can use. You can't use any old regular settings - it won't be compatible

[timsky]

Well I think it's H.264 in a MP4 container

[poisondeathray]

You should probably start another thread, because "blu-ray compatibility" is a big topic, separate from deshaker

Well I think it's H.264 in a MP4 container

That's about as generic as saying "I think it's a car" . Well there are blue cars, red cars, Honda's , Fords, SUV's, Sports Cars etc....

There are many different profiles/levels (ie. "varieties") of h.264/AVC . Only a tiny subset is compatible with blu-ray

If it's meant for strict reproduction, strict authoring - that entails very strict requirements. h.264 in mp4 is not compatible (you can't use any container, only raw avc streams when doing strict authoring)

Conversely, if it's a "homebrew" project you're doing yourself, it might not play in some players but ok in others . There is a "continuum" of strict, to relaxed settings and workflows

How are you "transferring to blu-ray" - what software or services are you using ? What authoring software ? That also determines what options you have

[timsky]

I'm using PPCS4 and the export setting is H.264 Blu-ray and it works

[poisondeathray]

I'm using PPCS4 and the export setting is H.264 Blu-ray and it works

That would work .

But how are you getting from deshaker to PPCS4 ?

How are you authoring the BD ? Encore CS4 ?

[timsky]

How are you authoring the BD ? Encore CS4 ? No exporting it to media, then burning it with Imgburn. Sometimes I use Encore.
But how are you getting from deshaker to PPCS4 ? I assumed I could simply import it into PP, but you're going to tell me it's not that simple?

[poisondeathray]

I assumed I could simply import it into PP, but you're going to tell me it's not that simple?

This is the part where your desire to "needs to have maximum quality" comes into play

If you truly want that, then you use a lossless codec. If the filesize on that sample was 190MB with a quantizer of 1 with any codec, it will be even larger than that using a lossless codec

Each time you re-encode with a lossy codec, bits of data are thrown out. Whenever you import between programs - those are stages where avoidable quality loss can occur

For example , if your original, original quicktime MOV was edited in PP and then you exported using h.264 blu-ray preset for use in deshaker - that's avoidable quality loss that you should avoid. You should be using losslessly compressed intermediates, or uncompressed. These are large data rates we're talking about here and you need plenty of HDD . But that's where you decide if you want to make any trade offs in terms of quality. For some people, "near lossless" is "good enough" . Maybe something like cineform, dnxhd, prores. Others insist on a lossless workflow

[timsky]

The quality of the original film is not that wonderful, so I'm not really worried about slight changes in color etc, so near lossless or even "not too bad" will probably do. As long as the motion is smooth and there's no loss of definition thats all I really care about. So I just need a rough idea where to set the Quality Rate Factor slider.

[poisondeathray]

The quality of the original film is not that wonderful, so I'm not really worried about slight changes in color etc, so near lossless or even "not too bad" will probably do. As long as the motion is smooth and there's no loss of definition thats all I really care about. So I just need a rough idea where to set the Quality Rate Factor slider.

Between 0-51 ?

Seriously - only you can answer that . Everyone has different opinions or eyes for "quality". What you see as no "loss of definition" somebody else will see the quality loss. Take a look at when it was 1, and when it was 23 . If those aren't good enough, pick something in between. It's going to look a bit worse if you re-encode it again using premiere / ame

If you frameserve out of vdub, you can avoid that extra stage of loss, you can avoid using large lossless intermediates

[timsky]

I'm using VideoPad. I'll try what you suggest and see if I can see any difference between 1 and 23. What sort of losses would you expect to see with high QRF numbers? Bad color? Loss of contrast? Bad definition? I assume however low the quality is, it will still be in 1920x1080? I'm going to bed now.