Deshaker problem

[poisondeathray]

I'm using VideoPad. I'll try what you suggest and see if I can see any difference between 1 and 23. What sort of losses would you expect to see with high QRF numbers? Bad color? Loss of contrast? Bad definition? I assume however low the quality is, it will still be in 1920x1080? I'm going to bed now.

I don't use videopad. I don't know what encoder you're using (videopad is an editor and probably has accesss to several encoders), but it will affect definition , not really contrast or color. The frame dimesions will be 1920x1080, but that doesn't mean much (1920x1080 can be blurry as hell. 720x480 can look better than 1920x1080 if the 1920x1080 is bitrate starved)

At high bitrates/low quantizers, more fine details are kept, even grain, noise . At low bitrates/high quantizers , fine details will be smoothed away, noise/grain will not be retained. You will start to see macroblocks, especially in dark areas first. Lines will become less distinct , you will get additional compression noise around lines . It's a continuum of quality. From very bad & unwatchable to same as original

[timsky]

Sounds bad to me. I think I'll go for the highest bitrate possible assuming the film will fit on my harddrive. I suppose though that slight loss of definition is equivelent to blurring, so later I can sharpen it without magnifying grain? Or will the low bitrate mean that fine detail is irrevocably lost so there will be nothing left to sharpen? A last question though: What advantage can a file size of 190MB give when the original file size was only 20MB?

[timsky]

I've just done an export at the lowest quality 51 and I see what you mean.

[Mephesto]

For reference, this what I mean by "rock stable" motion tracking with mocha (there is a "lite" version that comes with after effects). Because the motion data is precise, you can fix borders more precisely (so don't have to zoom in as much and make picture softer, or change field of view as much), remove various film defects more effectively, clean up light leaks more easily, composite/patch things correctly instead of having them fall apart.

1st one is stabilized with fixed zoom , and denoised
2nd one is the same motion tracking, but degrained, border fill instead of zoom, some fixes like patching a few problem areas (easier because it's rock stable) , regrained

If you compare with mercalli or deshaker results, you will notice jitters and zooming that shouldn't be there . Mercalli falls in the same category as deshaker - both aren't meant to give "rock stable" results - just to smooth out handheld camera motion . Even though the Mercalli has a "rock stable" option setting - the results aren't what I'd call "stable" enough for VFX compositing / tracked patch restoration work . Stab is better for slight jitters and film gate weave in my experience than deshaker, but does not give "rock stable" results either. Small pixel deviations mean your patches will fall apart and won't "stick" to your composite.

Some of the trees and buildings are "pumping" or "rippling" in Mephesto's example ; I think they might be from denoising defects, not deshaker artifacts, because they are clustered in localized patches . But sometimes it can indicate zoom fluctuation issues (compensation for zooming in/out , or on other shots motion blur differences between frames , causing increase/decrease blurring as the objects stay in the same position)

poisondeathray, a real life neural network machine. Lackeys like myself try to fill in for you to answer newbie's calls to ease your burdens because we'd be still be newbies without your wisdom. But evidently we're always one step behind.

That was amazing work. I need to use a professional video editor for once. I'm always lagging behind with these freeware tools. Without VDub and Avisynth I would've never learned to do anything for myself so I regret nothing about not starting with professional tools but it might be time to take this extra step.

How did you fix that purple rectangle that tinted over the people's heads from frame 220-240? There was another huge square in the sky as well. What did you use to sharpen the video, also something in AE?

[Mephesto]

Sounds bad to me. I think I'll go for the highest bitrate possible assuming the film will fit on my harddrive. I suppose though that slight loss of definition is equivelent to blurring, so later I can sharpen it without magnifying grain? Or will the low bitrate mean that fine detail is irrevocably lost so there will be nothing left to sharpen? A last question though: What advantage can a file size of 190MB give when the original file size was only 20MB?

The original 20MB file is a COMPRESSED file. To re-encode that file would be uncompressing and recompressing it, even to something larger than 20MB but smaller than what it would be uncompressed would destroy some quality.

Think of a piece of paper. It's scrunched up already in a small piece, when you unravel it it'll have wrinkles all over the place, then when you scrunch it up again and unravel again it'll have more wrinkles, even if you scrunch it up just a little bit.

So recompressing your 20MB video into a 40MB video WILL deprive it of quality. You may not notice the loss or care but that's another thing. Best to work with lossless or near-lossless.

[timsky]

If I use quality 1 on the whole film the file size will be 80GB which I may just be able to squeeze on my HD. I suppose quality 51 will not cause noticeable degredation? I tried doing test exports of a clip at quality 1, 10, 20, 30, 40 and 51 and while 40 and 51 look bad, 30, 20, 10 and 1 look the same to me.

[poisondeathray]

If I use quality 51 on the whole film the file size will be 80GB which I may just be able to squeeze on my HD. I suppose quality 51 will not cause noticeable degredation? I tried doing test exports of a clip at quality 1, 10, 20, 30, 40 and 51 and while 40 and 51 look bad, 30, 20, 10 and 1 look the same to me.

Yes, for most sources, 51 is terrible. 30 is terrible. But hey, they're your eyes . High values/ low bitrates might be usable for clean cartoon/animation source, but for grainy/noisy film, it wont work well. Between 16-24 might be usable . But remember, you're re-encoding it again in PP/AME. That will cause more deterioration

I suspect videopad has access to x264vfw . You could export a raw avc stream for import into encore using blu ray compatible settings directly . This means less HDD space, less quality loss , fewer steps, faster. (And x264 is a better encoder than Rovi/Mainconcept from AME/PP) . Or use vdub and frameserve, or the external encoder feature with x264CLI direcly.

[poisondeathray]

I need to use a professional video editor for once. I'm always lagging behind with these freeware tools. Without VDub and Avisynth I would've never learned to do anything for myself so I regret nothing about not starting with professional tools but it might be time to take this extra step.

I use whatever works for that goal, and there are usually more than one way to do things. There are many manipulations and tasks that those "freeware tools" like avisynth do better and/or faster. The more tools in your toolbelt, the more "problems" you can deal with. I see these tools more as complementary. I learn new techniques/approaches to do things from reading these boards all the time .

How did you fix that purple rectangle that tinted over the people's heads from frame 220-240? There was another huge square in the sky as well. What did you use to sharpen the video, also something in AE?

Those blue areas /flashes were originally light leaks. The rectangle was his mismatched "patch" repair job. Notice it falls apart and no longer matches with the rest of the frame, and the grain stops moving in areas where he used a still. That emphasizes why degrain/denoising is usually done first before compositing/repair work. If he had used a "rock stable" motion tracking prior to the patch reapir, it wouldn't have fallen apart like that .

"Rock stable" means you can fix things more easily because you can paste stuff from other frames - and they match (because everything is oriented properly) . Everything is easier for compositing, when you have good motion tracking data. You can "link" patches to the motion tracking data (for example when the camera moves in other shots, the patches "stick", or say when you want to cover up a face blemish, that repair "sticks" to the face as the head moves), your rotoscoping is much easier in general (less distance to move) . So even when there is intended lots of camera and subject motion, often you stabilize, do the masking/rotoscoping, then add back the motion. But for static shots like this, "rock stable" means you can often put parts of a still image to cover up bad areas on frames, because it matches up perfectly

I did a really quick, rough job on the "rectangle". If you look closely you can see flaws. If you were fixing it for real you would spend more time. It's masking or "rotoscoping". You just draw points around the foreground object (e.g. the man walking) , and replace the background areas. The points move every frame, but AE/mocha have keyframe interpolation (you don't have to manually do every frame, just a few in between) and various other masking tools that make it easier. Rotoscoping goes hand in hand with motion tracking which does a lot of the work for you. Unfortunately, this is one area where free tools suck (except maybe blender) .

There are other methods to get mask separation, for example difference mattes and motion masks, because this scene is largely static (and those techniques work better if the background was rock stable (ie. no motion in the background, especially when you use a temporal smoother and degrainer) - so the only motion is the walking people, thus you isolate the walking people). However, if it wasn't rock stable, the masks would be contaminated from the zooming and slight background motion

What did you use to sharpen the video, also something in AE?

Both AE, avisynth .

[timsky]

I think I'm going to have to do each shot in the film separately? The settings which suit one shot are terrible for another. Using all -1's on some shots the picture slowly rotates acw. I fixed this by having 400 in the rotation box, but -1 in the other three. The main problem on pan shots is, because it is a 4x3 picture in a 1920x1080 frame, the left and right borders move wildly.

[timsky]

Now I can stabilize images really well, but I still have this problem with the left and right borders moving about because it is a 4x3 picture and Deshaker obviously doesn't recognize the edges of my picture as borders because they are far away from the 1920x1080 frame edges. What can I do to stabilize my borders or fill them in? Here is an example

[jagabo]

Crop them away then add back new borders.

[timsky]

Sorry I don't know what you mean. Do you mean crop off the moving edges of the picture?

[jagabo]

Do you mean crop off the moving edges of the picture?

Yes.

[timsky]

OK, but what about this?

[timsky]

Stabilizing clips taken from 16mm film in Deshaker. When I stabilize a clip using Quality 24 I get perfect rock-steady results. But if I use Quality 1 for the same clip it is not perfect, there is slight movement, probably caused by film grain. My question is how do I take the data from the Quality 24 analysis, and use it for the conversion in Quality 1?

[poisondeathray]

Stabilizing clips taken from 16mm film in Deshaker. When I stabilize a clip using Quality 24 I get perfect rock-steady results. But if I use Quality 1 for the same clip it is not perfect, there is slight movement, probably caused by film grain. My question is how do I take the data from the Quality 24 analysis, and use it for the conversion in Quality 1?

Can you clarify what you mean ? Did you mean you encoded a quality 24 clip as INPUT into deshaker , NOT exported OUTPUT from deshaker ?

If that's what you meant, then you can do what was suggested earlier - apply denoise/degrain for the analysis pass only, but use the unfiltered video for pass 2

What's happening with quality 24, is it's dropping lots of the grain and details. You'd probably get better results with proper degrain & denoise filters instead of relying on the encoder to drop the grain

Eitherway - you save the log file after running the 1st pass (give it a good descriptive name, because if you have many clips it will get confusing very fast) , and then close the video, then open the 2nd unfiltered higher quality video using that log file for the 2nd pass

[timsky]

Thanks. In MP4 Encoding Settings I set quality to 24. I don't know how to save a log file after 1st pass. When I press OK it does both the 1st and 2nd pass without a break.

[poisondeathray]

I'm referring to deshaker in vdub . You're using videopad or something ? I don't know how to do it in videopad, I thought it would be the same

So it sounds like the export settings, you were referring to, not the INPUT file ?

Then I have no idea what is going on because the quality settings for export compression should come AFTER deshaker has been applied . Since the same file is INPUT into deshaker, you should get the same results from deshaker. That implies the only differences you're looking at are compression differences, not deshaker or analysis differences

[timsky]

In Videopad which I am using you R click on the clip you want to stabilize and it comes up with a window called Video Quality Setup. In that window you set the level of quality you want between 1 and 51, and you also set the Deshaker settings. Then you click OK and it goes on and does the entire stabilization process including 1st and 2nd pass, and it does it at the level of quality you have set. So there's no way you can do the 1st pass, save the settings, then do the 2nd pass. Or is there? Maybe I should use Virtual dub because now I have switched to exporting all clips from Premiere in Lagarith avi it will accept the clips (it wouldn't accept MP4). Can you recommend a good website to download Deshaker from because none of the sites I have tried will work?

[poisondeathray]

Download it from the author's site
http://www.guthspot.se/video/deshaker.htm

I would use 32bit vdub (and 32bit deshaker), because 64bit vdub has some issues

vdub can open MP4 with the ffinputdriver plugin

[timsky]

OK I'm using deshaker in virtualdub now. I'm a bit unsure how to do pass 1 on a clip, save the settings, then do pass 2 on another version of the clip.

[jagabo]

OK I'm using deshaker in virtualdub now. I'm a bit unsure how to do pass 1 on a clip, save the settings, then do pass 2 on another version of the clip.

Open first video. Set up pass 1. Return to the main window and select File -> Run Video Analysis pass. Return to the filter and configure the second pass. Open other video, view and save the result.

[poisondeathray]

OK I'm using deshaker in virtualdub now. I'm a bit unsure how to do pass 1 on a clip, save the settings, then do pass 2 on another version of the clip.

Look at the 1st tab, where it says log file. Enter an appropriate name, run the 1st pass analysis on the 1st clip . To do that you select the pass 1 button, and enter the settings. Then in vdub, you can choose file=>run video analysis pass and wait for it to finish

Close the video, load the 2nd video, it should be the same log file as the 1st clip. If you never change the name it will keep on overwriting it. Hit the 2nd pass and enter the settings you want

[timsky]

Thank you both. That works. Just one question: When I save the file I select Save AVI or Save old format AVI, and it saves. But the saved file is 3 times as large as the un-processed file, which is a Lagarith AVI file.

[poisondeathray]

video => compression => select something like lagarith

If you didn't select anything, it will be uncompressed RGB

[timsky]

Thanks

[Mephesto]

I did a really quick, rough job on the "rectangle". If you look closely you can see flaws.

I've looked very closely and I CAN'T see the slightest flaw. Your edit was a work of perfection.
It's not just that it was tinted but smoothed out and I expected loss of detail but didn't even see that in your edit. I guess regraining mitigated it.

Both AE, avisynth .

What Avisynth tool did you use to sharpen?

Also you were right about my edit having temporally smeared lines which indeed were the result of denoising but I could've avoided this if I used the deshaker data of the denoised video to deshake the noisy video and then denoise that stabilized footage. No smearing would be present and Stab() would probably have done a better job.

You got me interested in AE's stabilization feature now. Will I be able to try it out in a trial version? Will this be feasible for slightly wobbly animated films? I don't need it to work perfectly or work at all in action scenes but in most of the relatively static scenes I could use that kickass stabilization you showed us. I've never seen a work of perfection like that. Film wobble interferes with DeDup so it's one reason out of many why I want rock perfect stabilization.

[poisondeathray]

I did a really quick, rough job on the "rectangle". If you look closely you can see flaws.

I've looked very closely and I CAN'T see the slightest flaw. Your edit was a work of perfection.
It's not just that it was tinted but smoothed out and I expected loss of detail but didn't even see that in your edit. I guess regraining mitigated it.

Trust me, there are flaws. I just didn't bother/have the time to fix them. Your eyes probably aren't trained for looking for them; and yes, the grain helps hide some of the problems

What Avisynth tool did you use to sharpen?

The sharpening routine was just some unsharpmask in AE, and finesharp in avisynth. Care must be taken not to sharpen grain. I think I mentioned the various reasons for the degrain / regrain earlier, and sharpening is one of them. The levels/contrast were adjusted a bit too so that helps with subjective "sharpness"

You got me interested in AE's stabilization feature now. Will I be able to try it out in a trial version? Will this be feasible for slightly wobbly animated films? I don't need it to work perfectly or work at all in action scenes but in most of the relatively static scenes I could use that kickass stabilization you showed us. I've never seen a work of perfection like that. Film wobble interferes with DeDup so it's one reason out of many why I want rock perfect stabilization.

I mentioned this earlier - I used mocha for the tracking data. Basically it's a planar tracker - it tracks "planes" or flat surfaces. This shot is easy to track because there is a ground plane (the grass) . So the motion data is generated from mocha, but the compositing and stabilization is done in AE . The version that comes with AE is called "Mocha AE", but I don't know if it comes activated with the trial version of AE

AE's 2D stabilizer is based on point tracking . It tracks feature points instead of planes. It's not good for grainy material, because the grain can interfere with the tracking (feature points get messed up with dancing grain) . Degraining before tracking is a must for AE's point tracker, but usually not necessary for planar trackers. Blender has a planar tracker as well , but I'm not sure how good it is. There are different situations where you would use a point tracker, and situations where you would use a planar tracker , and other situations where you would use a 3d tracker (match moving)

Stab() is usually pretty good for film weave. If , by "wobble", you mean film warping - that's a whole different ball game. Much more difficult to repair, and a variety of different techniques and approaches used depending on the shot characteristics.

If the only reason you're exploring this is to get better results out of dedup - it would be way faster just to manually specify the dup frames

[Mephesto]

Trust me, there are flaws. I just didn't bother/have the time to fix them. Your eyes probably aren't trained for looking for them; and yes, the grain helps hide some of the problems

I guess we gurus at what we do are most picky about our work because we assume others perceive like we do. I get my panties in a twist over every video I do now over minuscule flaws, wishing I'd have doubled the bitrate and keep forgetting that most people STILL download 240p XVID videos and don't care.

The sharpening routine was just some unsharpmask in AE, and finesharp in avisynth. Care must be taken not to sharpen grain. I think I mentioned the various reasons for the degrain / regrain earlier, and sharpening is one of them. The levels/contrast were adjusted a bit too so that helps with subjective "sharpness"

Is grain not added after the sharpening?

Thanks for the AE deshaker info.

Stab() is usually pretty good for film weave. If , by "wobble", you mean film warping - that's a whole different ball game. Much more difficult to repair, and a variety of different techniques and approaches used depending on the shot characteristics.

Yes but it's the same problem the clip you fixed had. It was 1080p and the average shake was only 1 or 2 pixels.

If the only reason you're exploring this is to get better results out of dedup - it would be way faster just to manually specify the dup frames

It's one of the reasons, not the only one. The main reason is to get the video perfect. Actually, I've noticed most scenes in cartoons (sometimes film as well) have a screwed up keyframe. The first frame always has a white rip at the top or is zoomed in/out more than the following frames (likely to hide that rip). So it would be great if mocha also dealt with zoom as well as panning and fixed that first frame of every scene.

It's so irritating how these guys call these new Blu-ray releases "digitally remastered" yet they don't fix all the flaws.

[poisondeathray]

Is grain not added after the sharpening?

Yes - If you didn't degrain/denoise first then any sharpening would sharpen grain as well (along with the other reasons mentioned earlier why typically degraining is done first)

Yes but it's the same problem the clip you fixed had. It was 1080p and the average shake was only 1 or 2 pixels.

But that's not bad warping that you normally see in some film scans. Or bad lens distortions. Or rolling shutter wobble. Typically those are repaired with a mesh filter to dewarp, although you may use some tracking & patch repair to assist


Note this approach to "fix" was easier here because the shot characteristics - it was supposed to be static, tripod shot, no camera movement (locked off) . How often do you have that? It becomes more difficult when there is camera movement (x,y translation, + zoom = perspective shifts) and object movement with crossing over each other . You also have rotation (not video frame rotation along the z-axis, but object rotation of elements in the scene) - a 2D patch won't necessarily work in those cases. Those advanced cases definitely require "rock stable" tracking for various objects (you're not just tracking the background movement). The specific shot characteristics and context determine how you might approach it

I've noticed most scenes in cartoons (sometimes film as well) have a screwed up keyframe. The first frame always has a white rip at the top or is zoomed in/out more than the following frames (likely to hide that rip). So it would be great if mocha also dealt with zoom as well as panning and fixed that first frame of every scene.

Those are film splice edits. Difficult to fix. The usual "automatic" way in avisynth is to use the scselect script by didee (it basically replaces it with a dupe automatically, based on the threshold settings). You can't use "quick" interpolation techniques because you require 2 "good" end points, and presumably you only have 1 here because it's right before the scenechange (usually the last frame from the scene before, and 1st frame from scene after are affected). Mocha can assist with tracked elements, but it's only a tool to assist. If you want it "perfect", more often than not you have to do lots of manual work / photoshop / compositing

It's so irritating how these guys call these new Blu-ray releases "digitally remastered" yet they don't fix all the flaws.

Time and $ . Return on Investment. Only big remastered releases get truly fixed. e.g. Wizard of Oz .