Video Stabilizing Effect on Encoding Quality

[Gargalash]

Hello,
I have been wondering a lot on the effect of video stabilizing on the quality of an encoding. I pretty sure I'm right when I think that a scene with a still background and some action will look better when encoded then the same scene with a moving background.

In my case, the backgrounds are never still and constantly slightly shacking. I was thinking about stabilizing the videos I need to provide over the web to improve quality with the same bitrate or maybe get the same quality with lower bitrate (thus saving bandwidth and uploading time).

Technical info: I start with VOBs and end with 480x360 .FLV at 800k/s.

I'm sure it changes a lot from one video to another, but I wanted to have your thoughts on this and on what video stabilizer you would recommend. I work with AVISynth, but I'm flexible to find a good solution.

Thanks!

[poisondeathray]

In avisynth, depan, depanstabilize

You can also try deshaker in vdub

[creamyhorror]

Deshaker is great. I couldn't get depan(stabilize) to really work, but I doubt it could do as well as Deshaker.

[Gargalash]

Thanks for your suggestions! I will be exploring those.

Any thoughts on the effect of stabilizing on encoding quality?

[jagabo]

There's also Stab() for simple left/right/up/down (no rotations) shaking in AviSynth.

[poisondeathray]

Any thoughts on the effect of stabilizing on encoding quality?

Absolutely makes a HUGE impact on bitrate capped scenarios like yours.

More bitrate can be allocated to image quality instead of coding differences between frames

[Gargalash]

Thank you all for your replies.

[Xpenguin17]

Video stabilizing does not improve quality or make a video more compressible by much, maybe 5-20%. Modern standards such as H.264 are better equipped to efficiently handle panning and motion than previous standards, so removing panning doesn't gain that much efficiency, not to mention that stabilization artifacts introduced tend to add more entropy into the video, retarding most of the gains, so the final tradeoff comes to about 10%.

Also, use Deshaker for Virtualdub and make sure your configuration looks like this:

The default settings blow **** and won't always stabilize your video well, especially the highlighted setting which should always be on 1.

If you wanna make your videos more compressible, focus on denoising instead of deshaking.

[ronnylov]

10% is not that bad. Anyway a stable video is much more pleasant to view than shaky handheld footage. You always add denoising to make it even more compressible. But do not denoise too much, it will make it look unnatural and plastic.

Using h.264 compression instead of flv (unless it already is flv muxed with h.264 video) will gain quite a lot extra compression too.

[Gargalash]

Xpenguin17,
Thanks for your settings suggestion, I will keep testing with these.
As for the compressibility gain, 10% is already good, 10% less bandwidth, 10% less upload time, etc. I also denoise all the time after resizing to the final dimension (with avisynth).

ronnylov,
I would like to encode the flv with h.264, but so far I was only able to do that with the "new" f4v format available in Adobe Media Encoder.

Is there a way to encode flv with h.264 and will it still work on everybody's machine when streaming through a flash player on the web?

Thanks for your help!

[ronnylov]

First I encode the video to h.264 using MeGUI and x264 encoder and the audio to AAC format. Then I mux it to mp4 format using the muxing tool in MeGUI.

Following step may not be necessary (I do it anyway):
I use MP4 faststart to move metadata to the start of the file.
http://www.datagoround.com/lab/

Now I use ffmpeg to remux the mp4 file to flv format:
ffmpeg -i h264_aac.mp4 -vcodec copy -acodec copy h264_aac.flv

I read somewhere that ffmpeg has problems with metadata too and it can be fixed with flvtool2 (maybe not necessary):
flvtool2 -U h264_aac.flv

Anyway after following these steps the flv file plays perfectly in the flash player on blip.tv, which allows upload of flv files (they do not reencode them).

Here is an example of my videos:
http://blip.tv/file/2306069/

I learn how to do this by this thread on doom 9 forum:
http://forum.doom9.org/showthread.php?t=147758

[edDV]

Keep in mind that a solid tripod and a quality fluid head gets the best compression quality.

[Gargalash]

ronnylov, about this step:
"First I encode the video to h.264 using MeGUI and x264 encoder and the audio to AAC format. Then I mux it to mp4 format using the muxing tool in MeGUI." This step encodes to the final video bitrate right?

And then there is no re-encoding in this step right?:
"Now I use ffmpeg to remux the mp4 file to flv format:
ffmpeg -i h264_aac.mp4 -vcodec copy -acodec copy h264_aac.flv"

I will give this a try, it's very interesting!

edDV,
Unfortunately, I don't have control over the source material. This is why I'm looking for some stabilization as well.

Thank you all for helping!

[ronnylov]

Yes, MeGUI is doing the actual encoding to h.264 format using the x264 encoder to the final bitrate. I do also encode the audio to AAC with MeGUI using Nero AAC encoder. Then I mux the video and audio streams to MP4 format, still using MeGUI. Now the mp4 file contains the encoded audio and video streams.

Then the ffmpeg step is repackaging (remuxing) the streams from the MP4 file into a new file using the FLV container.

My example video is not stabilized other than using the video camera built in optical stabilizer. It was just as an example of an h.264 encoded flv file I uploaded to blip.tv and it seems to work.

If you want to test how much stabilizing affects the compressibility you can use the CRF mode of x264 encoder and encode at the same setting (for instance CRF=20) two versions, one with stabilizing and another without. The difference of filesize is the difference of compressibility. But one thing to remember is that software stabilizing of video will decrease the resolution of the video because the software zooms into a smaller area of the picture. A lower resolution of the video will by itself make it more compressible. However if your source have much higher resolution than your destination format then this might not be an issue anyway.

[Gargalash]

Thanks you very much for the explanation ronnylov!
I will get into this next week and see if I can improve my final files.

[Xpenguin17]

Gorgolosh, scratch what I said earlier. With the new mb-tree feature in x264, a video benefits greatly from stabilization.

[creamyhorror]

Gorgolosh, scratch what I said earlier. With the new mb-tree feature in x264, a video benefits greatly from stabilization.

That's very believable, but has anyone done tests pre- and post-stabilization?

[poisondeathray]

I have on some handheld footage, but not specific to mb-tree. It's very situation dependent, and problematic to measure precisely. (e.g. do you use CRF as a measure? because it's only a rough measure , and if you are using adaptive zoom, it's not really comparable)

Also, too many variables to make broad conclusions, except that it can help a lot (e.g. how shaky the footage was, what bitrate range your target was set at, noise in the footage etc...).

[creamyhorror]

Yeah, that was partly why I asked the question - because it didn't seem possible to make a real comparison. But I was curious if anyone had done any tests to come to that conclusion.

[Xpenguin17]

I have done tests before, but not with mb-tree yet. Usually, under the same CRF the bitrate would be reduced by 5-15% and have a higher SSIM with a lower bitrate, but sometimes stabilizing would increase entropy.

I'll do tests right now if you send me high quality shaky samples. If your camcorder has a high rolling shutter speed then we can avoid blurring/stabilization artifacts.

[creamyhorror]

If you stabilize, you're going to be changing the footage itself (either by zooming or by reusing material from previous frames); the footage also gets fully re-encoded during stabilization. So I don't see how SSIM or CRF can be used to compare, really.

What you can compare is the appearance of a stabilized video encoded with and without mbtree at the same bitrate to see which looks better. That would probably demonstrate how useful mbtree is for compressing stabilized.

Don't have any high quality shaky samples, just a low quality (SD) one.

[Xpenguin17]

I'm not a dumbass, I don't re-encode the stabilized video twice, it's saved as lossless before it's encoded.

If unstable clip with crf22 results in 1000 kbps and stable clip 900 under the same settings/crf then you got your answer.

Or if unstable clip at 1000 kbps yields 0.99000 SSIM to the original, and stable video at 1000 kbps gets 0.99200 to the lossless stable clip, then that again proves the benefit.

If anyone recorded Jimmy and Judy off HDTV then those would be ideal shaky samples, then again I heard they intentionally contain noise, so maybe no.

[creamyhorror]

I'm not a dumbass, I don't re-encode the stabilized video twice, it's saved as lossless before it's encoded.

True, my bad.

If unstable clip with crf22 results in 1000 kbps and stable clip 900 under the same settings/crf then you got your answer.

Or if unstable clip at 1000 kbps yields 0.99000 SSIM to the original, and stable video at 1000 kbps gets 0.99200 to the lossless stable clip, then that again proves the benefit.

But that's my point. You can't compare CRFs across different videos, generally. The unstabilized and stabilized videos are fundamentally different sources, when you consider the adaptive zooming and cropping involved. What if your 900kbps clip looks "worse" quality-wise than the unstable clip, despite being encoded with the same CRF? Or the unstable clip with the lower SSIM looks better?

I'd be willing to draw some sort of conclusion if a stabilized source was encoded with and without mbtree. That would show us how much of an improvement mbtree brings for stabilization, and that's a meaningful question to my mind.

[Xpenguin17]

Okay, I found a high quality shaky clip of some hot home-made porn (IN .MP4 FORMAT!!!) but wasn't able to stabilize 'cuz it consists of an empty white room with practically no background objects the motion vectors can map to deshake, plus with all the constant zooming and close-ups I doubt even that would make a difference.

If anyone finds anything better, PM me.

But that's my point. You can't compare CRFs across different videos, generally. The unstabilized and stabilized videos are fundamentally different sources, when you consider the adaptive zooming and cropping involved. What if your 900kbps clip looks "worse" quality-wise than the unstable clip, despite being encoded with the same CRF? Or the unstable clip with the lower SSIM looks better?

It would only look better with a lower SSIM if the compression destroyed noise in the original clip. Sometimes, the stabilized clip with the same CRF would be a higher bitrate than unstable video.

[Gargalash]

Interesting discussion going on!
Unfortunately, I'm unable to contribute to it anymore But I keep reading!

Thanks