Aggressive deblocking? Or what can I do with this video?

[Asterra]

My brother made an ill-advised purchase last year and now I am stuck with the video generated by said device. Here is the culprit:

http://www.amazon.com/Kodak-PlaySport-Waterproof-Pocket-Camera/dp/B0030MITDK

The Kodak PlaySport Zx3. Boasts the ability to record 1080p video underwater. Shrug. That it does. But here is a frame of that video (no filters, directshowsource):



At 29.97fps it gets 12.3Mbps (AVC Main@L4.2). The quality doesn't seem to live up to the promise of even that humble bitrate. So I've got to try to do what I can to make things look a little better. Step one is to deal with the overall poor quality.

Actually, the above frame looks a lot better than it did in KM Player. I can only assume that KM Player was failing some sort of process and causing the image to look way blockier than it needed to. So I don't even know if I should be trying to deblock the image. Maybe there is nothing that can be done. I did try a couple of permutations of DeBlock_QED, but if it did anything at all to the image, the effect was too subtle to discern. I also found an interesting script, with some impressive results posted later on, in this thread:

http://forum.doom9.org/archive/index.php/t-115632.html

But the script is unusable because it runs out of memory.

Step two - assuming something can be done to make the video a little more agreeable, is to interpolate new frames so I can have a proper 60fps. In its current incarnation, the video at 30fps exhibits little to no motion blur and the result is somewhat jarring. I'll be pretty happy if I can get a quality result out of this little camera.

[poisondeathray]

I also found an interesting script, with some impressive results posted later on, in this thread:

http://forum.doom9.org/archive/index.php/t-115632.html

But the script is unusable because it runs out of memory.

did you try adding setmemorymax()

e.g.
setmemorymax(512)
# rest of the script

Step two - assuming something can be done to make the video a little more agreeable, is to interpolate new frames so I can have a proper 60fps. In its current incarnation, the video at 30fps exhibits little to no motion blur and the result is somewhat jarring. I'll be pretty happy if I can get a quality result out of this little camera.

There are several derivative functions that use mvtools or mvtools2 . The mvtools2 documentation has an example for mflowfps . You can just copy & paste it & alter the values & names. SmoothFPS() is another function, or Interframe(), there are many other variants of those scripts if you search
http://forum.doom9.org/showthread.php?p=1486831


For motion blur, you can use mflowblur (See the docs)

[jagabo]

Actually, the above frame looks a lot better than it did in KM Player. I can only assume that KM Player was failing some sort of process and causing the image to look way blockier than it needed to.

KMplayer may have been skipping the h.264 deblocking step because it was running low on CPU. Or maybe you disabled the deblocking in the h.264 decoder settings.

You'll find motion interpolation doesn't work well with non-linear motion. I suppose you might as well try.

[Asterra]

did you try adding setmemorymax()

Well, I have now. ;p Even 1.5GB doesn't seem enough so there must be something fundamentally excessive (broken) about the script's memory demands.

The motion blur works great. And I'll see if PPro or AE can do a better job with the frame interpolation. What remains is the original task of hopefully making the video appear a little less obviously coming from a poor camera.

KMplayer may have been skipping the h.264 deblocking step because it was running low on CPU. Or maybe you disabled the deblocking in the h.264 decoder settings.

That's entirely likely. I hadn't known that h.264 had deblocking built-in. I find myself wondering if an Avisynth-based deblocking script could somehow do a better job than h.264's built-in process. Though presumably the h.264 process is privy to some guidance variables that help idealize the output.

You'll find motion interpolation doesn't work well with non-linear motion. I suppose you might as well try.

Well, my experience with this kind of interpolation tells me that I should at the very least be seeing better results than what I got from the mflowfps example, whose results looked more like a morph than an interpolation. I'll be testing Adobe's apps eventually.

[poisondeathray]

You'll find motion interpolation doesn't work well with non-linear motion. I suppose you might as well try.

Well, my experience with this kind of interpolation tells me that I should at the very least be seeing better results than what I got from the mflowfps example, whose results looked more like a morph than an interpolation. I'll be testing Adobe's apps eventually.

jagabo's right, you won't get good interpolation with "non linear" content like water . I have quite a bit of experience with twixtor, AE pixel motion, timewarp, nuke, and avisynth methods - you will get the morphing edge dragging artifacts . The pro versions like twixtor allow you to use mattes to reduce this, but it's almost impossible to use a foreground/background matte for water .

You will find avisynth will give similar results , but process much faster. Part of the reason is you can use it in YV12, whereas the other filters function in RGB - there is more data that has to be processed with RGB

From your example picture, I'm not sure if any filter will help much with the quality. You could resize to 1280x720

[jagabo]

When the motion estimation techniques can't find obvious motion vectors they resort to blending. With non linear motions they often misidentify motion and you get weird distortions. An extreme example:

source frame:


motion interpolated frame:


next source frame:


That was done in AviSynth with this function:

function DoubleFPS2(clip source)
{
super = MSuper(source, pel=2, hpad=0, vpad=0, rfilter=4)
backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
MBlockFps(source, super, backward_3, forward_3, num=2*FramerateNumerator(source), den=FramerateDenominator(source), mode=0)
}