Area interpolation filter?

[ndjamena]

I may have been imagining it but I vaguely recall mention of a filter that would fill in an area of video based on the areas around it.

I'm processing Doctor Who again right now and sometimes the borders are a tad tacky and I was wondering if in some cases I could handle it by inventing new contents.

I'm looking at a scene at the moment where the corners have gone black. What's supposed to be there is just fog, so it shouldn't be hard to mimic that by imitating the fog around it... IF such a filter exists...

[poisondeathray]

There are various inpainting plugins (several free ones in avisynth e.g exinpaint) , it's similar to "context aware fill" in photoshop if you're familar with photoshop or gimp (gimp's resynthesizer). The topic is discussed semi frequently, search for "inpainting" techniques if you want more info

But usually doesn't work well with video by itself only, because of interframe fluctuations. You usually have to do other manipulations/ massaging as well for video

[ndjamena]

Are you saying none of them have any temporal abilities?

[poisondeathray]

Are you saying none of them have any temporal abilities?

I haven't seen any with temporal components. They all use spatial interpolation algorithms.

You might be able to do some homebrew temporal component by processing every nth frame border (or area of interest) and interpolating those with mvtools2

[ndjamena]

Well, thankfully it's only a four second clip.

It's really not hard to imitate billowing fog obscuring random grey rubble recorded on old degraded film. The corners do look a tad busy but it's definitely an improvement on the encroaching blackness. There no problems with the corners of the scenes immediately before and after so hopefully I won't have to use this method much.

[manono]

Then you might be able to use frame interpolation techniques, rather than inpainting, by interpolating only the area in question, either by using a mask to leave the rest alone, or by cropping down to only the part needing interpolation, something like this:

A=Last
B=A.ReplaceFramesMC(first bad frame number, number of frames needing interpolation)
Mask=ImageSource("Fog.bmp")
Overlay(A,B,0,0,Mask)

I uploaded a JPG rather than a BMP, for the smaller size:

[jagabo]

For something near the edge I've occasionally overlaid a (blurred?) mirror image.

# mirror eight pixels at the right edge of the frame
mirror = Crop(704, 0, 8, -0).FlipHorizontal()
Overlay(mirror, x=712, y=0)

[ndjamena]

sample

Top right and bottom right. I can't tell if the random dark areas in the top left are damage or detail, but I got rid of it just in case.

The image pans and yet top right stays where it is, so that has to be damage. He walks into bottom right so I had to shrink the infill area as much as possible until the problems the filter creates weren't as noticeable.

At the moment it's just filling the areas with what amounts to random noise, considering the video pans and the billowing fog (dust?) I don't want anything too predictable or it will be noticeable.

[ajk]

That looks like just some mild vignetting, not damage. It's quite common to see this particularly with older film footage. It's so close to the periphery that it would not be visible on TV because it's in the overscan area.

I'd say it's not worth doing anything about, but if you really want to, you could brighten up the corners with a technique similar to this: http://rationalqm.us/hotspot/hotspot.html

[ndjamena]

That would be why it seems to be on all the corners, I've found several more scenes with the problem in the meantime. Unfortunately I don't think that particular filter will be of much use at this point. The problem is remarkably inconsistent between frames as the darkness ebbs and flows, the top left is definitely affected and I've found traces of it in the bottom left now too.