Removing water damage from film

[bmick23]

I have a kinescope that has water damage in some areas. Is there any method, manual or automatic, that might be able to help me repair some of this?

Sample: https://www.dropbox.com/s/j6cku92z53glpr2/00001-edit-na.m2ts?dl=0

[JVRaines]

Those are rather extensive defects. I would be inclined to work on them manually in a graphic editor, one frame at a time. In a static shot, you can often put a good frame behind a bad one and mask out the blotches. When you have motion, you will probably need to find two good "bookend" frames and then tween the rest with an interpolation filter. Recent versions of Photoshop have this capability as part of Timeline Animation.

[johnmeyer]

I do media restoration for a living. Movie film is a major part of what I do.

The huge defects on the left side persist for multiple frames so you cannot do motion estimation to synthesize new frames. However, in the short clip, there is very little movement and very little detail in the damaged sections, so you may be able to use a common trick of putting two copies of the video into your NLE (I use Vegas Pro). My NLE lets me draw a mask over the defect, letting the video from the second instance of the same video "show through" that mask. You then offset (in time) the second copy of the video so that good video shows through, but from a few frames before or after the damaged frame. Sometimes you have to move around (left/right/up/down/rotate) this second instance of video, which you are using as a patch, so that it lines up, as best you can, with the video in the damaged frame. Sometimes you have to do this frame-by-frame, and other times you can get a whole block of frames fixed with one mask and one offset copy.

For defects that only last for one frame, even if they are huge, you can mark the bad frame and then use an AVISynth script to replace all marked frames with a frame synthesized from adjacent frames. You'll find my "filldrops" script over at doom9.org. It is what I use to replace one bad frame. In many cases, the result is absolutely perfect: you cannot tell that the frame is completely synthesized.

Here is a near-perfect example of that where the film look in the camera disappeared, causing the frame to jump. Unlike the more common situation where this happens in the projector, this was "baked in" to the film. This restoration took a lot of work because I had to go through, frame-by-frame to mark the bad frames. I wish I had posted the second segment of this where the truck drove away, because it is far more impressive to see near-perfect restoration of a moving object. However, this still shows the idea:

https://www.youtube.com/watch?v=uzMFodrGHDs

This one is not quite as spectacular, but it shows how I used the same technique to restore some Polavision film (instant movie film) where the chemical pollution of sitting in the development chemicals for forty years caused the emulsion to delaminate in places:

https://www.youtube.com/watch?v=gzTistk2Ms0

[jagabo]

Here's a little trick (in AviSynth, after decimation to 23.976 fps):

part1 = Trim(0,50)
part2 = Trim(32,49).Reverse()
part3 = Trim(70,87).Reverse()
part4 = Trim(69,0)
part23 = Dissolve(part2,part3,18)

part1+part23+part4

[bmick23]

Thank you for the helpful replies. Because the damage rolls in and out, I'm going to try using Photoshop to repair the minor damage and if possible, interpolate the missing frames in between. I also like the method of repeating/reversing frames on still shots. I'll post results if it works out. Thanks again.

[bmick23]

Just to update: the Photoshop method works very well when the shot is static, but even slight movement causes a strange warping effect in the replaced area. I'm wondering if there is any software (or a plugin? paid or otherwise) that allows you to paint or create a mask over the bad area of the frame, and interpolate only that section with the information from the previous/next frames. Most of the damage is unique and lasts for only one frame on different areas. I think this method would be ideal, but does something like that exist?

[JVRaines]

Have you tried Photoshop's Timeline Animation with keyframes?

[poisondeathray]

Just to update: the Photoshop method works very well when the shot is static, but even slight movement causes a strange warping effect in the replaced area. I'm wondering if there is any software (or a plugin? paid or otherwise) that allows you to paint or create a mask over the bad area of the frame, and interpolate only that section with the information from the previous/next frames. Most of the damage is unique and lasts for only one frame on different areas. I think this method would be ideal, but does something like that exist?

Photoshop doesn't take into the 3d perspective differences, motion or anything like that. So you would have to account for the motion and perspective differences if "patching" data from a "good" frame . That's why motion tracking is often used for certain shots - you're essentially "eliminating" the motion so you can patch adjacent "good" frames (or parts of frames) into "bad" frames, so they "fit" perfectly (at least in theory for 2d shots). It's faster and more accurate than manually trying to fit or position patches. Typically something like after effects + mocha would be used. But 2D motion tracking isn't suitable for every type of shot - it really depends on the specific shot characteristics. When you have things film warping, lens abnormalities, they have to be fixed first otherwise the "patches" won't fit correctly either. Also, individual frame patch (with masking + clone + healing brush etc...) repairs will tend to "flutter" frame to frame (unless you spend an extraordinary amount of time), so you usually need to apply some temporal smoothing after

But to answer your other question about interpolating part of a frame - A "cheap" way to do that is avisynth then combine in a free compositor like blender, fusion, natron, (or nuke has a free version too) with rotoscoping/masks. After Effects would be commonly used too. Masking in that scenario can be done before or after, but is usually better before interpolation, because you would eliminate possible "bad" motion areas that could propogate errors into and contaminate the new interpolated frame, but you usually have to do some refined masking afterwards too to composite the fix in. It's possible to "bridge" more than 1 frame with different interpolation functions. You can also combine with clean plates (fix intermediate frames, so less interpolation) to increase the accuracy. Motion interpolated frames only works well in specific scenarios, other times it's a complete failure

An advanced way is 3d camera projection in software . You project clean plate textures onto 3d geometry and matchmove using 3d camera tracking. This is how many special effects today are done in Hollywood , but you can use the same techniques for restoration . The camera motion is separated from the object motion(s) , so the projected textures "stick" on the designated objects. This way things like perspective are kept instead of a "flat" 2d patch which only has a fractional degrees of rotation before it looks like it "falls" off.

[johnmeyer]

Just to update: the Photoshop method works very well when the shot is static, but even slight movement causes a strange warping effect in the replaced area. I'm wondering if there is any software (or a plugin? paid or otherwise) that allows you to paint or create a mask over the bad area of the frame, and interpolate only that section with the information from the previous/next frames. Most of the damage is unique and lasts for only one frame on different areas. I think this method would be ideal, but does something like that exist?

I already described exactly what you are asking for. Please re-read my previous post. Vegas Pro has precisely this feature, and I use it all the time, as I already explained.

It is true, however, that if the camera is moving towards or away from the object, then the perspective can change and you will need tracking, etc. (like After Effects, as poison described). However, Vegas lets you feather the mask, and that feathering can hide all sorts of anomalies that happen at the mask boundaries.

[bmick23]

I already described exactly what you are asking for. Please re-read my previous post. Vegas Pro has precisely this feature, and I use it all the time, as I already explained.

It is true, however, that if the camera is moving towards or away from the object, then the perspective can change and you will need tracking, etc. (like After Effects, as poison described). However, Vegas lets you feather the mask, and that feathering can hide all sorts of anomalies that happen at the mask boundaries.

I re-read the Vegas post, but the frame below is offset. Vegas is not creating a new frame or estimating motion between the two. My big problem is the shots are static, but some of the damage is over quickly moving objects like a flag or a person dancing, and it is noticeable even for one frame if it remains still in one spot. I might try cleaning what I can and use AviSynth to interpolate up to 60fps, then grab needed frames from there to patch in over the damage.

[bmick23]

To update, I found a method that works very well for what I need.

Using pt_FrameRestorer, I marked the damaged frames and the script automatically interpolates and replaces them. I laid that track under the original video in Photoshop and painted out the damage. The results are near perfect and are sufficient for this project.


Results sample:
https://www.dropbox.com/s/0qmefbl3brp0j57/Kine-waterdamage-restore.mp4?dl=0

[JVRaines]

Nice job!

[johnmeyer]

Those results are extraordinarily good. Well done!!

[jagabo]

Just for fun:

LSmashVideoSource("Kine-waterdamage-restore.mp4") 
Crop(0,0,920,696)
TDecimate() # restore film frame rate
src=last

ReplaceFramesMC(90, 21) # starting frame, number of frames
ReplaceFramesMC(112, 2)
ReplaceFramesMC(203, 22)

StackHorizontal(src, last)