Apply color fill to a portion of video frame

[hand123]

Hello,
I'm currently editing video digitized from 8mm film. The scanning process was wide screen, so therefore, I can see the sprocket holes which are represented in white squares on the top left and right covers of the video. There is film footage between the sprocket holes that I would like to save and therefore do not want to simply crop out the area with the white sprocket squares.

So my question is, are there any video editing tools out there that allow you to choose areas or sections of a video frame and either fill the color or overlay with something else to in effect cover the sprocket hole areas?

Does Apple Motion support the ability to overlay blocks of color in sections of the video frame?

Are there other programs like Adobe Premeire, Final Cut Pro, etc that can replace color or overlay fills in sections of video frames?

Thanks

[manono]

Using AviSynth you can do all that you asked. It won't be automatic, although I don't think any of the others will be either. You'll have to specify frame numbers (or start and end frame numbers if there are a bunch in a row). You can probably, depending on the content in between the sprocket holes, create a pretty good simulation of what would have been there had the sprocket holes not been there to begin with.

Unless I misunderstood and these white squares are throughout the video, in which case it's even easier to accomplish.

Maybe if you made available 10 seconds or so of the digitized footage, people can have a look to advise what can be done. Using masks, I expect most video editors can do what you want.

[hand123]

Hello,
Thank you for the reply. As you can see in the linked sample, there are continuous sprocket holes to the left top and bottom. There is also film content between the sprocket holes that I would like to preserve. Therefore, I'm looking for a graceful and appealing way to mask the white squares with something more appealing such as a neutral blurred area of color or similar.

--- Please ignore this YouTube link attempt. I will upload a sample. ------

Thanks

[manono]

"This video is unavailable.
Sorry about that."

No one wants something from YouTube. Can't you provide a piece of the real thing?

[hand123]

I will try an upload. Thanks.

[hand123]

Here is an upload attempt.

[manono]

There are several ways to do what you want, and others might have different and better ideas. I removed the duplicate frames to bring it back to its original 18fps. I cropped away the black sides and also shrunk the resolution (so I could see what I was doing). I took that little bit of video on top and added it to the bottom, but it doesn't always line up properly. There's a lot that can be done to restore that, but I didn't attempt it.

I stretched what was below the top white block and above the bottom block, blurred it and then replaced the white blocks with the blurred and stretched video. Was this video shot as 'widescreen', or is it in the wrong aspect ratio, or is it missing picture on top and bottom? Here's the script used:

FFVideoSource("Sample.mp4")
Crop(56,0,-40,-34)
TDecimate(Cycle=5,CycleR=2)
X=Last
A=X.Crop(0,0,0,-1000)
A=A.Trim(1,0)
B=X.Crop(0,80,0,0)
StackVertical(B,A)
Crop(0,0,0,-14)
LanczosResize(912,500)
A=Last
Crop(0,72,0,98)
GaussianBlur(100,100,100)
BilinearResize(912,500)
B=Last
Mask=ImageSource("White.bmp")
Overlay(A,B,0,0,Mask)

[hand123]

That is absolutely incredible. I have so much to learn. Can you direct me to where you learned this?

I'm predominantly a Mac user. Any suggestions for Mac software that can do this? I already own Final Cut, Adobe Premiere Elements, and Wondershare Filmora. I would also consider buying Motion if that had good masking features.

Again, what you have done is really great. Thanks.

[hand123]

Now after reading the following link, I'm wondering if I'm getting the most out of the scanning service I'm using. Too many things to consider!
http://www.thebattles.net/video/8mm_restoration.html

[jagabo]

Or you could just overlay the sprocket holes with black. That is very easy to do in any editor.


[Attachment 40684 - Click to enlarge]

[manono]

Can you direct me to where you learned this?

The masking process I learned here from jagabo and poisondeathray. The rest I learned from years of use, with some learned from reading in the AviSynth forum at Doom9. None of it is what might be termed 'advanced AviSynth'. But almost all of it is documented at the AviSynth site:

http://avisynth.nl/index.php/Main_Page

jagabo mentioned that simple masking can be accomplished in about any NLE.

Now after reading the following link, I'm wondering if I'm getting the most out of the scanning service I'm using.

Is the video supposed to be 'widescreen'? Is there supposed to be part of the previous frame showing on top of the current frame? The guy's link speaks favorably of AviSynth. If you really want to see what it's capable of in the hands of an expert, have a look at this thread and check out the comparison videos:

https://forum.doom9.org/showthread.php?t=144271

The scripts are included but you'll have to have at least some experience (and the use of a Windows machine) to take advantage of them. Based on that short sample, there's a lot that can be done with your video to greatly improve it.

[hand123]

[QUOTE=jagabo;2478334]Or you could just overlay the sprocket holes with black. That is very easy to do in any editor.

Now that looks like it might be the easiest. jagabo, how did you do it? Based on your example, I will try to figure it out with the software I have available. Thanks.

[jagabo]

Or you could just overlay the sprocket holes with black. That is very easy to do in any editor.

Now that looks like it might be the easiest. jagabo, how did you do it? Based on your example, I will try to figure it out with the software I have available. Thanks.

I did it in AviSynth with a simple overlay.

LSMASHVideoSource("sample.mp4") # open the source video
amask = ImageSource("mask.png", start=0, end=last.framecount, fps=last.framerate).ConvertToYV12(matrix="PC.601")
black = BlankClip(last) # a black frame to overlay
Overlay(last, black, mask=amask) # overlay the black video where the mask is white

This is the alpha mask image:


[Attachment 40690 - Click to enlarge]

There's no need to set up and learn AviSynth just for this. Every NLE has the ability to do it. But you may want to learn AviSynth for the other processing it can do.

[hand123]

I removed the duplicate frames to bring it back to its original 18fps. Was this video shot as 'widescreen', or is it in the wrong aspect ratio, or is it missing picture on top and bottom?

1) FPS
I have a lot to learn about fps. I was not at all concerned with this as I was simply taking the scanned MOV file to my Mac and editing the video in Wondershare Filmora and Adobe Premiere Elements for simple cropping, cutting and rendering for easy family sharing on YouTube or DVD. The family has loved this so far as they have not seen these films in over 40 years. So, I did not even bother to consider fps in my handling of the MOV files provided by the scanning service. I see now from the following link that frame rate conversion and interpolation can greatly enhance the results. I need to look into that. http://www.thebattles.net/video/8mm_restoration.html

2) Widescreen.
The service I'm using has two choices: SD or HD. SD is described as low resolution and 4:3. HD is described as high resolution and wide screen. I chose HD mainly so they would capture the entire width of the film including the sprocket holes because I knew there might be footage in there. I also asked them for "over scanning" which provided a sliver of footage on the top and bottom of the adjacent film frames. I was happy with both of these because I knew that I would be cropping while editing anyway.

[manono]

1) FPS
I have a lot to learn about fps.

If you take your source and advance a frame at a time, you'll see all the duplicate frames. They can be easily removed to bring it back to a more smoothly playing video. For many output formats 18fps is fine. For NTSC DVD you'd have to make it 29.97fps, but there are much better ways to do that other than having all the repeated frames in there. Me, I'd probably interpolate frames to bring it up to 20fps, encode that and then add 3:3 pulldown afterwards to have it output 29.97fps. But there are different schools of thought about that and not all would agree. Some might even vote to keep it jerky and stuttery to add to that 'old-timey' effect.

2) Widescreen.

It's one or the other? No Hi-def and original aspect ratio? And you're okay with losing over a quarter of your original video? I realize you didn't really have much of a choice there. And even with that service, hi-def and widescreen might be okay if - if - they made it widescreen by adding black bars to the sides, while retaining the complete original video in the center. The effective resolution will be somewhat lower but at least you'd have all there was. Then you'd be in charge of what gets cropped away.

[hand123]

This is the alpha mask image:

How did you get that mask so precise, smooth and feathered? I will now be trying to teach myself how to create these masks in software that I have.

Thanks to both of you for all the continued information.

[jagabo]

I started with a frame of the video, reduced it to greyscale, and changed the brightness and contrast so that only the sprocket holes were white and the rest of the picture was black. If there were any white areas in the rest of the picture I painted them black by hand. I ran a Gaussian blur to soften the edges. I then cut/pasted parts of the sprocket holes to make them larger. You can see where I didn't quite align the paste properly in the lower hole.

start:

[Attachment 40702 - Click to enlarge]

greyscale:

[Attachment 40703 - Click to enlarge]

brightness/contrast:

[Attachment 40704 - Click to enlarge]

blur:

[Attachment 40705 - Click to enlarge]

cut/paste (offset here so you can see the section):

[Attachment 40706 - Click to enlarge]

pasted to make the bottom hole extend further up the frame:

[Attachment 40707 - Click to enlarge]

I used cut/paste operations to extend the hole left and right too. Then the top hole was enlarged similarly.

[manono]

I saved out a frame from the video, drew white on the parts I wanted white, drew black around the white and used a Gaussian Blur to feather it. You don't really have to be all that precise for what you need. This was done using the freeware PhotoFiltre although, of course, any photo editing program can do it.

[hand123]

I saved out a frame from the video, drew white on the parts I wanted white, drew black around the white and used a Gaussian Blur to feather it. You don't really have to be all that precise for what you need. This was done using the freeware PhotoFiltre although, of course, any photo editing program can do it.

Excellent work. And so that resulting image is what is used as a mask? Great. Thanks for the direction. I'll try my best.

[manono]

And so that resulting image is what is used as a mask?

Yes, just as jagabo used his black and white picture as a mask also. I blurred a version of the video after stretching it as I described earlier, and used that mask so the blurred part would show only where the white blocks were in the original, with the rest left alone.

[jagabo]

I didn't notice this before, but your video was originally captured on a device that stored the upper field below the lower field. It needs to have the field positions swapped for best viewing and editing. For example, the original video on the right, and after the Field Swap filter in VirtualDub (4x point resize):


[Attachment 40729 - Click to enlarge]

[jagabo]

Here is an example of the original video decimated to 18 fps and then converted to 60 fps with motion interpolation. This will give you some idea of the type of distortions you get with motion interpolation techniques.

Exactly how jerky the 18 fps version will play depends in part on what the refresh rate of you monitor or TV is. On a 60 Hz monitor some frames are viewed for 3/60 second, and some for 4/60 second. This adds some judder in addition to the jerkiness of the low frame rate. The judder could be eliminated by speeding the video up to 20 fps (each frame viewed for 3/60 second) or slowing it down to 15 fps (4/60 second per frame). But the low frame rate is still jerky.

Speeding or slowing the base frame rate also helps a bit when using motion interpolation. For example, if you speed up to 20 fps and motion interpolate to 60 fps, every third frame is an exact copy from the source and only the two in-between frames are interpolated.

[hand123]

Here is an example of the original video decimated to 18 fps and then converted to 60 fps with motion interpolation. This will give you some idea of the type of distortions you get with motion interpolation techniques.

This is very educational. Thanks for the samples. This is yet another example of some of the improvements I would like to see in my old films. I'm seeing some great things here and at the following link that I keep reading. http://www.thebattles.net/video/8mm_restoration.html