Complementing missing frames

[Bencuri]

I am about to start working on an old video clip. It has no serious issues, but there are some segments where the camera is showing the lead singer, and the lip sync was not set correctly. It is about 0.2-0.4 seconds off. I would like to shift those segments to align the video properly, but that will result in having gaps as well. Is there a way to complement them? Any AI tool that can do it?

[rgr]

Don't believe the ads
Current "AI" tools are not even suitable for upscaling a video, let alone filling in the gaps.

[aramkolt]

I think what you need to do is separate the audio and leave that alone first. Then break up your video clips into each camera angle switch and move them on the nonlinear editor timeline to get each clip to match the audio for that segment. That should result in all the video having correct audio sync at least. You might then have very small video gaps between clips and you'll have to come up with something where you freeze the frame or do some sort of transition to take up those gaps in the video.

[poisondeathray]

You can use motion interpolated frames to fill the gaps . There are ways in modern NLE's using optical flow, or morph cuts. They involve "tweening" between frames. A free way is Avisynth/vapoursynth , which have many threads discussing motion interpolated frames

I'm not aware of any automated tool that is accurate enough to sync automatically to the singer, and especially not for an "old video clip" which probably isn't great quality. ie. It would involve manual edits and manual identification of which/how many frames to interpolate