Fixing inconsistent A/V desync in old video

[Ygramul]

Source video

Direct download

I know the quality is absolute garbage, but I'm a fan of that bassist, so please help me out.

The synchronisation is all over the place and I don't know what to do about it. The video is at variable ~15 fps and simply adjusting audio delay in MPC-BE doesn't do anything. I'm not sure if the fact that a frame freezes for the last 5 seconds makes the issure worse, so please skip to the end to see for yourself.

I tried resampling FPS in Avidemux (codec: x264 -> Filters -> Transform -> Resample FPS -> Mode: 29,97; Interpolation: Blend). Still can't sync the two tracks.

[LigH.de]

There are some filters in video scripting engines like Avisynth or VapourSynth like source filters which force sync to a fixed frame rate while reading the video stream from the source (e.g. DirectShowSource or FFMS2 with fps related parameters) while keeping a constant speed reading the audio stream (Avisynth: EnsureVBRMP3Sync()). But still, there is no guarantee that it helps in an automatic workflow. Some sources cannot be fixed automatically anymore because YouTube already has processed them in a way that "cements" the mistakes.

[Sharc]

In Avisynth you may try timestretch for the audio, something like

v=LWLibavVideoSource("Emeth.mp4",fpsnum=25000,fpsden=1000).trim(0,4380) #make 25fps CFR by inserting (repeating) frames
a=LWLibavAudioSource("Emeth.mp4")
s=timestretch(a,tempo=102) #adjust the tempo in %
Audiodub(s,v)

https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/corefilters/timestretch.html

Or you can do similar in the timeline of an NLE.

[ProWo]

The source video is also asynchronous.
It is very difficult to find sync points for this video.
The videostream is still image only after frame 2699.

[Alwyn]

Here's my attempt. I did this in my NLE (I applied Santiag and RIFE to the video with AVISynth beforehand). I anchored the audio at the start and stretched the audio to fit. There are good sync check-points at 1:32 and at the very end (pronounced finger-picks, if that's what they're called ).

[Ygramul]

Here's my attempt. I did this in my NLE (I applied Santiag and RIFE to the video with AVISynth beforehand). I anchored the audio at the start and stretched the audio to fit. There are good sync check-points at 1:32 and at the very end (pronounced finger-picks, if that's what they're called ).

This looks great, thanks!

I guess I should've tried doing this in Premiere myself. I just thought there would be a simple ffmpeg command to achieve this effect.

P.S. the techniques at the end are slapping (hitting strings with your thumb) and plucking (lifting the string with your finger and immediately letting it go so that it his the frets).