Muxed Audio Slowly Goes Out Of Sync

[StaticSense]

Alright, So I figured I'd come to the experts on this one. Wasn't sure where to put this, as I guess it's technically a video question too, but this seemed more fitting.

I'm trying to take a foreign dubbed video, and mux in an english audio track. I'm by no means an expert, but I have some experience with muxing/syncing, but can't figure this one out. I assume this has something to do with the audio/video possibly being sped up or slowed down, and a ntsc vs pal issue, but not positive.

Video 1 - 59.940 FPS - 48.0 Khz
Video 2 - 23.976 FPS - 44.1 Khz

I've tried a few different methods. I've converted the audio frequency of video 2, to match video one, but that didn't do it. I also converted the video 1 fps, to match video 2, but no luck. Essentially what happens is, I sync up the audio, and it seems to be synced for about 10 seconds or so, and then it loses sync (this is while i'm working on it using avidemux).

I work with a few different tools when I mux audio; avidemux, mkvtoolnix, and audacity. I've ran into this problem before, but never looked into it, so I figured I'd ask this time.

I'm wondering if there's any way to correct this, and if not, does anybody know of a better method to splice the files in 10 second increments and merge them? avidemux often has issues with precise cuts, and relies on keyframes most of the time, and that wouldn't work out.

Thanks!

[davexnet]

You say you've done this before, yet you have not mentioned one of the most the most important things:
the running time of the original and the running time of the new audio

[johns0]

If the other audio has different scene changes in timing then the other video/audio then it's not going to be easy to fix since you will need to edit sections that are around 6-10 minutes throughout the video.

[StaticSense]

You say you've done this before, yet you have not mentioned one of the most the most important things:
the running time of the original and the running time of the new audio

I've done this before, but I don't rely on matching lengths. I normally use avidemux, sync the audio, and go from there. I skim through the video, and if everything is in sync, then it's all good, and if not, I splice the video at the moment it starts going out of sync, and i adjust the audio once again, and so on. I then combine all of the files.

[StaticSense]

If the other audio has different scene changes in timing then the other video/audio then it's not going to be easy to fix since you will need to edit sections that are around 6-10 minutes throughout the video.

The videos appear to be identical, and I've been going through them very closely, to see if there are any changes. It's possible that there are some dropped frames, which could be causing this issue, but in terms of full scenes, definitely not that. I wish it were 6 minutes chunks, but it appears to lose sync after 10 seconds or so, which makes me think that the audio/video has been sped up/slowed down ever so slightly.

[davexnet]

You say you've done this before, yet you have not mentioned one of the most the most important things:
the running time of the original and the running time of the new audio

I've done this before, but I don't rely on matching lengths. I normally use avidemux, sync the audio, and go from there. I skim through the video, and if everything is in sync, then it's all good, and if not, I splice the video at the moment it starts going out of sync, and i adjust the audio once again, and so on. I then combine all of the files.

But the length may give a clue as to what to target, it may infer a difference due to to slowdown/speedup (frame rate related)
or it may be arbitrary (the possibilty of extra scenes, closing or ending titles,etc.)

[StaticSense]

You say you've done this before, yet you have not mentioned one of the most the most important things:
the running time of the original and the running time of the new audio

I've done this before, but I don't rely on matching lengths. I normally use avidemux, sync the audio, and go from there. I skim through the video, and if everything is in sync, then it's all good, and if not, I splice the video at the moment it starts going out of sync, and i adjust the audio once again, and so on. I then combine all of the files.

But the length may give a clue as to what to target, it may infer a difference due to to slowdown/speedup (frame rate related)
or it may be arbitrary (the possibilty of extra scenes, closing or ending titles,etc.)

I definitely see what you're saying, but I can confirm it's not that, or at least nothing significant. There's a 14 second difference between the videos, but that's just because of the end credits on one of the videos, which goes on a little longer. I've played the videos side by side, and they stay in sync for 10 seconds or so, and then slowly lose sync, but there's no "scene" change, commercial breaks, nothing, it's just in the middle of a scene. Which makes me think that the video is either sped up/slowed down slightly, or there's some dropped frames that aren't noticeable during playback. I guess i could technically go frame by frame, and see where the break lies, but that may take a while.

Is there a problem with muxing audio from a 23.976 fps video to a 59.940fps video just in general? does frequency play a factor in that as well? If one is 48.0, and the other is 44.1Khz, would that alone cause an issue? or can all of those things be mixed without issue, and an issue like this would solely have to be the result of the actual video being sped up/slowed down and missing frames?

Sorry for all the questions, just trying to learn.

[davexnet]

Hollywood movies are shot at 24 FPS,
if you look at the 59.9 FPS version frame by frame in Virtualdub2, probably find a lot of repeated frames.

The 59.9 FPS version could have been sourced from a 24 or a 23.976 file - giving a 5 second offset by the end of the movie.
The sampling rate of the audio is irrelevant.

[StaticSense]

Hollywood movies are shot at 24 FPS,
if you look at the 59.9 FPS version frame by frame in Virtualdub2, probably find a lot of repeated frames.

The 59.9 FPS version could have been sourced from a 24 or a 23.976 file - giving a 5 second offset by the end of the movie.
The sampling rate of the audio is irrelevant.

So if a 59.9 FPS video was sourced from a 24 fps video, there's guaranteed to be a 5 second difference by the end of the movie, is what you're saying?
I guess my only option is to find these repeated frames that you speak of, and remove them. Is there a program where I can remove frames, that precisely? Can I do that with virtualdub2?

[davexnet]

You don't need to remove the frames because they don't affect the running time.
The 5 second difference assumes, of course, the exact same cut of the movie.

I would find 2 frames that you can identify, one as close to the start as possible and one as close to the end.
Find these frames in each version of the movie and see how many seconds offset they have

[StaticSense]

You don't need to remove the frames because they don't affect the running time.
The 5 second difference assumes, of course, the exact same cut of the movie.

I would find 2 frames that you can identify, one as close to the start as possible and one as close to the end.
Find these frames in each version of the movie and see how many seconds offset they have

Alright, I did what you said, it's actually a show, not a movie:


Video 1 - 59.940 FPS - 48.0 Khz

frame start - 00:11.645
Frame end - 21:22.656

difference = 21:11.011


Video 2 - 23.976 FPS - 44.1 Khz

frame start - 00:12.303
frame end - 21:14.147

difference = 21:1.844


the start time is slightly different, just because of the intro start point, but again, I've been using avidemux to sync, so i'm seeing the audio and video at the same time, and syncing it up. Within 10 seconds or so, of the video being in sync, I notice the sync start to drift away (while playing side by side). I know you are saying that dropped frames wouldn't affect duration, but wouldn't it technically? for instance, if I were to remove 10 frames, that means there's 10 less frames of video, and wouldn't it technically cause the audio to eventually become out of sync, because those frames are in one of the videos, but not in the other?

[davexnet]

If you don't mind sharing you can upload the original video and the new audio and I'll take a look.
If you'd rather do it privately you can always upload to a hosting site and PM me the location

[StaticSense]

If you don't mind sharing you can upload the original video and the new audio and I'll take a look.
If you'd rather do it privately you can always upload to a hosting site and PM me the location

I've sent you a PM. I'd rather keep the contents private, but of course, we should continue to discuss the issue here, in order to help people in the future. Thanks for all the help again!

[davexnet]

Used offset and time stretch (Change Tempo in Audacity)
It's in sync beginning and end, wanders in and out during the middle.
Best I could do without dividing it into pieces

Send link via PM

[StaticSense]

Used offset and time stretch (Change Tempo in Audacity)
It's in sync beginning and end, wanders in and out during the middle.
Best I could do without dividing it into pieces

Send link via PM

Thanks so much, can't wait to see the result.

Because i'm no expert, i'd love to hear how the process actually works, what you did to correct it, and what the problem actually was.

Do you have any idea why it's wandering off throughout the middle of the video?

I'll wait until I take a look at it, before I ask further questions.

[davexnet]

Used offset and time stretch (Change Tempo in Audacity)
It's in sync beginning and end, wanders in and out during the middle.
Best I could do without dividing it into pieces

Send link via PM

Thanks so much, can't wait to see the result.

Because i'm no expert, i'd love to hear how the process actually works, what you did to correct it, and what the problem actually was.

Do you have any idea why it's wandering off throughout the middle of the video?

I'll wait until I take a look at it, before I ask further questions.

I forgot to update the thread, there was a similar thread posted today that made me think of it.
My method did not work, the edit of the source file had a bunch of joins that threw the sync out,
and the OP had to manually sync it with a series of small edits and merge them all after it was done.
It ended up with a successful sync.

Perhaps if he sees this update he'll comment in the other thread and lend his expertise.
https://forum.videohelp.com/threads/393990-Synchronize-dubbed-audio-from-bad-TV-record...g-with-DVD-rip

[abolibibelot]

Just seeing this thread – indeed, it's a quite similar issue.

[Jay55]

The easiest way would be to use a multitrack video editor like Magix (formerly Sony) Vegas.

It has a simple timestretch feature (hold CTRL-drag end of audio clip and just move it - audio will stretch with or without adjusting pitch). With the audio waveform of both visible you can stretch the new audio to match the old audio. Then if there are issues in the middle you can make small cuts in the audio, make an adjustment, and then adjust the stretch to fix the sync to the end.

Not free....but.....your sources may have dropped frames/glitches that can make doing it timing based not work properly.