Help perfectly syncing two audio files

[teodargent]

Hello,
I want to add an audio file with a different language to a movie but the audio is out of sync. These are the files:

Audio 1 (original audio): language A, AAC, 5.1, FPS 23.97
Audio 2: language B, AC3, 5.1, FPS 23.97

I tried syncing the two files using Audacity and MKVToolNix. This is what I did:

This is how the two central channels look like in Audacity:


I shifted Audio 2 so that it was in sync with Audio 1:


By doing this I found out that Audio 2 was 4881 ms out of sync. I opened Audio 2 with MKVToolNix and wrote -4881 in "Delay", but for some reason the result isn't perfect: as you can see in the picture below, the two audio files are still out of sync by 12 ms.


I then reopened the shifted Audio 2 with MKVToolNix and entered -12 but that changed nothing: the result was identical to the input. What am I doing wrong?

[Cornucopia]

Languages don't all come out at the same speed; you have to accommodate some rushing/dragging, accumultion of lag, etc.
Editing one to fit the other is possible (have done many times) but it can often make the edited one have an unnatural cadence (too quick or too many pauses), unless you are fluently bilingual enough to make the proper global microadjustments.

Scott

[JVRaines]

Could it be that MKVToolNix is constrained by keyframes? At any rate, 12 ms will not be a noticeable delay for lip sync.

[sneaker]

Compare timecodes of all files and tracks.
(mkvextract timestamps_v2 "file.mkv" "0:track0.txt" "1:track1.txt")
Look at first timecodes.

Note that AC3 frames have a fixed length. (32ms at 48 kHz) You cannot arbitrarily cut them. But in Matroska you can use the timecodes for more accurate delaying (which mkvmerge does).

[teodargent]

Compare timecodes of all files and tracks.
(mkvextract timestamps_v2 "file.mkv" "0:track0.txt" "1:track1.txt")
Look at first timecodes.

Note that AC3 frames have a fixed length. (32ms at 48 kHz) You cannot arbitrarily cut them. But in Matroska you can use the timecodes for more accurate delaying (which mkvmerge does).

I managed to get the timecodes but I don't know how to used them, I'm new to this. Can you help me up, please?

[sneaker]

I meant look at them to get an understanding about the problem.

Is the original file also mkv? How do you load them from mkv into Audacity?

[teodargent]

I meant look at them to get an understanding about the problem.

Is the original file also mkv? How do you load them from mkv into Audacity?

There are 2 files:

file 1.mkv (duration -> 02:18:16): video 1 (HEVC, 23,976 fps), audio 1 (acc, language A)

file 2.mkv (duration -> 02:18:55): video 2 (HEVC, 23,976 fps), audio 2 (ac3, language B)

My goal is to have:

file 3.mkv: video 1, audio 1, audio 2.

I am attaching the timestamps.

I load the audios in audacity by draging the .mkv files.

[sneaker]

4 tracks should equal 4 timecode files.

[teodargent]

4 tracks should equal 4 timecode files.

oh ok, sorry about that.

[sneaker]

Audacity doesn't obey the mkv timecodes. audio 1 starts 20ms after the video so your delay calculation was wrong. Then it seemingly didn't do anything when you remuxed with another -12ms delay because the first timecode must have been >= 12ms. Since no packet was deleted and Audacity doesn't obey the mkv timecodes it looked like mkvmerge did nothing when in fact it did.

It should be correct if you use -4861ms delay. But you cannot verify in Audacity alone for the reason mentioned above (but you can extract timecode of your final file, look at first timecode and add that to the delay you see in Audacity. Then verification is possible.)

[teodargent]

That makes sense. I tried and it looks like it is working. Thanks a lot! I have two more questions:
1. Is there a program like audacity that accounts for the mkv timecodes?
2. Is there a GUI for extracting the timecodes?

[sneaker]

1. IDK. Maybe it would make sense to ask Audacity team to change this? Personally, if I wanted this I ran the file through AviSynth+ffaudiosource() to wav, then opened the wav in Audacity.
2. gMKVExtractGUI

[teodargent]

ok, thanks again!

[teodargent]

1. IDK. Maybe it would make sense to ask Audacity team to change this? Personally, if I wanted this I ran the file through AviSynth+ffaudiosource() to wav, then opened the wav in Audacity.
2. gMKVExtractGUI

Sorry for bothering again, I tried to use AviSynth + ffms2 but I get an error. I installed Avisynth, then copied ffms2.dll and ffms2.lib in Avisynth\plugins, then created a test.avs file containing this code:

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\ffms2.dll")
ffaudiosource("C:\Users\teo\Desktop\test.mkv")

When I open the test.avs with windows media player or media player classic I get this error: "LoadPlugin: unable to load "C:\Program Files (x86)\AviSynth\plugins\ffms2.dll", error=0xc1 (C:\Users\Teo\Desktop\test.avs, line 1)

I can't figure out what to do. Can you help me, please?

[sneaker]

1. Make sure you are using 32 bit plugin with 32 bit AviSynth and 32bit MPC or 64 bit plugin with 64 bit AviSynth and 64bit MPC. You cannot mix those.
2. You may be missing some redistributables. Try: https://aka.ms/vs/15/release/VC_redist.x86.exe (for 32 bit ffms/AviSynth) https://aka.ms/vs/15/release/VC_redist.x64.exe (for 64 bit ffms/AviSynth)

If that does not lead to success run avsmeter -avsinfo to further debug the problem.

[teodargent]

1. Make sure you are using 32 bit plugin with 32 bit AviSynth and 32bit MPC or 64 bit plugin with 64 bit AviSynth and 64bit MPC. You cannot mix those.

That was the issue. It's working now, but I cannot open it with audacity (I have FFmpeg import/export library installed and setup in Audacity).

[sneaker]

I don't know if it's possible to open AviSynth scripts directly in Audacity. If it's not you can:
1. write script to wav first, e.g. avs2pipemod -wav "script.avs" > "output.wav"
2. or use AviSynth Virtual File System (avfs)

[teodargent]

1. write script to wav first, e.g. avs2pipemod -wav "script.avs" > "output.wav"

That worked! I loaded the same track in Audacity twice, the first time directly from the .mkv file, the second time I used the .wav. Comparing the two tracks, the second had a 20 ms delay, same as the timestamp.

I noticed one thing, though. When trying to sync two tracks in two different languages, the delay is not the same in every point of the tracks. For example, in some points it might be 12ms in others 5ms and so on. I'm not comparing the spoken parts of course, only the parts containing the same sound effect. I thought that was because of the timestamps (I don't know how they work) but now I'm guessing that they aren't very precise when dubbing.

Anyway, thanks again for the help!