Synchronize dubbed audio from bad TV recording with DVD rip

[abolibibelot]

Hi,

I am trying to add a french dubbed audio track from a bad VHS recording to a good quality DVD rip of the same movie, namely “The toughest man in the world” (not exactly a brilliant flick but my brother is a fan of the actor Mr. T). That movie was broadcast a few times in France but was never released commercially, and this is the only recording I could find. It plays at 25 FPS, but seems to have been converted through some sort of “telecine” process, I'm not sure how exactly (don't know much about these techniques, I can only see that some frames are blended). I would guess that it was converted from the US broadcast in 29.97 FPS, which was itself converted from the original 24 FPS movie. The non-dubbed MKV plays at 23.97 FPS in VLC Media Player but is strangely reported as 29.97 FPS by MediaInfo and 24.2517 by AVSPMod. The point is, both sources play at slightly different speeds and framerates, plus the introduction is totally different (same music but different shots), also there's a random shot near the begining which is surprisingly longer in the dubbed version (the camera pan lasts a few seconds more before it cuts to the next shot – that's the “++” in the script below), thus having both synchronized is turning into a nightmare. And the dubbed audio track is also bad, heavily clipped with a thick noise floor, but again it's the only source I've got; I already tried to improve as much as I could (declipping, noise reduction).

I tried the following :
– display both movies side-by-side in AVSPMod ;
– find a suitable image as a starting point for analysis and trim everything before ;
– go a few thousands frames further and try to adjust the framerate of the non-dubbed version to match the picture of the dubbed version.

VF = FFVideoSource("L'homme le plus fort du monde (1984 VF).avi").BicubicResize(640,480)
VF_cut = VF.trim(15952,23883) ++ VF.trim(24032,0)
VO_MKV = LWLibavVideoSource("K:\The Toughest Man In The World (1984).mkv").BicubicResize(640,480)
VO_MKV_cut_FPS = VO_MKV.trim(15496,0).ChangeFPS(25000,996).info()
StackHorizontal(VF_cut, VO_MKV_cut_FPS)

But I can't find a value which works consistently, the denominator has to vary between 987 and 1004 depending on the shot (and I only analysed a small portion of the movie).

Does that mean that the playback speed of the dubbed source is variable ? Or that frames were lost during the VHS conversion ? Or that slightly different edits of the original movie were used in the first place ?
What sense can I make of the different FPS values given for that non-dubbed MKV version ? (23.97 FPS / 29.97 FPS / 24.2517 FPS)
Is there a precise way to identify the “telecine” pattern, and, based on that, calculate a conversion factor to have (is possible at all) both versions synchronized ?
Since the end goal is to add the audio track from the dubbed version to the picture of the non-dubbed version, what would be the best way to do that ? Generally speaking is it better to change the speed of the audio (definitely seems to be the best approach if I want to include both the dubbed and the non-dubbed audio track in the modified video file), or to change the framerate to a non-standard value ? How to change the speed of the audio with no noticeable effect on the pitch ? (Audacity has an option which seems to do just that, which I'll have to try, but right now I don't know by how much the speed should be increased or decreased to roughly match the picture from begining to end.)

Here are untouched cuts from both versions (edited with Avidemux) :
VF 0:10:38.080-0:14:20.080
TTMITW VF 10;38.080-14;20.080.avi
VO 0:10:43.076-0:14:25.022
TTMITW VO 10;43.076-14;25.022.mkv

[jagabo]

Sections of the video(s) have been slowed and/or sped up, maybe cut. You won't be able to just change a single frame rate (or a simple audio length adjustment) to get the two to sync throughout. It's going to be a long arduous task.

[abolibibelot]

Thanks for the reply. Could you please elaborate ? How did you analyse the samples, and what did you see ?
Shouldn't it be audible if the speed was constantly varying ? (Although, admittedly, with such a crappy audio it'd be hard to notice anyhow.)
AVSPMod might not be the best tool to compare two similar videos with different framerates, as it only allows to play them frame by frame instead of relying on a common time reference. I'll try to load both into a NLE and see if I can get it roughly synchronized from there. Otherwise I'll give up – I really don't feel like getting lost in a long arduous task right now... é_è
But, for the sake of curiosity, what would that task entail ? Manually stretching / squeezing the audio shot by shot ? Or are there specific methods to deal with that kind of defect more efficiently ?

[johnmeyer]

I've done a lot of this. I assume you understand both languages.

1. Put your DVD rip (video and audio) on your NLE timeline.

2. Put your French audio track AND the VHS video track into the NLE as well. Use the video to help line up with the DVD rip. Also, use this to determine if there is material included in one that is not included in the other and cut and move the VHS/French version so it matches the DVD version.

3. Don't worry about the intro and the credits. Many dubbed films leave those in the original language.

4. Scrub quickly through the movie to find if the French audio is too slow or too fast. If it is either, use your NLE to stretch or compress the length of the audio. I use Vegas, and it can do this without changing pitch and, unless you need to change by more than 10%, this process creates no discernible artifacts.

You may have to cut the audio at certain points if there are discontinuities.

[jagabo]

You may have to cut the audio at certain points if there are discontinuities.

There are many. And the MKV source is VFR, further complicating matters.

[johnmeyer]

You may have to cut the audio at certain points if there are discontinuities.

There are many. And the MKV source is VFR, further complicating matters.

Oh, that does sound difficult. I was in a hurry and so didn't download his clips.

[abolibibelot]

Well, the MKV trimmed by Avidemux is indeed reported by MediaInfo as having a "Variable framerate", but the source MKV (as already mentioned) is reported as having a 29.970 FPS framerate. Yet the source MKV plays at “23.976216” FPS in VLC Media Player and is identified as 24.25 FPS in AVSPMod. The trimmed file uploaded in the first post plays at “30.303030” FPS in VLC Media Player, yet the audio is in sync. If I remake the trim with MKVToolNix, using default parameters, and analyse the output with MediaInfo, it still reports the framerate as “variable” but gives a value of 24.133 FPS, while the audio is also in sync ; if I set the framerate in MKVToolNix to 24000/1001, the reported framerate becomes “constant” at 23.976 FPS, while the file size is significantly reduced, and the audio is way out of sync. (Also I had to change the first timing for trimming, with the same value as in Avidemux it would cut several seconds further, at the next key frame I would guess.)

Source MKV, MediaInfo report :

Général
Identifiant unique                       : 92354692747076039069217499496311834882 (0x457ADDE85991052BE7BAA14284D24902)
Nom complet                              : The.Toughest.Man.In.The.World.1984.DVDRip.x264.mkv
Format                                   : Matroska
Version du format                        : Version 2
Taille du fichier                        : 1,57 Gio
Durée                                    : 1 h 34 min
Débit global moyen                       : 2 382 kb/s
Application utilisée                     : HandBrake 0.9.6
Bibliothèque utilisée                    : libmkv 0.6.5

Vidéo
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Profil du format                         : High@L3.1
Paramètres du format                     : CABAC / 9 Ref Frames
Paramètres du format, CABAC              : Oui
Paramètres du format, RefFrames          : 9 images
Identifiant du codec                     : V_MPEG4/ISO/AVC
Durée                                    : 1 h 34 min
Débit                                    : 2 111 kb/s
Largeur                                  : 708 pixels
Hauteur                                  : 480 pixels
Format à l'écran                         : 4/3
Format à l'écran d'origine               : 4/3
Type d'images/s                          : Constant
Images par seconde                       : 29,970 (29970/1000) Im/s
Espace de couleurs                       : YUV
Sous-échantillonnage de la chrominance   : 4:2:0
Profondeur des couleurs                  : 8 bits
Type de balayage                         : Progressif
Bits/(Pixel*Image)                       : 0.207
Taille du flux                           : 1,39 Gio (89%)
Bibliothèque utilisée                    : x264 core 120
Paramètres d'encodage                    : cabac=1 / ref=9 / deblock=1:-2:-2 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=6 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=18.5 / qcomp=0.60 / qpmin=3 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.20
Langue                                   : Anglais
Default                                  : Oui
Forced                                   : Non
Gamme de couleurs                        : Limited
Coordonnées de chromaticité              : BT.601 NTSC
Caractéristiques du transfert            : BT.709
Coefficients de la matrice               : BT.601

Audio
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Nom commercial                           : Dolby Digital
Identifiant du codec                     : A_AC3
Durée                                    : 1 h 34 min
Type de débit                            : Constant
Débit                                    : 224 kb/s
Canaux                                   : 2 canaux
Channel layout                           : L R
Echantillonnage                          : 48,0 kHz
Images par seconde                       : 31,250 Im/s (1536 SPF)
Profondeur des couleurs                  : 16 bits
Mode de compression                      : Avec perte
Taille du flux                           : 151 Mio (9%)
Langue                                   : Anglais
ServiceKind/String                       : Complete Main
Default                                  : Oui
Forced                                   : Non

Menu
00:00:00.000                             : Chapter  1
00:10:43.075                             : Chapter  2
00:21:46.388                             : Chapter  3
00:29:13.485                             : Chapter  4
00:38:54.115                             : Chapter  5
00:48:12.172                             : Chapter  6
00:55:30.860                             : Chapter  7
01:03:32.675                             : Chapter  8
01:08:44.320                             : Chapter  9
01:18:24.991                             : Chapter 10
01:33:25.049                             : Chapter 11

Sample trimmed by MKVToolNix, default settings :
TTMITW VO MKVToolNix 10;36-14;25.mkv
Sample trimmed by MKVToolNix, framerate set to 24000/1001 :
TTMITW VO MKVToolNix 10;36-14;25 FPS=24000,1001.mkv

Anyway, the movie should play at 23.976 FPS, right ? It couldn't possibly play at a truly “variable framerate”, right ? Where do these 24.25 FPS / 24.133 FPS values come from, then ?
And anyway, what matters here is the audio. How could the audio (which is an AC3 track taken straight from the DVD) be synchronized with a non-standard FPS value ?
According to ffmpeg :

Stream #0:0(eng): Video: h264 (High), yuv420p(tv, smpte170m/smpte170m/bt709, progressive), 708x480 [SAR 8:9 DAR 59:45], SAR 207:233 DAR 12213:9320, 23.98 fps, 23.98 tbr, 1k tbn, 180k tbc (default)

According to the “error.log” from AVFS (which I had to use as the NLE won't accept an MKV file as input) :

Video stream :-
  Duration:   137471 frames, 01:34:28.498
  ColorSpace: YV12
  Width: 708 pixels, Height: 480 pixels.
  Frames per second: 24.2517 (3468/143)
  FieldBased (Separated) Video: No
  Parity: Bottom field first.
  Field order: Unspecified
Audio stream :-
  Audio length: 272094720 samples. 01:34:28.640
  Samples Per Second: 48000
  Audio Channels: 2       
  Sample Type: Integer 16 bit

If I import both video files into the (Magix) NLE (direct import for the dubbed version, while the MKV file is imported through an AVS script loaded with AVFS), synchronize them precisely at a scene change near the begining, and scrub through the movie, I can see that it quickly goes out of sync (at this point I only check the picture), the MKV plays sligthly faster. Now if I listen to the audio, I notice that it gets progressively out of sync. It's also out of sync in VirtualDub2 with the same AVS script :

V = LWLibavVideoSource("The.Toughest.Man.In.The.World.1984.DVDRip.x264.mkv")
A = WAVSource("The.Toughest.Man.In.The.World.1984.DVDRip.x264_track2_eng_DELAY 0ms.wav")
AudioDub(V, A)

I simply extracted the AC3 track with gMKVExtract, and converted it to WAV with ffmpeg. If I import the WAV audio into the NLE and synchronize it with the non-dubbed video, it does have the same length and the wave form looks identical throughout. What's wrong here ? (And I thought that only the dubbed file would be an issue... I don't like it when there are so many moving parts and I don't know which one(s) I can rely on !...)

[jagabo]

For what it's worth, you can use ffprobe to generate a list of timestamps for each frame of a video. Put this in a batch file:

"G:\Program Files\ffmpeg\bin\ffprobe.exe" -threads %NUMBER_OF_PROCESSORS%*1.5  -v quiet -pretty -print_format compact -show_entries "frame=coded_picture_number,pkt_pts_time,pkt_duration_time,pict_type," -select_streams v:0 %1 > %1.txt

and you can drag/drop files onto it. The output will be a file with the same name but with .txt appended. It will look like:

frame|pkt_pts_time=0:00:00.277000|pkt_duration_time=0:00:00.033000|pict_type=I|coded_picture_number=0
frame|pkt_pts_time=0:00:00.327000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=3
frame|pkt_pts_time=0:00:00.361000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=4
frame|pkt_pts_time=0:00:00.411000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=2
frame|pkt_pts_time=0:00:00.444000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=5
frame|pkt_pts_time=0:00:00.494000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=6
frame|pkt_pts_time=0:00:00.527000|pkt_duration_time=0:00:00.033000|pict_type=P|coded_picture_number=1
frame|pkt_pts_time=0:00:00.578000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=9
frame|pkt_pts_time=0:00:00.611000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=10
frame|pkt_pts_time=0:00:00.661000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=8
frame|pkt_pts_time=0:00:00.711000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=11
frame|pkt_pts_time=0:00:00.761000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=12
frame|pkt_pts_time=0:00:00.794000|pkt_duration_time=0:00:00.033000|pict_type=P|coded_picture_number=7
...
frame|pkt_pts_time=0:03:48.088000|pkt_duration_time=0:00:00.033000|pict_type=B|coded_picture_number=5501
frame|pkt_pts_time=0:03:48.147000|pkt_duration_time=0:00:00.033000|pict_type=P|coded_picture_number=5498
frame|pkt_pts_time=0:03:48.188000|pkt_duration_time=0:00:00.033000|pict_type=I|coded_picture_number=5502

That was from the original MKV file that was provided in post #1. I believe the starting time of 0.277 ms is because of the negative audio delay (254 ms) rounded to the frame rate (or something like that). The pkt_pts_time appears to override the pkt_duration_time since the difference in successive pkt_pts_time is usually more than 33ms. If you divide the (last frame time - first frame time) by the number of frames you get (228.188 - 0.277) / 5502 = 0.0414233 seconds per frame, about 24.141 fps on average. That's the same frame rate that LWlibavVideoSource() gives.

I wrote a program to display the duration of each frame based on the difference in successive pkt_pts_time values (from the ffprobe text). It generated a list like:

0.050000
0.034000
0.050000
0.033000
0.050000
0.033000
0.051000
0.033000
0.050000
0.050000
0.050000
0.033000
...

The sum of all the deltas came to 227.911128. As expected, the same as the duration of the video stream to 3 decimal places: 228.188 - 0.277 = 227.911.

It would be interesting to see how the original MKV file looks to ffprobe.

[abolibibelot]

Well, I'm not sure what this all means, or entails, as I had never seen that kind of analysis, but here is the result :
The.Toughest.Man.In.The.World.1984.DVDRip.x264.mkv.txt

frame|pkt_pts_time=0:00:00.000000|pkt_duration_time=0:00:00.041000|pict_type=I|coded_picture_number=0
frame|pkt_pts_time=0:00:00.041000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=3
frame|pkt_pts_time=0:00:00.083000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=2
frame|pkt_pts_time=0:00:00.125000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=4
frame|pkt_pts_time=0:00:00.166000|pkt_duration_time=0:00:00.041000|pict_type=P|coded_picture_number=1
frame|pkt_pts_time=0:00:00.200000|pkt_duration_time=0:00:00.041000|pict_type=P|coded_picture_number=5
frame|pkt_pts_time=0:00:00.233000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=8
frame|pkt_pts_time=0:00:00.266000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=9
frame|pkt_pts_time=0:00:00.300000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=7
frame|pkt_pts_time=0:00:00.333000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=10
...
frame|pkt_pts_time=1:34:28.295000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=137465
frame|pkt_pts_time=1:34:28.329000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=137463
frame|pkt_pts_time=1:34:28.362000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=137466
frame|pkt_pts_time=1:34:28.396000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=137467
frame|pkt_pts_time=1:34:28.429000|pkt_duration_time=0:00:00.041000|pict_type=P|coded_picture_number=137462
frame|pkt_pts_time=1:34:28.462000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=137470
frame|pkt_pts_time=1:34:28.496000|pkt_duration_time=0:00:00.041000|pict_type=B|coded_picture_number=137469
frame|pkt_pts_time=1:34:28.529000|pkt_duration_time=0:00:00.041000|pict_type=P|coded_picture_number=137468

I can see that 1 : 0.033 = 30.303030 which is the framerate reported by VLC Media Player for that sample. But 1 : 0.041 = 24.39, yet the framerate reported by VLC for the original MKV is 23.976215.

At which level are those timings defined, container or elementary stream ? How do they relate with the audio stream(s) ? And what does this tell me about the intended task and the right way to accomplish it, if at all possible without wasting countless hours doing scene-by-scene adjustments which would be required as you seem to imply ?

[jagabo]

The table shows one line of text for each frame. The lines are in display order, so the first line is the first displayed frame, second line is the second displayed frame... the last line is the last displayed frame.

pkt_pts_time is the time at which the frame is displayed
pkt_duration_time is how long the frame is displayed
pict_type is the compressed frame type, I, B or P
coded_picture_number is the frame number in within the compressed video *

The table shows the last frame starts at 1:34:29.529 and it is frame number 137468 (first frame at time = 0). That makes the average duration of each frame 0.04123426 seconds, and the frame rate 24.3511 fps.

What does it all mean? From what I understand (and I'm not an expert in this) the original video isn't truly constant frame rate. If each frame was displayed for .041 seconds the full video would be 1:33:56.188 in length. What do players show the running time as?


* Out-of-order codecs store frames in decoding order, not in display order. For example, with a series of frames in display order IBBP, the I and P frames need to be decompressed before the two B frames can be decompressed. They are stored in decompression order IPBB so that the decoder doesn't have to seek forward to the P frame then seek back to the B frames, then seek forward again to decompress the next batch of frames. After decompression the frames are put back in display order, IBBP, for display.

[abolibibelot]

* Out-of-order codecs store frames in decoding order, not in display order. For example, with a series of frames in display order IBBP, the I and P frames need to be decompressed before the two B frames can be decompressed. They are stored in decompression order IPBB so that the decoder doesn't have to seek forward to the P frame then seek back to the B frames, then seek forward again to decompress the next batch of frames. After decompression the frames are put back in display order, IBBP, for display.

Thanks for this concise and clear explanation.

What does it all mean? From what I understand (and I'm not an expert in this) the original video isn't truly constant frame rate. If each frame was displayed for .041 seconds the full video would be 1:33:56.188 in length. What do players show the running time as?

In VLC Media Player or SMPlayer, 1:34:28.

But there seems to be a lot of duplicated frames in the original MKV (or the samples), at least when watched in VirtualDub2, they are identified by a “[+]” sign. If I check “Disable caching” in the options for “Caching input driver”, those frames appear all grey and there's a warning “Error reading source frame 19: requested frame not found; next valid frame = 20”.
But those frames appear to be valid in AVSPMod...
Anyway, I still don't understand what it all means. A movie (even a TV movie in this case) is always supposed to play at 24 FPS / 23.976 FPS – right ? Then were does this discrepancy come from ? Could it be an error in the production of the DVD ? Or an error in the DVD to MKV conversion ? Or something else entirely ? Can any of this be fixed, with or (preferably) without re-compressing ?

[jagabo]

But there seems to be a lot of duplicated frames in the original MKV (or the samples), at least when watched in VirtualDub2, they are identified by a “[+]” sign.

That's because VirtualDub thinks the file is 30.3030... fps. To keep the same running time it has to increase the number of frames to make up for the difference between the source's ~24 to ~30.

A movie (even a TV movie in this case) is always supposed to play at 24 FPS / 23.976 FPS – right ?

Not necessarily. NTSC DVD is always 29.97 (30000/1001) fps after pulldown, soft or hard. But the underlying frame rate of the content can be something other than 23.976 (24000/1001) fps.

Then were does this discrepancy come from ? Could it be an error in the production of the DVD ? Or an error in the DVD to MKV conversion ? Or something else entirely ?

Any of those things. We don't know the provenance of the file you have so we can't say for sure.

Can any of this be fixed, with or (preferably) without re-compressing ?

Fixed? Maybe. Without compression? Probably not.

[abolibibelot]

Fixed? Maybe. Without compression? Probably not.

So that means that the duration of each frame is hard-coded in the elementary video stream, and can't be adjusted at the container level, like the aspect ratio ?
Interestingly (?), if I remux the MKV to MP4

ffmpeg -i "H:\path\input.mkv" -i "H:\path\input.wav" -map 0:0 -map 1:0 -c:v copy -c:a aac "output.mp4"

MediaInfo analyses the resulting file as :
Type d'images/s : Variable
Images par seconde : 24,251 Im/s
Images/s mini : 14,925 Im/s
Images/s maxi : 30,303 Im/s
Looking for a potentially suitable switch inside ffmpeg's humongus “full help” text (at 800KB it's bigger than Dante Alighieri's Divine Comedy and a tad smaller than Fiodor Dostoyevsky's Crime and punishment), I tried -force_fps with several possible values (“24000/1001”, “24000:1001”, “23.976”), it failed with the error (for instance) “Unable to find a suitable output format for ’24000/1001’ 24000/1001: Invalid argument” ; also tried -framerate : it works but it doesn't change anything.

So if I do have to re-compress anyway, what is the best approach ? Would Avisynth's AssumeFPS(24000,1001) do the trick ? Or can this be done directly with ffmpeg ?

[jagabo]

The duration of the frames for VFR videos is part of the metadata at the containter level. But you can't just change a VFR video to a CFR video by making every frame have the average duration. Consider for example a video with 1000 frames at 20 fps followed by 1000 frames to 30 fps. The first 1000 frames plays in 50 seconds the second 1000 frames in 33.3 second, a total of 83.3 seconds. Say you remux and specify an average frame rate of 24 fps. The first 1000 frames will now play in 41.66 seconds, the second 1000 frames in 41.66 seconds, a total of 83.3 seconds. The audio will be in sync at the start and end but will be off in the middle.

The original mkv file you posted isn't that extreme but by experiment I found these values that give a CFR clip where the audio and video are in sync at the start and end:

audio = LWlibavAUdioSource("TTMITW VO 10;43.076-14;25.022.mkv")
video = LWlibavVideoSource("TTMITW VO 10;43.076-14;25.022.mkv").AssumeFPS(24.17)
AudioDub(video, audio)
DelayAudio(-0.35)

You'll see that the audio is off by a few hundred milliseconds in the middle.

[abolibibelot]

I get it but that's really weird...
I happen to have found (years ago actually) another non-dubbed version of lesser quality, this one in AVI / Xvid format, and this one has a standard framerate of 23.976 FPS. But the strange thing is that, if I synchronize both files and scrub through them frame by frame, I can see that the AVI one has duplicated frames, which are absent in the MKV one, while they have the same duration at 1:34:28. Shouldn't it be the other way around, i.e. shouldn't it be the MKV file having extra frames to account for the fact that it has a slightly higher (average) framerate ? Well, scratch that -- actually there are many duplicated frames on the MKV (and possibly blended frames too), I don't know how I missed them earlier.
Sample from the AVI file, approximately same cut points :
TTMITW VO 10;42.641-14;24.571.avi
AVI and MKV samples stacked with 142 frames trimmed on the MKV :

VO_AVI = FFVideoSource ("K:\TTMITW VO Avidemux 10;42.641-14;24.571.avi").BicubicResize(640,480).info()
VO_MKV = LWLibavVideoSource("K:\TTMITW VO Avidemux 10;43.076-14;25.022.mkv").BicubicResize(640,480).info().trim(142,0)
StackHorizontal(VO_AVI, VO_MKV)

– frames 346-347 are duplicated on the AVI ;
– frame 926 seems to be blended on the MKV (?), if both files are synchronized at that point the cut to the next shot happens 1 frame later ;
– frames 790-791 are duplicated on the AVI ;
– frames 1793-1794 and several frames between 1823 and 1832 seem to be duplicated on the MKV -- hard so say as it's a static shot with very little motion, but there's a significant shift introduced right afterward ;
– frames 1874-1875 seem duplicated on the MKV ;
– frames 1891-1892 seem duplicated on the MKV...
and probably many more...
Does this mean that the “inverse telecine” process was badly done on the MKV, and possibly also on the AVI ? Or was there something done wrong on the source DVD ? Or is it impossible to guess ?

But if I put either of those files and the dubbed one on the NLE's timeline, and synchronize them right after the introduction, they stay synchronized (give or take 1-2 frames) up until that discontinuity at around 16m00s. Then once the audio track is cut and adjusted they stay synchronized again up until another discontinuity at 31m45s : a weird glitch on the MKV, with several frames missing and a sudden crack in the audio track, in the middle of a shot (that glitch is also present on the AVI file so it must be on the source DVD).

[Attachment 49793 - Click to enlarge]

[Attachment 49794 - Click to enlarge]
So it should be manageable to get something watchable, if it takes a few adjustments every 15 minutes or so...

Some years ago (4 already, that's damn scary ! ), I had a similar issue, coincidentally with another movie featuring that same shakespearian actor : I had the commercial PAL DVD and tried to improve the horrendous picture quality, but gave up as it proved to be a nigh impossible task, so I resorted to, likewise, mux the dubbed audio (edited and processed separately) with a mediocre 700MB Xvid conversion of the non dubbed DVD, which was still much better visually (no color cast, balanced levels), as I couldn't find the NTSC DVD locally, and wouldn't have paid a hefty import fee for an even lousier movie. The synchronization was correct right away, except for one section where the audio on this commercial, official DVD suddenly switched to what seemed to be another movie (or some completely unrelated content anyway) for about 10 seconds... (I managed to fix that almost flawlessly by copy-pasting from the non dubbed track, as it was mostly crowd noise and “indistinct chatter”.) Which implies that 1) there are people actually getting paid for that kind of butcher job, and 2) apparently there's noone to notice and complain, not even Mr. T to pity those foools and predict them paaain.

[jagabo]

Does this mean that the “inverse telecine” process was badly done on the MKV, and possibly also on the AVI ?

Could be.

Or was there something done wrong on the source DVD ?

Maybe.

Or is it impossible to guess ?

Not reliably.

So it should be manageable to get something watchable[/QUOTE]
In theory, DirectShowSource("filename.mkv", ConvertFPS=true, fps=xx.xxx) should convert VFR to CFR without A/V sync errors (frames will be duplicated or decimated as necessary). It gave very jerky video with the MKV in your first post. Maybe it will work better with the original MKV. Unfortunately, DirectShowSource() isn't frame accurate when seeking. You may want to convert to an all intra-frame lossless intermediate with DirectShowSource() then use the intermediate for further processing.

But since your real task is to sync the French audio to the NTSC video you should try the simple method first. Find identifiable reference points near the start of the two audio tracks (the first spoken word, for example) and another pair near the end. Adjust the length of french audio track (Audacity?) so those time between those two reference points in the French audio match the time between those reference points in the NTSC audio. Then mux the original MKV contents with the new French audio (with appropriate audio delay to sync the start). Then check the resulting file.

If it works, great, you're done. If not you may have to work piecewise with the French audio to get it synchronized.