How do I sync Pal Audio to 30FPS NTSC Video?

[Reclusive Eagle]

Hi, so I have a slight issue and none of the old tools, such as dgpulldown or BeSweet currently work. At least not for me.

I'm doing some restoration work on a series. This series has a 25fps PAL English DVD release and a 30 fps NTSC Japanese Blu-ray both are interlaced. The English DVD was very badly encoded and the Japanese Blu-ray release is far superior in terms of image quality. So I would like to use the Japanese Blu-ray with the English Audio.

From what I understand, had the Blu-ray been at 24.97 fps, I could have slowed the Audio by 4.1% and pitched up the audio to 104.1% or applied a pulldown. Unfortunately that is not the case.

Requirements:

Ideally I would prefer to keep the video at the current 30fps NTSC frame rate. If that's not possible then either 24.97 fps with synced audio or 25fps is fine. I have attempted to use MKVToolNix and Lossless cut to change the FPS to 24001/1001 and 25fps, but within 5 seconds the audio becomes out of sync with the video.

Is there anyway I can get PAL Audio to sync to 30FPS NTSC video?

Media Info

Media Info for PAL DVD

Format : MPEG-PS
File size : 1 024 MiB
Duration : 5 min 23 s
Overall bit rate mode : Constant
Overall bit rate : 26.5 Mb/s
Frame rate : 25.000 FPS

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings : BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Default
Format settings, GOP : M=3, N=13
Format settings, picture structure : Field
Duration : 5 min 23 s
Bit rate mode : Constant
Bit rate : 25.8 Mb/s
Maximum bit rate : 5 000 kb/s
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 2.487
Time code of first frame : 01:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Open
Stream size : 995 MiB (97%)
Color primaries : BT.601 PAL
Transfer characteristics : BT.470 System B/G
Matrix coefficients : BT.470 System B/G

Audio
ID : 189 (0xBD)-128 (0x80)
Format : AC-3
Format/Info : Audio Coding 3
Commercial name : Dolby Digital
Muxing mode : DVD-Video
Duration : 5 min 23 s
Bit rate mode : Constant
Bit rate : 224 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 8.64 MiB (1%)
Service kind : Complete Main
Dialog Normalization : -31 dB
compr : -0.56 dB
dynrng : -0.56 dB
dialnorm_Average : -31 dB
dialnorm_Minimum : -31 dB
dialnorm_Maximum : -31 dB


Media Info for NTSC Blu-ray

Format : Matroska
Format version : Version 2
File size : 1.45 GiB
Duration : 24 min 22 s
Overall bit rate mode : Variable
Overall bit rate : 8 539 kb/s
Frame rate : 29.970 FPS
Movie name : 東京ミュウミュウ BD-BOX DISC1
Encoded date : 2020-05-01 13:02:01 UTC
Writing application : MakeMKV v1.15.1 win(x64-release)
Writing library : libmakemkv v1.15.1 (1.3.5/1.4.7) win(x64-release)

Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 6 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 6 frames
Format settings, Slice count : 4 slices per frame
Codec ID : V_MPEG4/ISO/AVC
Duration : 24 min 22 s
Bit rate mode : Variable
Bit rate : 7 000 kb/s
Maximum bit rate : 10 000 kb/s
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Original display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Scan type, store method : Interleaved fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.676
Stream size : 1.19 GiB (82%)
Writing library : x264 core 155
Encoding settings : cabac=1 / ref=6 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=1 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / slices=4 / nr=0 / decimate=1 / interlaced=tff / minigop=1 / stitchable=1 / constrained_intra=0 / bframes=3 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=1 / weightp=0 / keyint=60 / keyint_min=1 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=7000 / ratetol=1.0 / qcomp=0.60 / qpmin=3 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=10000 / vbv_bufsize=10000 / nal_hrd=vbr / filler=0 / ip_ratio=1.40 / aq=1:1.00
Language : English
Default : No
Forced : No
Color range : Limited
Color primaries : BT.601 NTSC
Transfer characteristics : BT.601
Matrix coefficients : BT.601
Original source medium : Blu-ray

Audio
ID : 2
ID in the original source medium : 4352 (0x1100)
Format : PCM
Format settings : Little / Signed
Codec ID : A_PCM/INT/LIT
Duration : 24 min 22 s
Bit rate mode : Constant
Bit rate : 1 536 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 30.000 FPS (1600 SPF)
Bit depth : 16 bits
Stream size : 268 MiB (18%)
Title : Stereo
Language : Japanese
Default : Yes
Forced : No
Original source medium : Blu-ray

More information:
Apparently it's slightly more complicated.

The English 25FPS PAL DVD has the audio set 31.250 FPS yet syncs perfectly to the 25 FPS PAL video. What SPF value would sync to 30 FPS?
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)

[davexnet]

What is the running time, to the nearest second, of both versions?

[johns0]

The problem you most likely have is the scene changes have different running times between both versions so your audio won't line up unless you edit the scene changes so the times are the same,that means a lot of editing.

[Reclusive Eagle]

What is the running time, to the nearest second, of both versions?

Well the Japanese Blu-ray has a slightly different episode structure and is uncut compared to the PAL DVD, so I will have to splice key frames to match the video, however the shared segments seem to be about the same time.

I extracted the same segment matched by key frames and they both turn out to be 1 min 38 seconds.

[Reclusive Eagle]

The problem you most likely have is the scene changes have different running times between both versions so your audio won't line up unless you edit the scene changes so the times are the same,that means a lot of editing.

Which I am fine with. I will have to extract and concat keyframes from the Blu-ray to recreate parts of the English DVD. But the parts I am testing and have extracted have no cuts or any differences apart from the fps. So it's matched key frame for key frame.

[hello_hello]

You don't need to pay any attention to the audio frame rate. Lossy audio is stored in small segments called "frames" but it has nothing to do with the frame rate of the video.

31.250 FPS for the audio only refers to the audio "frames", and 1536 SPF (samples per frame) means each one contains 1536 audio samples.
31.250 x 1536 = 48000 or 48.0kHz.... the sample rate of the audio.

I'm not sure why MediaInfo displays a frame rate for the PCM audio. Maybe it's something to do with the way it's interleaved with the video in an MKV container, but MediaInfo will show 31.250 FPS (1536 SPF) for the AC-3 audio stream regardless of the video frame rate.

Not knowing what the video is I'd guess it could've been 23.976fps to begin with. If that's the case it would have been converted to 29.97fps by applying 2:3 pulldown so the duration wouldn't have changed.
Assuming the PAL version was created by increasing the frame rate from 23.976fps to 25fps and the audio was sped up to match, all you should need to do is slow it back down to the original speed.

So that's where I'd start. Re-encode the audio with your favorite converter while applying the usual PAL to NTSC 4.1% slowdown, and I'd recommend slowing it without pitch correction unless you know it was pitch corrected when it was sped up, which it very, very likely wasn't, so slowing it down without pitch correction will give you the original pitch. I know MKVToolNix lets you apply some sort of stretch to the audio (you can't specify an audio frame rate as such, just a stretch factor) but I wouldn't recommend it due to the possibility of playback glitches.

Of course there's a multitude of horrors that video can be subjected to in respect to PAL <-> NTSC conversions so it mightn't be so simple, and unless you're very, very, very, very lucky the DVD and Bluray versions will have been edited slightly differently and you'll need to edit either the audio or video so they'll match (after you've slowed the audio), but one bridge at a time....

[davexnet]

The first thing is to understand whether they're running at different speeds or not.
Match up some footage where the start and end point matches at the same frame and compare the times.

As hello_hello alludes to, much of the details you have given are irrelevant

[Reclusive Eagle]

You don't need to pay any attention to the audio frame rate. Lossy audio is stored in small segments called "frames" but it has nothing to do with the frame rate of the video.

31.250 FPS for the audio only refers to the audio "frames", and 1536 SPF (samples per frame) means each one contains 1536 audio samples.
31.250 x 1536 = 48000 or 48.0kHz.... the sample rate of the audio.

I'm not sure why MediaInfo displays a frame rate for the PCM audio. Maybe it's something to do with the way it's interleaved with the video in an MKV container, but MediaInfo will show 31.250 FPS (1536 SPF) for the AC-3 audio stream regardless of the video frame rate.

Not knowing what the video is I'd guess it could've been 23.976fps to begin with. If that's the case it would have been converted to 29.97fps by applying 2:3 pulldown so the duration wouldn't have changed.
Assuming the PAL version was created by increasing the frame rate from 23.976fps to 25fps and the audio was sped up to match, all you should need to do is slow it back down to the original speed.

So that's where I'd start. Re-encode the audio with your favorite converter while applying the usual PAL to NTSC 4.1% slowdown, and I'd recommend slowing it without pitch correction unless you know it was pitch corrected when it was sped up, which it very, very likely wasn't, so slowing it down without pitch correction will give you the original pitch. I know MKVToolNix lets you apply some sort of stretch to the audio (you can't specify an audio frame rate as such, just a stretch factor) but I wouldn't recommend it due to the possibility of playback glitches.

Of cource there's a multitude of horrors that video can be subjected to in respect to PAL <-> NTSC conversions so it mightn't be so simple, and unless you're very, very, very, very lucky the DVD and Bluray versions will have been edited slightly differently and you'll need to edit either the audio or video so they'll match (after you've slowed the audio), but one bridge at a time....

So I just extracted the audio for both the PAL DVD and NTSC Blu-ray, synced them to start at the exact same moment and you'll notice the PAL DVD audio (Bottom Track) is slightly longer for the same scene.


[Attachment 81112 - Click to enlarge]

How can a 25 fps video have a longer runtime than the 23.97 fps NTSC?

NTSC audio runs for 19.806s for selection
PAL audio runs for 20.380s for the same section

So the PAL audio is around 2.89% faster for this selection

Edit:
Now it gets even more interesting.

The above clip is extracted from a longer uninterrupted scene. Both the PAL and NTSC have this scene unmodified in respect to each other.
However, while in the above clip the PAL version is longer by 2.89%, the uninterrupted 1m 36s scene shares the exact same runtime between the PAL and NTSC versions.


[Attachment 81113 - Click to enlarge]

My only guess is that in the PAL version, extra frames have been added or dropped to sync the audio per key frame? Or is there another simple explanation? Could also be that in addition to the PAL conversion parts of the PAL audio were sped up or slowed to match the PAL frames.

- NTSC has 2836 Frames for this section. Key frames are very irregular.


[Attachment 81115 - Click to enlarge]



- PAL has 2353 Frames for this section with evenly spaced key frames.

[Attachment 81116 - Click to enlarge]

Both are 8x zoom in LosslessCut

[ProWo]

From post #1:

Media Info for PAL DVD:
Duration : 5 min 23 s
Frame rate : 25.000 FPS

Media Info for NTSC Blu-ray
Duration : 24 min 22 s
Frame rate : 29.970 FPS

It is not possible to exchange the audio tracks with this data.
Something must be wrong here.

[Reclusive Eagle]

From post #1:

Media Info for PAL DVD:
Duration : 5 min 23 s
Frame rate : 25.000 FPS

Media Info for NTSC Blu-ray
Duration : 24 min 22 s
Frame rate : 29.970 FPS

It is not possible to exchange the audio tracks with this data.
Something must be wrong here.

The Blu-ray is an mkv file while the DVD is in ISO format. I just extracted one of the .VOB files. The Blu-ray is reporting the correct time for the episode but the DVD is reporting, probably the menu time, and not the actual episode time.

[ProWo]

From post #1:

Media Info for PAL DVD:
Duration : 5 min 23 s
Frame rate : 25.000 FPS

Media Info for NTSC Blu-ray
Duration : 24 min 22 s
Frame rate : 29.970 FPS

It is not possible to exchange the audio tracks with this data.
Something must be wrong here.

The Blu-ray is an mkv file while the DVD is in ISO format. I just extracted one of the .VOB files. The Blu-ray is reporting the correct time for the episode but the DVD is reporting, probably the menu time, and not the actual episode time.

Extract the full DVD movie as mpg and post the mediainfo from this mpg here.

[hello_hello]

Reclusive Eagle,

I'll confess I'm confused by the information in your last post, but I don't think anyone will be able to give you useful advice without seeing the videos. Can you upload a couple of short samples from both the DVD and Bluray versions? You can attach them to a post here.

Regarding keyframe placement.... older (and some newer) encoders create keyframes at regular intervals. Newer encoders generally try to be more clever. For the x264 encoder, as an example, the defaults are a minimum of 1 second between keyframes and a maximum of 10 seconds (at 25fps), but it puts a keyframe on the first frame of each scene so the distance between them is quite irregular.

For the screenshots showing the keyframes, 2836 frames at 29.97fps is 94.629 seconds, and 2353 frames at 25fps is 94.12 seconds. It's not much, but the durations are slightly different (a bit more than a single PAL frame different).

It's possible the NTSC version (assuming the original was NTSC) was converted to PAL using a method other than simply speeding it up, and without necessarily changing the duration, so the two versions could have the same duration in theory, but they might have been edited differently. A couple of samples would be helpful. Just a minute or two from each one should do.