audio out of sync after reversing pal->ntsc telecining

[codemaster]

Source is NTSC, and the audio track is in sync with the video track. File extension is .m2ts and duration is 01:45:32. File size is huge, and I want to transcode it, to reduce file size, but I encountered a problem.

While analyzing part of the source frame by frame in VirtualDub2, to check if it is interlaced or progressive, I noticed it is progressive and every 6th frame is a duplicate of the previous frame. So it was filmed PAL, and someone deinterlaced it, and then converted it to NTSC progressive.

After decimating and increasing speed from 24.98 fps to 25 fps, the video track duration is the same as the duration of the source audio track. But the audio is out of sync.

I extracted the audio with ffmpeg using these parameters:

ffmpeg -i infile.m2ts -map 0:1 -codec:a pcm_s24le outfile.wav

This resulted in a .wav file with the duration 01:45:26. So the audio inside the .m2ts file, is 6 seconds shorter than the duration of the .m2ts file.

I reversed PAL->NTSC telecining with this AviSynth+ script:

SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("MPEG2Source", 3)
MPEG2Source("index.d2v")
tdecimate(cycle=6, cycler=1)
AssumeFPS(25, 1, false)
Spline64Resize(854, 480)
Prefetch(4)

And encoded with x264 using that .avs script and these parameters:

x264 --preset slower --tune film --crf 23 --colormatrix bt709 --min-keyint 25 --keyint 250 --output outfile.264 infile.avs

This resulted in a video file with the duration 01:45:26. The same duration as the audio, but audio is out of sync with this video file.

Then I used ffmpeg with these parameters to reverse PAL->NTSC:

ffmpeg -i infile.m2ts ^
-r 25 ^
-filter:v "decimate=cycle=6, setpts=N/(25*TB), zscale=854:480:f=spline36, setsar=1" ^
-codec:v libx264 -preset slower -tune film ^
-x264-params crf=23:min-keyint=25:keyint=250:colormatrix=bt709 ^
-an ^
outfile.mp4

This resulted in a video file with the duration 01:45:26. The same duration as the audio, but audio is out of sync with this video file.

FFmpeg filters produced the same result as the AviSynth+ filters: video track with same duration as audio, but out of sync with audio.

What else can I try, that should result in a video file that is in sync with the original audio?

[johns0]

The audio is out sync since it is 6 seconds shorter,you will have to shorten the audio as well.

[davexnet]

Perhaps a mediainfo (text view) of the original file may help the analysis

[Luke M]

You sped up the video by 1.001, so you need to speed up the audio by 1.001.

For example using ffmpeg option:

-af asetrate=48048,aresample=48000

[manono]

After decimating and increasing speed from 24.98 fps to 25 fps...

Yet you didn't do that with the audio as well?
Much easier is to do the audio speed up within the AVISynth script. Then you can save out a WAV file.

But it begs the question - why keep it at PAL speed to begin with since it's so easy to slow it to film speed?

[codemaster]

I sped up audio 1.001 x with sox:

sox infile.wav outfile.wav tempo -m 1.001

Now audio is in sync with my 25 fps transcoded video, encoded from 29.97 fps source.

@manono: I chose pal speed because it was recorded at 25 fps or 50 fps, and slowed down 0.999x to make a ntsc bluray. If I slow it to film speed, I slow it 0.96 x. Content is live concert, and I confirmed that original speed is 25 fps by looking at the waveforms in kdenlive, using the live concert and the studio version of one of the songs. Studio song waveform matches with live song waveform at 25 fps speed, and doesn't match with live song waveform at 24 fps speed.

[jagabo]

Since you're using AviSynth why not just adjust the audio length with AssumeFPS?

http://avisynth.nl/index.php/AssumeFPS#AssumeFPS

AssumeFPS(25, 1, true) # adjusts length but leaves you with a non standard sample rate
ResampleAudio(48000) # back to a standard sample rate, 44100, 48000, etc.

[codemaster]

why not just adjust the audio length with AssumeFPS?

Because it doesn't allow me to use the best resampler. And because I choose high quality over simplicity. Also, I don't know how to change speed without changing pitch in AviSynth+, but I know how to do it in SoX, and I think that the person that slowed audio 0.999x to make ntsc bluray probably did it without changing pitch.

In the comparison of resamplers at src.infinitewave.ca i noticed that SoX resampler uses a higher quality algorithm than SSRC resampler from AviSynth+. SoX resampler is better than SSRC, and SSRC is better than ResampleAudio().

[Luke M]

why not just adjust the audio length with AssumeFPS?

Because I don't know if the quality of the AviSynth+ resampler is as good as that of the SoX resampler. Also, I don't know how to change speed without changing pitch in AviSynth+, but I know how to do it in SoX, and I think that the person that slowed audio 0.999x to make ntsc bluray probably did it without changing pitch.

That seems unlikely, since if they didn't care about audio quality they would have done the video conversion by slowing down 4% (which looks better than duplicating frames). Anyway who cares. The pitch shift (if it's not fixing the pitch shift!) is insignificant.

And you keep talking about high quality resampling but _you're not doing resampling_..."tempo" change is a more complicated process, inherently lower quality than simple resampling.

[codemaster]

"tempo" change is a more complicated process, inherently lower quality than simple resampling.

I wasn't aware of this. Then I'll use the resample filter instead of the tempo filter, and let the pitch change. I used tempo because I assumed tempo automatically invokes the resample filter in order to do what it does. Now I realized that tempo uses a completely different algorithm than resample, and is not just an alias for the resample filter.

I think I did it properly this time:

ffmpeg -i infile.m2ts ^
-map 0:1 ^
-filter:a "asetrate=48048, aresample=48000:resampler=soxr" ^
-codec:a pcm_f32le ^
outfile.wav

Main reason for not doing this in AviSynth+ is that I prefer the SoX resampler over ResampleAudio().

[codemaster]

Now I have two additional questions.

On this ntsc bluray, audio max amplitude is -0.1 dB, and my sped up audio max amplitude is 0.0 dB. I listened to parts of the sped up audio to check if there's clipping, and I don't hear clipping. And ffmpeg didn't display any warning in Windows Terminal about clipping when I sped up and resampled the audio.

In the filter list, would it be necessary (or recommended, or best practice) to normalize to -1 dB before changing speed to 48048 Hz and resampling to 48000 Hz?

Is the SoX resampler designed to automatically guard against clipping?

[Luke M]

If the peak is exactly 0dB then it's a safe assumption that it's clipped. You don't need to normalize, just reduce the volume, e.g.:

-af volume=-3dB

[codemaster]

I think I finally did it properly:

-filter:a "volume=-0.9dB, asetrate=48048, aresample=48000:resampler=soxr"

Volume of this output file is:

[Parsed_volumedetect_0 @ 0000022cbfe9f540] n_samples: 606694586
[Parsed_volumedetect_0 @ 0000022cbfe9f540] mean_volume: -11.6 dB
[Parsed_volumedetect_0 @ 0000022cbfe9f540] max_volume: -0.3 dB
[Parsed_volumedetect_0 @ 0000022cbfe9f540] histogram_0db: 65212
[Parsed_volumedetect_0 @ 0000022cbfe9f540] histogram_1db: 2704238

Before, I didn't decrease volume of input file:

-filter:a "asetrate=48048, aresample=48000:resampler=soxr"

Which caused the volume of that output file to be:

[Parsed_volumedetect_0 @ 000002170b80f4c0] n_samples: 606694586
[Parsed_volumedetect_0 @ 000002170b80f4c0] mean_volume: -10.7 dB
[Parsed_volumedetect_0 @ 000002170b80f4c0] max_volume: 0.0 dB
[Parsed_volumedetect_0 @ 000002170b80f4c0] histogram_0db: 2418076

So this time I decreased volume of bluray audio from -0.1 dB to -1 dB. Then the speed change and resample filters caused it to increase from -1 dB to -0.3 dB. Which means it required a headroom of minimum 0.7 dB. It was incorrect to give it 0.1 dB headroom, like I did initially.