Accurate syncing of demuxed tracks

[ZetaStax]

I know this may sound stupid, but I am wondering if there was an accurate way to synchronize audio/video when ripping material. My current workflow is to use DGIndex (or DGIndexNV rather) to index video and demux audio. I then encode the video and the audio separately and mux the result, which often requires me to check they're in sync by painstalkingly opening the source and the encoded version, manually sync'ing both audios and using the audio delay feature from MPC to infer the correct delay. It's a chore, and it's not accurate enough to my liking. (This also applies to synchronizing the subtitles)

More precisely the video is encoded by feeding ffmpeg with a script containing DGSource("project.dgi") while the audio track is fed directly to ffmpeg to convert in wav and then to aac using nero.

What has me puzzled is that a software like Handbrake can do all that while retaining the synchronicity (is that a word?), so there must be a way to retrieve the info. Is there?

PS: Not entirely sure this is the approriate section for this post, sorry...

[gonca]

What version of DGIndexNV do you have?
Have you tried remuxing the elementary streams to say MKV before encoding and then checking
Basically narrow down where the loss of sync happens

[ZetaStax]

DGIndexNV 2053, the latest.
Unfortunately remuxing elementary streams isn't always possible, for instance when I only cut part of the source with DGIndex. Even when I can, sometimes the audio tracks only start after a couple of milliseconds after the video track, so there must be something within the IFO/VOB/M2TS whatever, that knows when the track should start.

However there is an additional issue. When converting say an AC3 track to WAV, the track length is the same, but when converting that WAV to AAC with nero, the length is rarely the same, which can lead to additional delay.

[jagabo]

Unfortunately remuxing elementary streams isn't always possible, for instance when I only cut part of the source with DGIndex.

DGIndex demuxes the cut audio and includes the audio delay in the file name. It's not always right in my experience but it's easy enough to mux it with the new video.

[ZetaStax]

Like you said, this information is not always right. It's usually a good place to start, but it can be off by a few hundreds ms and since it's not 100% reliable I still have to manually check. But at times the desync can happen even if there is no seeking involved, and the delay indicated by DGIndex is 0ms. However Handbrakes still manages to know what the delay should be, so how does that happen?

Sadly I cannot use Handbrake to say, encode a video with ultrafast setting just to find out what the "delay relative to video" value for the track is, because I've found out that when I specify a delay within MKVMerge, the result is correct but the value reported by MediaInfo is not the one that I entered, so I cannot rely on that either.

[poisondeathray]

You can check by adding audio in the script (AudioDub) . Use DelayAudio() with the value written into the audio name , preview the script. Adjust if needed. Encode audio from the script.

Nero adds about 20ms extra compared to other AAC encoders . You can cut the padding by using QAAC instead of Nero . (All AAC encoders add about 40-60ms)

[ZetaStax]

You can check by adding audio in the script (AudioDub) . Use DelayAudio() with the value written into the audio name , preview the script. Adjust if needed. Encode audio from the script.

That's gonna be a problem. If I open the audio track through AviSynth, I have no other option but to use ffmpeg's aac codec, I don't see how I can feed that to nero without demuxing the resulting mkv and loosing the synchronicity (assuming it's maintained through the script).

[poisondeathray]

You can check by adding audio in the script (AudioDub) . Use DelayAudio() with the value written into the audio name , preview the script. Adjust if needed. Encode audio from the script.

That's gonna be a problem. If I open the audio track through AviSynth, I have no other option but to use ffmpeg's aac codec, I don't see how I can feed that to nero without demuxing the resulting mkv and loosing the synchronicity (assuming it's maintained through the script).

I don't see how that's a problem .

People use to use neroaacenc all the time with avs scripts. (Before it became out of favor, the popular one seems to be QAAC, quality seems better according to tests if you can believe them). You just pipe to NeroAACEnc

People avoid this when they re-use the original audio (no quality loss) . But you' re re-encoding anyways so it doesn't matter

[poisondeathray]

Works ok

This example was a 2 ch , 44.1Khz input . Upsample to 48Khz, pipe to neroaacenc, 128kbps 1 pass encode

ffmpeg -i input.avs -vn -acodec pcm_s32le -ac 2 -ar 48000 -f wav - | "neroAacEnc.exe" -if - -br 128000 -ignorelength -of "outputaudio.mp4"

If you wanted to do everything with ffmpeg, taking that nero output as input (streamcopy) for the video encode & mux (some people don't like using ffmpeg to mux, some don't like it for encode; you can replace with whatever)

ffmpeg -i input.avs -vn -acodec pcm_s32le -ac 2 -ar 48000 -f wav - | "neroAacEnc.exe" -if - -br 128000 -ignorelength -of "outputaudio.mp4"

ffmpeg -i 1.avs -i outputaudio.mp4 -map 0:0 -map 1:0 -c:v libx264 -crf 18 -c:a copy "final.mp4"

[ZetaStax]

I don't see how this would be any different from what I'm already doing. The only differences is 1) I'm muxing audio and video with MKVMerge and 2) I'm feeding ffmpeg the ac3 directly instead of through an AviSynth script. But because both are not encoded at the same time using AudioDub has no effect.

I still tried it to be sure and yeah, the result is exactly the same: I still cannot find a reliable and accurate way to infer the necessary audio delay.

[poisondeathray]

The delay is in the script, that's the difference.

Did you check the avs in a media player and adjust the delay? It should be hardcoded into the audio .

For fun, encode that avs script with ffmpeg aac , together with video and audio to test it. Just encode very small dimensions in the script (resize to 640x360 or smaller) , -preset:v ultrafast . Is that in sync ?


Are you sure it's a constant delay ? Or is there progressive sync issues ?

[ZetaStax]

This is what I tried:

V=DGSource("VTS_01_0.dgi")
A=NicAC3Source("VTS_01_0 T81 2_0ch 48KHz 448Kbps DELAY 0ms.ac3")
AudioDub(V,A)

Then encoded using this...

ffmpeg -i script.avs -vn -c:a pcm_s32le -ac 2 -ar 48000 -f wav - | neroAacEnc -if - -q 0.5 -ignorelength -of audio.mp4
ffmpeg -i script.avs -i audio.mp4 -map 0:0 -map 1:0 -c:v libx264 -crf 18 -preset ultrafast -c:a copy "final.mp4"

Result: not sync'd. Also tried...

ffmpeg -i script.avs -c:v libx264 -crf 18 -preset ultrafast -c:a aac "final.mp4"

Result: not sync'd.

Or did I misunderstand what you were saying perhaps? The required delay is always the same (-400 ms in this example). And I don't see how it can be any different, I don't think the audio tracks demuxed by DGIndexNV have any knowledge of what kind of delay should be applied relative to the video.

[poisondeathray]

Yes you misunderstood . The purpose of the script is to preview the AV sync +/- using it to encode. So you don't have to waste time encoding a dozen times. Preview the .avs script in a media player like mpchc

If it's already out of sync..., of course any encode from that script is going to be out of sync

V=DGSource("VTS_01_0.dgi")
A=NicAC3Source("VTS_01_0 T81 2_0ch 48KHz 448Kbps DELAY 0ms.ac3")
AudioDub(V,A)
DelayAudio(SOME VALUE HERE)

[poisondeathray]

My opinion - I try to have zero delay in the actual streams. I either cut the audio or video to match, or pad. This way it's in sync even if you demux it. If you had a (-) audio delay relative to video, this means audio comes early. I would cut the audio if it's not important, or pad the video. If you had a (+) audio delay , I would add silence to the beginning of the audio to match. Unless the beginning of the video is useless, then I would cut both. I like to check the ends of the streams too. I try to make them flush , same length. Because some applications might mishandle it otherwise at the end.

[ZetaStax]

Oh well, yes it is out of sync. What I'm looking for is a way to determine that delay without any sort of guess-work, and I'm not wasting time reencoding. I encode the video and audio separately and fiddle with the delay only when muxing with MKVMerge. The time loss is only from checking sync and determining the delay if necessary.

[poisondeathray]

Oh well, yes it is out of sync. What I'm looking for is a way to determine that delay without any sort of guess-work, and I'm not wasting time reencoding. I encode the video and audio separately and fiddle with the delay only when muxing with MKVMerge. The time loss is only from checking sync and determining the delay if necessary.

You waste time muxing. Faster checking in a script

[poisondeathray]

The other option is to use another source filter on the container . Since DG* demuxes, and can sometimes be wrong with the delay value.

Personally , I prefer to have zero delay, not mismatched AV lengths or instead of relying on container +/- delay. Even if it takes a few minutes extra.

[ZetaStax]

You waste time muxing. Faster checking in a script

Not really, because I can't simply watch the video. Delay may be acceptable for dialogue but off for short sound effects, I feel compelled to be more accurate than that, which is why I playback both the encode and the source to have them as close to a perfect sync. I also have a script that does the muxing, it's just a matter of recalling the previous command and changing the delay value.

The other option is to use another source filter on the container . Since DG* demuxes, and can sometimes be wrong with the delay value.

I guess, but as far as I remember I originally started using DG's because I ran into some problems when trying to open the videos directly through AviSynth, especially when seeking was required. I'll see if that keeps the original delay in check.

[ZetaStax]

Personally , I prefer to have zero delay, not mismatched AV lengths or instead of relying on container +/- delay. Even if it takes a few minutes extra.

Interesting point here, MKVMerge actually cuts the track when using negative delay if it results in negative timestamps (which I guess is the reason why the delay reported by MediaInfo is not the same you enter in MKVMerge)