Audio Delay Issues

[hss2293]

I am converting audio and video separately and then muxing it using mkvmerge. The audio is out of sync. I am using libfdk_aac to encode audio into aac.
Here is the mediainfo of the converted file and the audio delay is -83ms. Please tell me how can i fix this issue.

Format                                   : Matroska
Format version                           : Version 4 / Version 2
File size                                : 303 MiB
Duration                                 : 1h 46mn
Overall bit rate                         : 399 Kbps
Encoded date                             : UTC 2014-05-04 05:24:48
Writing application                      : mkvmerge v6.9.1 ('Blue Panther') 64bit built on Apr 18 2014 21:09:51
Writing library                          : Lavf55.33.101

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 16 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1h 46mn
Nominal bit rate                         : 330 Kbps
Width                                    : 720 pixels
Height                                   : 304 pixels
Display aspect ratio                     : 1.85:1
Original display aspect ratio            : 1.85:1
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.063
Writing library                          : x264 core 142 r2 956c8d8
Encoding settings                        : cabac=1 / ref=16 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=330 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No

Audio #1
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : A_AAC
Duration                                 : 1h 46mn
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Delay relative to video                  : -83ms
Default                                  : Yes
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : HE-AAC / LC
Codec ID                                 : A_AAC
Duration                                 : 1h 46mn
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 KHz / 22.05 KHz
Compression mode                         : Lossy

[ndjamena]

MediaInfo delays are unreliable, it considers NULL frames at the beginning of a stream to be a delay, yet if you re-encode using MPEG2SOURCE or similar those NULL frames will be converted to actual frames. Did you tell MKVMerge to add a delay? What was the last source container for each stream? What did you use to encode them? Was there a delay in the source?

[hss2293]

MediaInfo delays are unreliable, it considers NULL frames at the beginning of a stream to be a delay, yet if you re-encode using MPEG2SOURCE or similar those NULL frames will be converted to actual frames. Did you tell MKVMerge to add a delay? What was the last source container for each stream? What did you use to encode them? Was there a delay in the source?

No i didn't added any delays. I used ffmpeg to encode audio to aac and video to mkv. There was no delay in the source.

[ndjamena]

A Negative Audio Delay means the audio begins BEFORE the video, the audio is ahead of the video no?

The stupid answer is to remux the thing with MMG and add a 83ms container delay to the audio, but that doesn't answer the question of why it's happening in the first place.

Was the original source an MKV? MediaInfo seems to handle delays differently for each container type.

-Edit- Apparently MKVMerge adds a 9ms delay if it takes AAC from an m4a/mp4, yet doesn't add a delay if the same stream is taken from an AAC elemental stream. These things are complicated.

[El Heggunte]

The audio is out of sync.

Do you actually hear it, OR you're just considering what MediaInfo says,

Anyway, you might:

--- use eac3to for removing part of the leading audio samples, OR

--- use qaac with the --no-delay option, OR

--- use the enc_aacplus DLL (from the old Winamp)

[hss2293]

The audio is out of sync.

Do you actually hear it, OR you're just considering what MediaInfo says,

Anyway, you might:

--- use eac3to for removing part of the leading audio samples, OR

--- use qaac with the --no-delay option, OR

--- use the enc_aacplus DLL (from the old Winamp)

I have watched the video and it is out of sync. The audio is coming before the words are actually spoken in video.

[hss2293]

A Negative Audio Delay means the audio begins BEFORE the video, the audio is ahead of the video no?

The stupid answer is to remux the thing with MMG and add a 83ms container delay to the audio, but that doesn't answer the question of why it's happening in the first place.

Was the original source an MKV? MediaInfo seems to handle delays differently for each container type.

-Edit- Apparently MKVMerge adds a 9ms delay if it takes AAC from an m4a/mp4, yet doesn't add a delay if the same stream is taken from an AAC elemental stream. These things are complicated.

ya the original source was also mkv. As you said mediainfo is not always providing correct audio delays then whats the use of remuxing it by adding an delay of exact 83ms. Is there any way to control it during the encoding process.

[ndjamena]

Remux it with MMG then, take note of the 'delay (in ms)' setting in the audio's 'format specific options' tab. If there's an actual number there then add 83ms to it. If not then there's NULL frames somewhere but add 83ms anyway. Container delays aren't optimal but it's better than nothing.

What containers did you use for the individual streams?

[ndjamena]

*sigh* I just discovered MKVMerge doesn't output delay information, therefore MMG doesn't fill it in the 'delay (in ms)' either. Actually, every time you enter a delay, it adds it to the last one rather than replaces it... So you'll need to give it a positive 83ms delay.

It's possible 3 Null frames were added to your video stream for some reason, or the delay may be a container delay that has been inherited from the re-encoded container.

[El Heggunte]

1) Ooops, I hadn't noticed the audio delay was negative

2) you might:

a) either demux the AAC stream with MKVextract, and only then remux with MKVmerge,

b) or decode the audio from the original MKV source, re-encode with qaac, then add the result to the re-encoded video.

[ndjamena]

'Delay relative to video' may be a misnomer in this case. The MKV container does not allow negative delays in it's streams, now that I think about it, maybe no container does. Anyway 'relative' is the word. If the audio begins at -83 and MKV doesn't allow negative, then that means the audio actually starts a 0 and it's the other two streams with a delay (ie 83ms). 3 null frames would explain the video delay, but I'm not sure about the other audio. Which would be why I keep asking about the containers the OP stored the re-encoded streams in. The delay could have been initiated in any one of them.

[ndjamena]

Wow, who'd have thought such a simple thread would have so much influence:

* mkvmerge: new feature: When identifying a Matroska file in
verbose identification mode track-specific tags will be output as
well. The format is »tag_<tag name in lower case>:<tag value>»,
e.g. for a tag named »BPS« with the value »224000« the output
would be »tag_bps:224000«. Enhancement for #1021.

* mkvmerge: new feature: mkvmerge will write track-specific tags
with statistics (»BPS« for the average number of bits per second,
»DURATION« for the duration, »NUMBER_OF_BYTES« and
»NUMBER_OF_FRAMES« for the track's size in bytes and its number of
frames/packets). Implements #1021.

No 'delay' tag as of yet, but I don't think MKVMerge took note of the first time-code, so the statistic won't be available without a small re-write.

It's called progress, fellow forum members...

https://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-amd64-6.9.1-build201...9215-setup.exe

[Selur]

mkv has time codes for each stream, comparing the time codes of audio&video should allow to show delays.
'delay'-tag sounds like a really bad idea as soon as I think about folks keeping their tags while reencoding streams. (!reencoding audio changes the delay!)

[ndjamena]

mkv has time codes for each stream, comparing the time codes of audio&video should allow to show delays.

Apparently Mosu doesn't know that, maybe you should explain it to him.
moritz@bunkus.org

'delay'-tag sounds like a really bad idea as soon as I think about folks keeping their tags while reencoding streams. (!reencoding audio changes the delay!)

If you're just going to blindly copy tags, then I agree, it's a bad idea, in fact blind copying makes all of these tags a bad idea. I don't like the idea of this stuff being stored in the tags section but it looks like we have no choice. If I extract my Episode/Movie tags then I get a copy of the file tags along with it and I'm pretty sure if I ran my Episode/Movie Tag creation Powershell script on one of these new files all this new file information will be wiped.

But this is new, there's bound to be quite a few hiccups until it matures. That's the nature of new.

[ndjamena]

I'm going to comment on this first:

> I'm not sure how easy it is to determine a delay. I've read that all
> you have to do is find the first time-code for the track, but that's
> just gossip.

In general this is correct. Most container formats do not store the delay in the track headers. Instead mkvmerge would need to determine the first timecode of the first valid packet for each track. This could be very time-consuming, e.g. if there's a subtitle track with only a couple of entries an hour or so in (think of subs for Elvish in Lord of the Rings) then the whole file would have to be read up to that point at least.

Other formats are even worse. Delay in AVI audio tracks is achieved by prepending blocks/frames with pure garbage. Therefore in order to determine the first valid audio packet's timecode mkvmerge's AVI reader would have to actually try to decode the audio content as well, not just read a lot of blocks.

These points either add a high cost to detection or require significant re-writes of certain readers (or both).

> Would it be possible to display the delay in MKVMerge -I output?

Technically possible? Sure, everything's possible, like I've said above. Easy to implement, though? Not by a long shot.

> Would it be possible to add a prefix to the delay to specify that it
> is to start at absolute zero?

For similar reasons as to the ones laid out above this would require pretty much the same work to be done.

So for both cases: I won't do it, sorry.

Copied and pasted with no editing.

-Edit- Hang on, maybe he wasn't talking about Matroska containers at all. I forgot to factor in that MKVMerge reads from other containers as well.

-Edit2- It's hard to tell.

[ndjamena]

That's interesting, I'm looking at the cues in an MKV using the MKVInfo GUI and I can see all the Subtitle Cues. I'm assuming the Audio cues are missing because it starts at the same time as the Video and doesn't have an equivalent of GOPs.