0,5 frame audio shift after mpeg2 to mp4 conversion

[Bencuri]

This is something I noticed earlier, too. I have just converted a DVD source to Full HD mp4, 25i to 50p. The outcome is that after muxing, the audio of the mp4 starts about 0,5 frame earlier than the DVD. When I watch the video, the audio does seem to be a little off, but that is so little difference, that I guess I am influenced by the knowledge about the difference.


I have seen this earlier as well that when rendering to mp4, the audio is shifted a little. Why is this and should it be corrected or left as is.

I used DGMPGDEC for demuxing Mkvtoolnix for muxing.

[johns0]

If it doesn't bother you don't fix it.

[Selur]

If you convert the audio, additional delay caused by the encoder might need to be taken into account.
Also: What are you using to see the 0.5 frame delay?

[Bencuri]

If you convert the audio, additional delay caused by the encoder might need to be taken into account.
Also: What are you using to see the 0.5 frame delay?

Sony Vegas

[Selur]

Are you sure Sony Vegas properly shows the delay in mp4 with aac? (Haven't tried Vegas, but I know that a bunch of tools are not that reliable when it comes to mp4 and aac, especially he-aac and he-aacv2)

[jagabo]

Audio encoders like aac, ac3, and mp3 typically add 10 to 20 ms of silence at the start of the encoding. That would coincide with your "half a frame" of delay. As you indicated that's not usually noticeable when watching the video. But if it bothers you, remove the corresponding amount of audio at the start before before encoding. It's probably silence anyway.

[Bencuri]

I have noticed this shift when converting to mp4 in other cases as well, independent of the softwares used. Why I raised this question is because I wonder if there is a specific reason it is added: for example when players open the file, they realign it correctly, and the delay has actually a purpose to be there.

In this specific case, this is a music video, where not every scene is in proper sync with the audio, so it would be very difficult to align it "manually". But for 100% correctness, I would like to make it as authentic to the source as possible. So if you say that delay is just by a small incorrectness of the encoders, I would correct them. In this specific case I cannot rely on my own senses to remain faithful to the source.

[jagabo]

The reason for adding a short silence to the start of the compression has to do with the way audio compression works. Here's an example:


[Attachment 78775 - Click to enlarge]

On the top is the start of a WAV file (uncompressed PCM). Below is the same audio converted to aac. You can see the aac encoder added about 33 ms of silence to the start of the audio. You can't see it here but the aac encoding also added some silence to the end of the audio.

Rather than removing a short bit before compression you can add an audio advance (a negative delay) at the container level when muxing.

[Bencuri]

The reason for adding a short silence to the start of the compression has to do with the way audio compression works. Here's an example:


[Attachment 78775 - Click to enlarge]

On the top is the start of a WAV file (uncompressed PCM). Below is the same audio converted to aac. You can see the aac encoder added about 33 ms of silence to the start of the audio. You can't see it here but the aac encoding also added some silence to the end of the audio.

Rather than removing a short bit before compression you can add an audio advance (a negative delay) at the container level when muxing.

There has to be a different problem here, or multiple problems in my case.

When I started this topic, I tested the final video as mp4. I had the mpeg2 initially, converted it to mkv Full HD with x264.exe. However I have Sony Vegas version 12.0 (I only have Windows 7), and that one cannot open mkv's. So I converted the mkv to mp4 with Xilisoft converter. Then I muxed the final mp4 video and the original wav of the mpeg2 video together. That wav was demuxed by DGmpgdec initially. For muxing I used Mkvtoolnix. In this case Vegas showed that the mp4 starts half a frame earlier (the timeline was 25fps though, the mp4 is 50p, so actually it can be considered 1 frame earlier as well, depending on how you interpret the timeline).

But, I noticed that VSDC free video editor is capable of running on Windows 7 and open mkv at the same time. So, to stay faithful to the end product, I opened the muxed mkv in it (mkv+wav). The outcome was very strange. If I compare the original mpeg2 video to the muxed mkv, it shows that the sound in the mkv starts 2 frames earlier than the sound of the mpeg 2 (timeline is 50fps), but strangely, by the end of the project timeline the mkv will be 40 frame late in relation to the mpeg2. I added the wav to the project as well to examine that, the one that was demuxed with DGmpgdec from the mpeg2, so theoretically it should line up with the sound of the mpeg2 on the timeline. Yet, it lines up with the mkv, precisely. By the end, this wav that was demuxed from the same mpeg2 that is on the timeline as I mentioned, it becomes 40 frames late to the sound of that mpeg2, too.

So something was either messed up with DGmpgdec, or I don't know.

If you want I can upload these files for you to check but they are big, over 1GB.

[Bencuri]

Just as a follow up, in Vegas opening the mpeg2 and the demuxed wav they align. Cannot check the mkv as I mentioned. In Goldwave editor, all 3 align, the demuxed wav, the original mpeg2 and the mkv. All 3 is in sync regarding the audio.

[rgr]

If you convert the audio, additional delay caused by the encoder might need to be taken into account.
Also: What are you using to see the 0.5 frame delay?

Sony Vegas

1. In Vegas, enable "legacy drivers". Options->Preferences->File I/O. And always keep it on.
2. Restart Vegas.
3. Check again if you have this 0.5fps delay.

[Bencuri]

In my version there is still no menu panel like that in the Preferences.

I re-checked my whole workflow and the previous files. I made a mistake that the mp4 that showed the delay was not an mp4 video+wav muxed, I wrote that incorrectly, but an mp4+aac. I remembered wrongly. Yet it showed the delay in Vegas, very likely it is because of the reasons that Jagabo referred to.

VSDC editor is a complete mess when opening these files, so to keep this whole topic simple I will ignore the results from that.

As I cannot open the mkv in Vegas, I downloaded Wondershare Filmora. That could open the mkv. That mkv is an mkv+the original DVD video wav muxed. I compared it to the original mpg, they do align. No shift in sound. They play in sync together. I also opened the mp4 that was converted from the mkv, the same result like in Vegas, there is a delay compared to the original.

So the mp4 audio will have the shift. But now the question is how Youtube will behave? I intended to upload the mkv to Youtube (mkv+wav muxed) for maximum audio quality, as Youtube recommends to upload in the best possible quality. But I don't know what Youtube will convert this into? I tested that I added an extra amount of milliseconds to the beginning of the wav before muxing the mkv. Then I converted again that mkv to mp4. In that case the delay in the resulting mp4 was smaller compared to the original. So I could just repeat it for every video I want to upload to Youtube, but will the conversion on Youtube behave the same? Will Youtube also convert the audio to aac? Because if not, it is totally useless to try to solve the problem of that delay, if we don't know how Youtube does the conversion, and which direction it will shift the audio if it will at all.

[rgr]

In my version there is still no menu panel like that in the Preferences.

I re-checked my whole workflow and the previous files. I made a mistake that the mp4 that showed the delay was not an mp4 video+wav muxed, I wrote that incorrectly, but an mp4+aac. I remembered wrongly. Yet it showed the delay in Vegas, very likely it is because of the reasons that Jagabo referred to.

Vegas decoders shift the sound by approximately 3 frames. The bug has been around for years. If you change to "legacy drivers" the problem disappears.

https://www.vegascreativesoftware.info/us/forum/why-is-hevc-legacy-enabled-by-default--137154/ (last three posts)

[Bencuri]

In Vegas 12 that setting is still not not there. That whole panel is missing.