Audio out of sync when joining many clips together

[umbala]

Hi. I have thousands of short video clips (2s to 30s long) that I'm trying to join and the audio always goes out of sync. I'm usually joining 200-300 clips into a single video and the further along in the video the more out of sync the audio becomes.

First I thought it was because not all of the clips have the same audio sample rate. I tried joining only videos that have the same sample rate, but the final video still has sync issues. Then I thought it was because not all the clips use the same frame rate, some are 24fps, 25fps, 30fps, 60fps. So I tried joining only videos that have the same frame rate and same audio sample rate, but it still goes out of sync.

When all the frame rates and sample rates match the sync issue is far less noticeable, but it's still there. Most of the videos were cut using SolveigMM Video Splitter and also to join the resulting clips back together. None of the clips have audio sync issues when I check them individually, only when they're joined together. I tried a few other tools for cutting and merging clips, but the audio sync issue is always there.

Are there any tools out there that can merge the clips and also fix the audio sync issue? Or a tool that can fix individual clips so they won't have sync issues when I join them together? I would prefer not to re-encode the video or audio, but if that's the only way to fix the issue then I would be willing to do that too. Any suggestions would be greatly appreciated.

[davexnet]

See AlignedSplice
http://avisynth.nl/index.php/Splice

[abolibibelot]

What is the format of the clips ? Once I had a similar issue trying to join 3 MP4 files losslessly, every method resulted in synchronization issues, except one : with ffmpeg, first converting each clip to TS format (Transport Stream), then merging the clips as TS back to MP4.

ffmpeg -i "input 1.mp4" -c copy -bsf h264_mp4toannexb "temp1.ts"
ffmpeg -i "input 2.mp4" -c copy -bsf h264_mp4toannexb "temp2.ts"
ffmpeg -i "input 3.mp4" -c copy -bsf h264_mp4toannexb "temp3.ts"
ffmpeg -i "input 4.mp4" -c copy -bsf h264_mp4toannexb "temp4.ts"
ffmpeg -i "concat:temp1.ts|temp2.ts|temp3.ts|temp4.ts" -c copy -absf aac_adtstoasc "input 1-4 merged.mp4"

I'm not sure which formats can be converted to TS other than MP4 (H.264 / AAC).
Also, I don't know what the "-absf aac_adtstoasc" option does, or if it's still working with recent versions of ffmpeg.

The suggestion above with Avisynth would require transcoding.

Another option would be to join the clips in a non linear editor.

[umbala]

What is the format of the clips ? Once I had a similar issue trying to join 3 MP4 files losslessly, every method resulted in synchronization issues, except one : with ffmpeg, first converting each clip to TS format (Transport Stream), then merging the clips as TS back to MP4.

ffmpeg -i "input 1.mp4" -c copy -bsf h264_mp4toannexb "temp1.ts"
ffmpeg -i "input 2.mp4" -c copy -bsf h264_mp4toannexb "temp2.ts"
ffmpeg -i "input 3.mp4" -c copy -bsf h264_mp4toannexb "temp3.ts"
ffmpeg -i "input 4.mp4" -c copy -bsf h264_mp4toannexb "temp4.ts"
ffmpeg -i "concat:temp1.ts|temp2.ts|temp3.ts|temp4.ts" -c copy -absf aac_adtstoasc "input 1-4 merged.mp4"

I'm not sure which formats can be converted to TS other than MP4 (H.264 / AAC).
Also, I don't know what the "-absf aac_adtstoasc" option does, or if it's still working with recent versions of ffmpeg.

The suggestion above with Avisynth would require transcoding.

Another option would be to join the clips in a non linear editor.

Thanks for the reply. All my clips are 1080p MP4 (H.264) format. Would the ffmpeg method require that all the clips have matching frame rates and audio sample rates? Any idea if a non linear editor would be able to join different frame rate and sample rate clips without having to transcode the video? What would be a good editor for that? Something like Adobe Premiere or is there something better?

[abolibibelot]

Would the ffmpeg method require that all the clips have matching frame rates and audio sample rates?

I would think so. It may work but the output might not play correctly.
Or you could leave it in TS, as a quick and dirty workaround, if it doesn't have to be processed further. Just replace the last line by :

"concat:temp1.ts|temp2.ts|temp3.ts|temp4.ts" -c copy -absf aac_adtstoasc "input 1-4 merged.ts"

(Again, I'm not sure if the "-absf" part is required.)
The particularity of the TS format, as I understand it, is that video and audio streams are organized in small "packets" with a dedicated header, each packet being a self contained unit which can be read individually, so if a packet is missing (typically because of a transmission issue, as it is primarily used for broadcasting), the playback can continue at the next packet, there's only a glitch and the video doesn't stop right there as it would with most other formats, and the video/audio synchronization is not lost (each packet containing a split second of synchronized video/audio streams). And it also allows to join parts with different parameters. For instance I have some TV recordings made in MPEG-TS around 2008 (when the broadcasting standard aspect ratio was still 4:3) which suddenly switch the aspect ratio from 4:3 to 16:9 at the beginning of a documentary. Converting them losslessly to MPEG-PS works (MPEG-PS is also structured in small “packets” with a new header at regular intervals, although I don't know exactly what distinguishes MPEG-PS from MPEG-TS), but if transcoding them to MP4 for instance, only one aspect ratio could be defined for the whole duration of the video.

Any idea if a non linear editor would be able to join different frame rate and sample rate clips without having to transcode the video?

It would have to transcode when exporting. Some editors have a so-called "smart rendering" feature, which is meant to export losslessly segments with no processing except simple cuts, but it probably wouldn't work in this situation.

What would be a good editor for that? Something like Adobe Premiere or is there something better?

I've never Used Adobe Premiere, but it's supposed to be the very best there is, or at least among the most complete editing softwares available. Although I don't know if it's the best suited for that particular purpose. For simple editing tasks, I see Video ReDo regularly recommended, which is much cheaper and supposedly has the “smart rendering” feature, although I haven't tried it either.

[Sharc]

@umbala
Are your clips VFR (variable frame rate) from a mobile phone?
If so, convert them to CFR (constant frame rate) before joining them, e.g. using the utility mp4fpsmod.exe or perhaps by remuxing via mkvtoolnix.

or you could try to add to the concatenation commandline
ffmpeg -i "concat:temp1.ts|temp2.ts|temp3.ts|temp4.ts" -c copy -video_track_timescale 60 -absf aac_adtstoasc "input 1-4 merged.mp4"

[abolibibelot]

@ Sharc
You mean this ? How to make an .exe file with what's provided ? Perhaps there's a pre-compiled version somewhere ?
It might not be relevant to the issue exposed in this thread, but it might be useful to solve an issue I explained there. In short : a MKV video is not read by a standalone BRD player (which normally plays MKV files), most likely because of a "variable framerate" flag, although it shouldn't be variable at all (it's a movie in standard 23.976 FPS), so I need to losslessly process it in such a way that the player can read it.

[Sharc]

Yes.
You should find the mp4fpsmod.exe + xx.dll included in MeGUI in the tools\mp4box\mp4fpsmod folder.

[abolibibelot]

Alright, thanks. It's been a long time since I updated it... just did : it's there indeed.