H264 2pass messed up private data for joining files

[FireEmerald]

I've got several 2-part (two separate files) movies which I encoded as MP4, MPEG-4 AVC / H.264 using 2-Pass ABR (6.000 bitrate / 12.000 vbv max bitrate) from it's sources.

Now i wanted to join all 2-part movies into a single file using MKVToolNix - but unfortunately only about 20% could be processed, the rest resulted in a warning:

• The codec's private data does not match (lengths: 40 and 40).

These 80% play the first part correct and then show a blue/red/green screen where the the second part should be. The lengths are always the same - but it seems like the 2pass encoding messed up a tiny part, but which?

There's also a bug report on the x264-devel mailing list.

My question is now:

• Is 2pass encoding in general a really bad idea if you plan to join two files?
• What was going wrong while encoding? What messed up the encoder?
• What exactly does "length: 40" mean? Is there a free tool to inspect this parameter?
• Is there anything i could do with these already encoded files to get them together? Re-encode the second track one more time lossless? Which tool would you recommend here?

[sneaker]

2pass isn't bad but you need to use --stitchable parameter. Then private codec data will match. x264 does header optimizations if you don't do that.

For already encoded flles try to use ffmpeg concat demuxer to append instead of mkvtoolnix.

[FireEmerald]

Which GUI based tool offers the possibility to add the --stitchable parameter?

[sneaker]

Many do allow custom x264 parameters. Examples: VidCoder, HandBrake, StaxRip, Hybrid, Megui, ...
What have you been using until now? Maybe it's possible with that software as well.


Personally, I prefer to append in AviSynth or by other means before/as I encode. Then I can be sure the resulting file is monolithic.

[FireEmerald]

I used XMedia Recode until now - but i don't think that there's such an option...

[sneaker]

Yes, it seems Xmedia is one of those who do not offer the option.

[Atak_Snajpera]

I've got several 2-part (two separate files) movies which I encoded as MP4, MPEG-4 AVC / H.264 using 2-Pass ABR (6.000 bitrate / 12.000 vbv max bitrate) from it's sources.

Now i wanted to join all 2-part movies into a single file using MKVToolNix - but unfortunately only about 20% could be processed, the rest resulted in a warning:

• The codec's private data does not match (lengths: 40 and 40).

These 80% play the first part correct and then show a blue/red/green screen where the the second part should be. The lengths are always the same - but it seems like the 2pass encoding messed up a tiny part, but which?

There's also a bug report on the x264-devel mailing list.

My question is now:

• Is 2pass encoding in general a really bad idea if you plan to join two files?
• What was going wrong while encoding? What messed up the encoder?
• What exactly does "length: 40" mean? Is there a free tool to inspect this parameter?
• Is there anything i could do with these already encoded files to get them together? Re-encode the second track one more time lossless? Which tool would you recommend here?

Demux video streams to raw format

Part1.264
Part2.264

Combine them using
copy /b Part1.264+Part2.264 Combined.264

Mux back to .mkv using mkvtoolnix.

[sneaker]

I recommend the ffmpeg concat demuxer over that method. It will copy SPS/PPS to all GOPs, mkvmerge with copy /b method only to second part. You will have problems in some players (e.g. VLC) if you seek back into first part. Of course best is to encode properly in the first place.


P.S.: mkvmerge has function equal to copy /b inbuilt: https://github.com/mbunkus/mkvtoolnix/wiki/Appending-files-vs.-adding-as-additional-parts
But like I said I don't recommend it for this case. It's useful for e.g. DVD VOB.

[FireEmerald]

Demux video streams to raw format

I'm not that familiar with the encoding processes/usage of ffmpeg - how do i demux them to raw format? (Part1.mp4 and Part1.mp4)

SPS/PPS to all GOPs

What's SPS, PPS and GOP's?

[sneaker]

I'm not that familiar with the encoding processes/usage of ffmpeg - how do i demux them to raw format? (Part1.mp4 and Part1.mp4)

You don't need to do a multi-step process with demuxing. The function is just called "concat demuxer" for some reason.
https://trac.ffmpeg.org/wiki/Concatenate#demuxer

What's SPS, PPS and GOP's?

GOP is Group Of Pictures. Such a group is formed from all frames from one keyframe to the next one. (simplified explanation)
SPS and PPS are header data in the H.264/AVC video standard needed by the decoder for correct decoding. The problem with appending is that if you seek these header datas might mismatch and the output will be garbled/grey/green/whatever. It's not really important to fully understand. Just stick to the guidelines I gave you.

[FireEmerald]

I already had this link open @sneaker - the result looks great! The only thing what i'm still searching for is: How can i tell ffmpeg to copy the existing chapters to the output file?

ffmpeg.exe -f concat -safe 0 -i concat.txt -c copy output.mp4

___________________________________
Edit: Seems like ffmpeg doesn't maintain chapters when concatenating two MP4 files using the concat demuxer...

[sneaker]

Not sure what the most straight-forward way to solve this would be. Maybe two step process:
ffmpeg -f concat -i concat.txt -map 0:v:0 -c copy "temp.mkv"
mkvmerge -o "output.mkv" "temp.mkv" --no-video "part_1.mp4" --no-video +"part_2.mp4"

concat.txt:

file 'part_1.mp4'
file 'part_2.mp4'

That should work but I'm not sure sync after the splicing point is guaranteed in all cases.