"The codec's private data does not match"

[Elegant]

Hi, I'm editing a GOP in one of my videos so that I can concatenate it with another video. To better define it I have videos A and B. I am removing part of A and encoding (from the source, not a re-encode!) the end of the current scene since the GOP I want to cut on does not terminate at the end of the scene. As such we have this new video C that tacks on to the end of A (after being trimmed) and then B is concatenated as well:

A (trim'd) + C + B

Video C is encoded using the same x264 settings (encode settings, level, etc.) as well as the same x264 version as both videos A and B. When I append them in mkvmerge I get:

Warning: The track number 0 from the file 'F:\Output\K01-Ending.mkv' can probably not be appended correctly to the track number 0 from the file 'F:\Output\K01-Main.mkv': The codec's private data does not match (lengths: 45 and 43). Please make sure that the resulting file plays correctly the whole time. The author of this program will probably not give support for playback issues with the resulting file.
Warning: The track number 0 from the file 'F:\Output\K01-Preview.mkv' can probably not be appended correctly to the track number 0 from the file 'F:\Output\K01-Ending.mkv': The codec's private data does not match (lengths: 43 and 45). Please make sure that the resulting file plays correctly the whole time. The author of this program will probably not give support for playback issues with the resulting file.

K01-Ending is video C which I just encoded from the source; what can I do at this point (given that every setting matches) to correct this? I know if I just append them as h264 I get no warning but I feel that something still isn't right.

[poisondeathray]

Using --stitchable on the commandline will increase your chances... But it doesn't always work

The 100% foolproof way is to cut on GOP boundaries (make sure they are IDR frames, true delimiters, not "i" frames) . And use --sps-id of a different number for the middle segment e.g. --sps-id 2 . Then do a dos binary join on the 3 elementary streams. Then you can re-wrap with audio into a container

[aedipuss]

i'd encode all parts to lossless avi, then edit and re-encode to final.

[Elegant]

Stitchable is not accessible in this version (r2334 current, r2342 it was implemented). I did test with a later version with stitchable and it had some artifacting but the same message remained. I am currently cutting on the GOP boundaries which is why video C currently exists as the the frame I wanted to cut on was mid GOP and I can't do that properly so I cut on the GOP prior to it and then encoded (from source) where the following GOP without the unwanted frames. I also tried specifying a different sps-id in an encode but we still arrive at the same issue. All these issues disappear doing a binary join with my current material but it doesn't seem right to do this unless the codec issue is strictly in the mkv and not the h264 stream itself.

I would rather avoid any further encoding, several filters were applied to various scenes and I do not know which ones they were. Otherwise the simplest solution would be to redo the video.

[poisondeathray]

You won't get that message or any errors if you do the binary join. That' s how distributed encoding is done (e.g. youtube over multiple servers) - it has nothing to do with MKV or any container.

[Elegant]

Results from mkvmerge (using h264 files not mkv) appear to be the same as the binary join (used HxD) for what it does for elementary streams. I guess I'll just roll with this, thanks.

When I double check these new files in an AviSynth editor/preview mode I see some "artifacting" on the initial and final frames of interim clip (video C) but I don't see them during playback (Maybe they'll appear if I try to start playback from that point?). Is this just an issue with FFVideoSource trying to interpret the file or is there something else I'm not considering?

[poisondeathray]

When I double check these new files in an AviSynth editor/preview mode I see some "artifacting" on the initial and final frames of interim clip (video C) but I don't see them during playback (Maybe they'll appear if I try to start playback from that point?). Is this just an issue with FFVideoSource trying to interpret the file or is there something else I'm not considering?

This indicates a problem - Which shouldn't happen

Are you sure you did the procedure correctly? This suggests you didn't cut on IDR boundaries

It only fails if you don't do a binary join, or didn't use GOP boundaries. Or the artifact was there in the first place in your re-encoded C. Or you used open-GOP in your "C" section (heh C-section LOL). The first frame should start with an IDR frame

If you still can't figure it out, upload the section around the original source files (at least 2 GOPS away), your encode and result

[Elegant]

Okay, I've recreated the scenario with clips A, B and C as was initially described. A and B are 2 GOPs away from the interim C clip that joins it all together. They are in mkv format (currently) but when concatenated (see A+C+B) you should see the issue in any AviSynth editor with FFVideoStream. Also, I wouldn't call "C" a re-encode since it is from the source material and not from the already encoded video.

Clip A
Clip B
Clip C

Clip A+C+B

[poisondeathray]

I see the problem in A+C+B, right at the "seams" .

But what is your A+C+B ? Is that your with your Clip C re-encoded with 10bit AVC using similar settings ? with different --sps-id , binary join?

Please say you used 10bit AVC for the C re-encode ?



Anyways it works fine here . Perfect, no glitches

If you need detailed instructions, I'll try to walk you through it

[Elegant]

C is a encoded from the original source using the same x264 version (r2334 10bit) and settings as A and B to the letter (which are also encoded from the original source) but does not include a different sps-id as it literally made no difference in this scenario. This is also a binary join with A+C+B but I cannot seem to get rid of that problem at the seams.

Were you able to combine them without the issue occurring at the seems?

[poisondeathray]

Yes it works fine .





1) I extracted with mkvextract. It gave me (you don't actually have to extract C, because you are re-encoding it)

A_track1_eng.h264
B_track1_eng.h264
C_track1_eng.h264

2) I created FFMS2 script for track C. Make sure the preview is correct (some versions of FFMS2 might be buggy)

FFVideoSource("F:\_Video Tests\sps-id\C.mkv")
AssumeFPS(24000,1001)

3)
I re-encoded C with 10bit x264 using x86 version because I'm too lazy to pipe (direct avs input)

x264k10_32 --level 4.1 --profile high10 --force-cfr --fps 24000/1001 --bitrate 8000 --b-adapt 2 --deblock -1:-1 --sps-id 2 --sar 1:1 -o C2.264 C.avs

I set the --sps-id of 2 here

You can actually use different encoding settings . But there are a few other obscure conditions that can make it fail. --sar is one of them that can cause problems with some decoders when it switches mid stream. It was 1:1 not "undefined" in A and B. It doesn't even matter if the same x264 version or core was used. Or different encoder even.

You can use h264_parse to list some other settings if you don't recall what you used or don' t know in the other files. But mostly it doesn't matter what settings you use

The other situation is it may break VBV constraints - e.g. some situations where VBV is important like BD . If your stitched segment has much higher bitrate or some high peaks, theoretically it might cause buffer underruns . This won't affect FFVideoSource or computer playback

When I said 100% success - I meant 100%. There is no reported failures in 8-9 years since this method has been around. It's just "clunky" and cumbersome to do. The most common cause of failure is not cutting on GOP boundaries correctly

4) I renamed "C2.264" to "C2.h264" Probably not necessary, but just to match whatever mkvextract extracted

5) Next I joined in a specific order. The newly appended file takes on the --sps-id of the first file. I don't know if this is necessary either, but this is the way I do it and it never fails. My thinking is if I append A+C, that will have sps-id of zero. If I join (A+C)+B, zero and zero will have problems

So instead, I join C+B (so (C+B) now has sps-id of 2, then A+(C+B) . zero and two are ok, because they are different

copy /b c2.h264 + B_track1_eng.h264 cb.h264

copy /b A_track1_eng.h264 + cb.h264 new.h264

so "new.h264" is the final file. You can use mkvmerge or whatever muxer

I might have missed some steps, but I posted this before in another thread, there might be more detail there

[Elegant]

I'm trying to get on the same page as you so I'll outline my steps (these steps are not solving the issue currently):

1. Extract h264 streams from A and B track.

2. I create a script for track C as follows:

DGSource("E:\Output\00000.dgi",fieldop=0)
AssumeFPS(24000,10001)
Trim(36401, 36635)

3. I encode C using the following:

avs4x264mod --x264-binary "x264-10b_64" --seek-mode "safe" --colormatrix bt709 --level 4.1 --profile high10 --force-cfr --fps 24000/1001 --bitrate 8000 --b-adapt 2 --deblock -1:-1 --sps-id 2 --sar 1:1 -o "F:\Output\K01-Ending.h264" "E:\Output\K01-Ending.avs"

4. I join C with B and then CB with A.

copy /b C.h264 + B.h264 CB.h264
copy /b A.h264 + CB.h264 Final.h264

5. View in editor to see my problem still exists.

[poisondeathray]

I'm trying to get on the same page as you so I'll outline my steps (these steps are not solving the issue currently):

I can't reproduce your problem (when using the proper steps)

I used komisar's x264 build, but I doubt that will make a difference .
http://komisar.gin.by/

I'll upload my "C" re-encode (which is "c2.h264") and try to repeat the steps with appending on the same sample files (step #5 in my post)

Your samples should be long enough to reproduce the problem, so I don't know what is going on

[poisondeathray]

How are you checking afterwards ? Exactly what steps?

And what is the exact problem on the longer files ? Is it the same artifact on your first A+C+B (ie. the glitch at the joins) ?

I used mkvmerge and FFVideoSource , no problems

[Elegant]

I used komisar's x264 build, but I doubt that will make a difference .
http://komisar.gin.by/

We're gonna file that under "Yes, yes it does". The question is why. What does komisar's build have that's different.

[poisondeathray]

I used komisar's x264 build, but I doubt that will make a difference .
http://komisar.gin.by/

We're gonna file that under "Yes, yes it does". The question is why.

So it works for you now ?

What build did you use that "failed" ?

[Elegant]

x264 core 133 r2334 a3ac64b

10 bit version; this is what A and B were encoded with originally.

[poisondeathray]

But is it a "vanilla" version or what patches were applied? Did you build it yourself , or where did you get it ?

[Elegant]

That's a good question... perhaps I should try and hunt it down again. Mind you most copies online are currently 8bit only.

[poisondeathray]

It actually shouldn't matter which version you use, as long as the output bitstream conforms to AVC standard. There is a lot of "wiggle room" here. If it didn't work, it suggests that the output of that binary is buggy

[Elegant]

Yeah, I'm a stickler though for making sure this stuff matches up. I'll browse a bit for another x264 10b r2334 unless someone still has it lying around. Thanks for the help though!

[poisondeathray]

32bit
http://x264.nl/x264/?dir=./32bit/10bit_depth/revision2334
64bit
http://x264.nl/x264/?dir=./64bit/10bit_depth/revision2334

These are "vanilla" builds from the old x264.nl host

[poisondeathray]

And it works fine, at least the 32bit one does

[Elegant]

Yep, I just finished encoding as I refreshed this page XD All good here! Can't believe that was the issue. Thanks!

One last question, you mentioned that A and B had SAR 1:1 but C had it as unknown. I could not find that using MediaInfo what's the best way to determine that info?

[poisondeathray]

One last question, you mentioned that A and B had SAR 1:1 but C had it as unknown. I could not find that using MediaInfo what's the best way to determine that info?

The free way is to use h264_parse ; I mentioned it above. or You can use various stream analyzers, but they are pretty expensive

e.g for "A", this will output to a text file

"h264_parse" "A_track1_eng.h264" 1>A.txt

The portion that indicates sar is this:

   vui_parameters_present_flag: 1
    aspect_ratio_info_present_flag: 1
     aspect_ratio_idc:1

If you run it on "C" it looks like this which is "undefined"

   vui_parameters_present_flag: 1
    aspect_ratio_info_present_flag: 0