Splitting into 3, editing 1, can't rejoin bc private codec data don't match

[Travillion]

Hello all,

I am taking a single MKV file (h264 codec). I wish to make a small edit and don't want to re-encode the whole file (and thus incur quality loss). So I split the mkv into three mkv files (1 large, 1 small for the edit, and then the large remainder) with mkvmerge. If I simply rejoin these three files as they are with mkvmerge, there are no problems and it looks seamless. However, once I take the mkv into Adobe Premiere Pro and make my modification, I can no longer rejoin the three files as I get a Codec's Private Data do not match error. There is very limited information I can find about the codec private data, but I get the sense that something about the header of the edited file is modified and this causes problems.

I have tried all the obvious things like making sure frame rate, resolution, etc. are the same. I thought maybe there was something about Adobe's h264 encoder that was messing things up and so I tried saving to a lossless codec (UT video) from premiere and then using ffmpeg to convert back to h264 and mkv.

Is anyone aware of why importing/exporting from premiere would make these files unjoinable? Is there a different approach I could take that would allow me tweak this one clip without importing the whole unsplit file into Premiere and re-encoding it?

[sneaker]

can no longer rejoin the three files as I get a Codec's Private Data do not match error. There is very limited information I can find about the codec private data, but I get the sense that something about the header of the edited file is modified and this causes problems.

Yes, H.264 SPS/PPS are stored in the codec private data. For maximum player compatibility these should match 100% though some things are more problematic than others. There are tools to view them if you want to compare but it is not trivial:
free: ffmpeg trace_headers bitstream filter, h264_parse
expensive (but with free demos): Zond, CodecVisa


What encoder did you use for the original MKV file? Can you use said encoder for Premiere?


You may get away with different SPS/PPS as long as they are in front of every keyframe. Set encoder/demuxer to do that if possible. Try ffmpeg's concat demuxer instead of mkvmerge for appending.

[Travillion]

What encoder did you use for the original MKV file? Can you use said encoder for Premiere?


You may get away with different SPS/PPS as long as they are in front of every keyframe. Set encoder/demuxer to do that if possible. Try ffmpeg's concat demuxer instead of mkvmerge for appending.

Thanks for the suggestions sneaker. Can I assume you are the same as sneaker_ger on Doom9? I stumbled upon some of your threads there that were very helpful! here and here.

The source comes from makemkv so there is nothing I can do about that. The source blu ray was already in h264 and stream copied in makemkv. I assume that Premiere uses a commercial encoder but I can't find any information on this.

From what I gather (predominantly from the threads cited above), the problem is with the mkv container specifically and it's inability to store multiple headers. Some people find success by using --stitchable when cutting the files, but I imagine whatever stitchable does will be thrown out the window once premiere reencodes the file. Others mentioned trying to append the raw h264 files and then remuxing into mkv but I believe that might hide the error but maintain the problem.

I could change the whole file to the mp4 container, split into the three mp4s using avidemux or something, make the edit and re-save as mp4, and then try and stitch them back together into 1 large mkv. Would that allow mkvmerge or ffmpeg concat to build an appropriate header? I can always take timeline, chapters, audio, etc from the untouched mkv if necessary.

[budwzr]

[QUOTE=Travillion;2525785]Hello all,

I am taking a single MKV file (h264 codec). I wish to make a small edit and don't want to re-encode the whole file (and thus incur quality loss).

*******************************

There's no discernable quality lost. It's digital not analog. According to "Filmmaker's Handbook" you can go 3-4 generations.

[Travillion]

Hello all,

I am taking a single MKV file (h264 codec). I wish to make a small edit and don't want to re-encode the whole file (and thus incur quality loss).

There's no discernable quality lost. It's digital not analog.

I know that stream copying will not incur quality loss, but reencoding will since it's compressing already compressed data. Now, whether or not it will be "noticeable" is subjective and debatable. I'm a purist and am trying to avoid that, plus it bothers me that something that should work in my mind is not working in reality, and I'd like to find a solution to that if possible.

Edit: I just saw your edit about the filmmaker's handbook. While the simplest solution is to just reencode the whole thing, I would like to keep as much of it untouched as possible for the reasons stated above. Maybe there is no way to do that, but my head won't drop the idea that I should be able to rejoin these files.

[smrpix]

Try VideoRedo or TMPGenc Smart Renderer. If the job is really as simple as you describe you can finish it during the free trial period.

[Travillion]

Try VideoRedo or TMPGenc Smart Renderer. If the job is really as simple as you describe you can finish it during the free trial period.

Thanks for the recommendations, I'll check those programs out. I'm hoping to find some way to cut a video (h264, mkv originally), manipulate part of it in an external editor (like premiere pro), and stitch the parts back together, ideally only reencoding the clip that was modified. It seems like it should work in my head, but maybe mkv's header handling just won't permit it.

[Travillion]

Ok, here's my last try. I got the codec private data from the various files in the pipeline, but I can't interpret them to see what is altered. I can remux with mkvmerge or ffmpeg with no issues (although for some reason fps changes between mkv and mp4 even when stream copying, private data remains the same--I remux because Premiere Pro does not play nice with mkv). The problem is when I try and export from Premiere Pro. Premiere Pro changes the private data and I don't know how or why. As you can see, the eighth entry changes from 35 to 36. Any ideas what this means? Below is the output:

parts-002.mkv (this is the segment cut from the source mkv, untouched)
================================================== ===
Video
================================================== ===
Codec 4CC: AVC1
Image Size: 1920 x 1080
Aspect Ratio: 1:1 (1:1)
Frame Rate: 23.976 fps
Total Duration: 00:00:09.426

================================================== ===
Extra Video Properties
================================================== ===
ExtraDataSize: 128
Extra data: 01 64 00 29 FF E1 00 35 67 64

================================================== ===
Audio
================================================== ===
No Audio

parts-002.mp4 (the same cut remuxed with ffmpeg, no private data changes, imported to Premiere Pro)

================================================== ===
Video
================================================== ===
Codec 4CC: H264
Image Size: 1920 x 1080
Aspect Ratio: 1:1 (1:1)
Frame Rate: 24.196 fps
Total Duration: 00:00:09.426

================================================== ===
Extra Video Properties
================================================== ===
ExtraDataSize: 128
Extra data: 01 64 00 29 FF E1 00 35 67 64

================================================== ===
Audio
================================================== ===
No Audio

parts-002-fromPP.mp4 (exported from Premiere Pro as mp4, private data changed)

================================================== ===
Video
================================================== ===
Codec 4CC: H264
Image Size: 1920 x 1080
Aspect Ratio: 1:1 (1:1)
Frame Rate: 24.390 fps
Total Duration: 00:00:09.218

================================================== ===
Extra Video Properties
================================================== ===
ExtraDataSize: 69
Extra data: 01 64 00 29 FF E1 00 36 67 64

================================================== ===
Audio
================================================== ===
No Audio

parts-002-fromPP.mkv (the mp4 exported from Premiere Pro remuxed to mkv, fps is back to spec but private data remains as exported mp4)

================================================== ===
Video
================================================== ===
Codec 4CC: AVC1
Image Size: 1920 x 1080
Aspect Ratio: 1:1 (1:1)
Frame Rate: 23.976 fps
Total Duration: 00:00:09.259

================================================== ===
Extra Video Properties
================================================== ===
ExtraDataSize: 69
Extra data: 01 64 00 29 FF E1 00 36 67 64

================================================== ===
Audio
================================================== ===
No Audio

[badyu17]

What happens when you join three parts other than mkvmerge warning about private data? Is there any glitch in the joined video where the middle part starts? In some cases parts encoded with different encoder can work without much problems, in some cases this is too much for some decoders. If there are glitches, problems, or if the video isn't playable after that part, encode your middle part with x264 and use --sps-id 1 (it can't be "0" because that's the parameter that was already used in your Blu-ray video) and join it. This is what h.264 spec allows for situations where there are different encoders used for parts of same video. I did this many times and it always works.

[Travillion]

What happens when you join three parts other than mkvmerge warning about private data? Is there any glitch in the joined video where the middle part starts? In some cases parts encoded with different encoder can work without much problems, in some cases this is too much for some decoders. If there are glitches, problems, or if the video isn't playable after that part, encode your middle part with x264 and use --sps-id 1 (it can't be "0" because that's the parameter that was already used in your Blu-ray video) and join it. This is what h.264 spec allows for situations where there are different encoders used for parts of same video. I did this many times and it always works.

Thanks for this tip, I will give it a try! I will need to find a new program because as far as I can tell ffmpeg does not allow me to use an --sps-id switch. FWIW, usually the middle section is blank--that is, once the join part is reached the picture goes black until the next join point where an unmodified section begins. However, I have found a very specific Premiere Pro export sequence wherein the video does play, at least in some cases. Still, I would prefer a workflow that doesn't involve the error messages.

I stumbled upon another thread in another forum where a member compared the header data and tried to explicitly match encoder settings. It is unclear if he/she succeeded, but I will probably try that if the sps-id does not work.

[poisondeathray]

Most x264 parameters can be translated to ffmpeg libx264 using -x264opts or -x264-params

for sps-id it would be

-x264opts sps-id=1

Output an elementary stream from ffmpeg, do not use a container, and join the elementary stream segments with binary join (dos) .

Only after it's joined, then you can use mkvtoolnix or whatever container

[Travillion]

Oh, great, I will try this when I get home. When you say 'join the elementary stream segments with binary join', I assume you mean copy /b the raw streams (.h264), then mux them into a container format? I will need to look at copy /b syntax again. I tried that earlier with test files (9-second source clip and the same clip exported from Premiere Pro)--my output was twice the size but only 9 seconds long (i.e., there didn't seem to be anything appended, even though the file size suggested there was).

[poisondeathray]

Yes, copy /b

e.g join "a" with "b" to make "c"

copy /b a.h264 + b.h264 c.h264

Make sure your streams were cut on GOP boundaries (IDR frames) in the first place

There are some "gotchas" - There are a few cases and settings that can cause problem, with some decoders - for example if your SAR switches midstream that can cause problems with some decoders. Check what it is in the original stream and use the same value in the replacement section . But for the most part the encoding settings don't have to match that much

[sneaker]

Mkvtoolnix has the capability to treat multiple input files as if they were binary joined (by putting them into parenthesis or in the GUI right-click->"Add files as additional parts"). It's not necessary to do it beforehand. Still, I'm not confident that is the best solution anyways as it doesn't guarantee SPS/PPS at every keyframe. Especially at the first part. IIRC ffmpeg concat demuxer can do that. Also, some players react badly to changing IDs.


(Mkvmerge has the ability to duplicate PPS/SPS but only will do from second part on. Maybe someone should ask the author to extend this capability.)

[Travillion]

Hmmm, this is really confusing/frustrating me.

I tried changing sps-id in the modified file and it still wouldn't join with a source file. I get the private data error and the screen is black while the modified clip plays in the newly created file.

I also couldn't binary copy the files. When I used the dos commands (copy /b), I don't get any errors and the file size increases (as if they have been joined), but the clip duration is limited to the first file (source) and it simply ends where it should have joined the modified clip. So I wondered if my code was wrong (even though it is simple copy /b a.h264 + b.h264 c.h264) and I used sneaker's process of doing this in Mkvtoolnix (Add files as additional parts"). Well, the exact same thing happened as when I tried to binary join. The file size doubles but the playback simply ends where the join should be.

I tried ffmpeg's concat demuxer several iterations back and got the same playback problems as when I tried to simply merge the files (that is, black screen for the duration of the modified clip).

Hmmph. I would blame this on Premiere Pro, but I think it is actually the source encode. Whatever settings they used are very different from what libx264 does as standard. I did an in-depth comparison of sequence parameters and came up with several differences that I can't even begin to figure out how to set in x264. There are a few easy fixes, like level (4.1 vs 4.0), but that is where I get lost. I'll post the settings below, but main differences that I can see are log2_max_frame_num_minus4, aspect_ratio_info, hrd_parameters, and lox2_mx_mv_length_horizontal. My guess is these differences are causing my main problems in joining these clips.

**Edit: Sorry, these did not transfer over in the formatting I entered. First # is the source, second # is the modified clip. When only 1 # is present, that is because it was present in the source but not the modified.**

Setting: Source: Modified Clip:
level_idc: 41 40
log2_max_frame_num_minus4: 12 0
log2_max_pic_order_cnt_lsb_minus4: 12 2
aspect_ratio_info_present_flag: 1 0
aspect_ratio_idc: 1
fixed_frame_scale: 1
nal_hrd_parameters_present: 1 0
cpb_cnt_minus1: 0
bit_rate_scale: 1
cpb_size_scale: 3
bit_rate_value_minus1[0]: 249999
cpb_size_value_minus1[0]: 234374
cbr_flag[0]: 0
initial_cpb_removal_delay_length_m1: 17
cpb_removal_delay_length_minus1: 17
dpb_output_delay_length_minus1: 5
time_offset_length: 24
vcl_hrd_parameters_present_flag: (same as nal_hrd_parameters above)
....
pic_struct_present_flag: 1 0
log2_mx_mv_length_horizontal: 13 11
num_reorder_frames: 4 2


Any ideas? Is this just a lost cause? Thanks

[poisondeathray]

Did you remember to fix your fps ? If the declared framerate in the video streams is 23.976 for one, but 24.930 for the other, it's not going to join properly

The most common cause of failure people have is not cutting at IDR frames, or the SAR changing . Another tricky scenario is where there is timecode written into the file as metadata . But fps mismatch is easy to address (I would have fixed it before importing into PP)

If you still can't figure it out, post a sample, (at least 3 GOP lengths) of the clips "a" and "b" you want to join with a short description of what goes where



That last info you posted suggests VBV parameters. (HRD was used, perhaps from BD for example) . Note that patching in a segment does not ensure VBV compliance - it might cause buffer underrun error for example if you were trying to reauthor a BD (it would work fine with a "normal" file where you don't have decoder or bitrate restrictions). If you need that you'd have to use a tool with proper segment re-encoding that abides by VBV compliance (no free ones, not even videoredo or solveigmm can do this properly)

[Travillion]

Did you remember to fix your fps ? If the declared framerate in the video streams is 23.976 for one, but 24.930 for the other, it's not going to join properly

In this case I did fix the framerate. I got sick of trying to troubleshoot with the full files and so I made two consecutive cuts from the source (mkv container), each 10 seconds long, and then imported cut #2 into PP and directly exported it as a lossless avi (ut video), ran it through ffmpeg back into h264 (mkv) using standard settings. It is these two files now that I am trying to join. MediaInfo lists identical fps (23.978) and dimensions.

The most common cause of failure people have is not cutting at IDR frames, or the SAR changing . Another tricky scenario is where there is timecode written into the file as metadata . But fps mismatch is easy to address (I would have fixed it before importing into PP)

I don't think this is an IDR issue because the clips play fine independently and if I rejoin them untouched, they play together seamlessly. It is only when I re-encode one of the files that they no longer rejoin. See above for fps and SAR info. There are different SPS flags for SAR (that is, one has it flagged and the other doesn't) but the dimensions are the same for both files.

If you still can't figure it out, post a sample, (at least 3 GOP lengths) of the clips "a" and "b" you want to join with a short description of what goes where

I'll work on this. I need to verify how many GOPs my clips contain.

That last info you posted suggests VBV parameters. (HRD was used, perhaps from BD for example) . Note that patching in a segment does not ensure VBV compliance - it might cause buffer underrun error for example if you were trying to reauthor a BD (it would work fine with a "normal" file where you don't have decoder or bitrate restrictions). If you need that you'd have to use a tool with proper segment re-encoding that abides by VBV compliance (no free ones, not even videoredo or solveigmm can do this properly)

Hmm, this sounds exactly like my problem. The source is a bluray I purchased and ripped through makemkv. I've never tried to modify one of my rips, but in this instance I wanted to prank my wife by tweaking a clip. I could certainly re-encode the whole movie, but it didn't seem like an issue to just cut and splice. Clearly it is.

I don't know much about VBV. I couldn't find anything in ffmpeg or libx264 literature about how to provide any of those values to the encoder. I'd still like to figure this out for proof of concept if possible, ... if possible.

[poisondeathray]

If you're not going back to BD or a device target, you don't need to worry about VBV, and can ignore the rest of this paragraph - It's just a way of constraining the bitrate so it complies with devices according to certain levels. Hardware decoders have limitations and it boils down to formula that takes into account input, output and current buffer level. Your patched segment won't take into account the current buffer level (because it doesn't "look" at the rest of the movie, so it can't calculate the current position or how full or empty the buffer is) , so it would have slight potential for problems. If you were going back to BD or device targets, you would err on the LOWer bitrate side. An bitrate peak or spike could empty out the buffer and you would either stutter or abort playback

To set sar in ffmpeg it's -vf setsar . If you don't include it, the sar value will actually be "undefined" . For retail BD it would be set to 1:1

-vf setsar=sar=1/1

Since you know it's directly from a BD source, it wouldn't hurt to use BD compliant settings either. Yes, the sps-id method enables joining segments that use different encoding settings, but there are a few that can still cause problems.
This reference is for x264, but you can translate some directly or others with -x264opts or -x264-params as mentioned earlier . For example -g is the GOP max keyframe interval for ffmpeg (--keyint in x264 syntax) and for a film BD, it would be 24
https://sites.google.com/site/x264bluray/home

Just an FYI, there is a voukoder plugin for PP that is based on libav/ffmpeg that can export x264 directly too.

[Travillion]

That's some great info Poisondeathray, thank you!

I'm not trying to go back to disc, just ending up with a playable mkv file. In that case, should I worry about VBV at all? The SPS gave two figures for bit rate, I'm guessing one was the max (bit_rate_value) and one was the buffer (cpb_size_value)? Would I gain any benefit by using those same values?

I'll dig deeper into the authoring settings you referenced. As you said, those are probably best practices considering the source.

PP has an h264 encoder, but I don't love it. I was exporting as lossless because I figured I had to reencode in ffmpeg anyway because PP's encoder is extremely limited and I initially thought it was the cause of my problems. I'll take a look at the encoder you referenced, it certainly couldn't be worse than what they included.

[poisondeathray]

I'm not trying to go back to disc, just ending up with a playable mkv file. In that case, should I worry about VBV at all? The SPS gave two figures for bit rate, I'm guessing one was the max (bit_rate_value) and one was the buffer (cpb_size_value)? Would I gain any benefit by using those same values?

No benefit . You can only enter --vbv-maxrate and --vbv-bufsize (in x264 syntax) for the encoder side, but since you're only looking at 10sec segment, the current buffer level will not necssarily be accurate . Just go for a bitrate target on the lower side of what the source was

I'll dig deeper into the authoring settings you referenced. As you said, those are probably best practices considering the source.

You probably don't need to. But mismatched sar, as I mentioned earlier, is one of the common ones that cause problems.

PP has an h264 encoder, but I don't love it. I was exporting as lossless because I figured I had to reencode in ffmpeg anyway because PP's encoder is extremely limited and I initially thought it was the cause of my problems. I'll take a look at the encoder you referenced, it certainly couldn't be worse than what they included.

Yes, the Adobe h264 one is licensed from Mainconcept. Not that good. Yes people use either that lossless workflow , or they used x264pro plugin ($) , until Voukoder came along

But Voukoder is free/open source and uses ffmpeg/ libav libraries . So the same thing you're using in ffmpeg. (ie. same library as ffmpeg libx264) . It's the best AVC/h.264 encoder overall in most scenarios. There is a commandline box that can take additional switches too. But maybe sort this problem out before you start playing around with new toys

[Travillion]

Ok, I'll try a new encode explicitly starting the sar. I've never set that before and just assumed it was derived automatically from the file. ffmpeg -i input always displays the sar, but maybe in that case it is determining it from the file but to join them it needs to be stated. Maybe that's causing the problem, if the untouched segments have a specified sar field and the modified segment switches to "undefined." Crossing my fingers.

I'm off to bed now, I'll pick this back up tomorrow evening. Thanks again!

[sneaker]

Can you upload the first 30 seconds of your source?

[Travillion]

Here are the first 30 seconds. I was troubleshooting with smaller (10 second) chunks, reencoding 1 segment and trying to rejoin it with the preceding 10 seconds, all to no avail. I took all the audio, subtitles, etc. out to make it simpler. If someone is able to figure out reencoding settings that will allow clips to join, I would be most appreciative!

[poisondeathray]

Can you add the short description - Where are your edits ? What is joining to what or what goes where ? Or upload the stuff you're actually trying to process (ie. the PP export)

Or is that your "A" piece ? And you just want to see if you can append anything to the end successfully ?

[sneaker]

First look at the sample: it uses OpenGOP and 4 slices per frame. (4 slices is very common on Blu-Ray)

[Travillion]

Can you add the short description - Where are your edits ? What is joining to what or what goes where ? Or upload the stuff you're actually trying to process (ie. the PP export)

Or is that your "A" piece ? And you just want to see if you can append anything to the end successfully ?

Sorry for the confusion. This is the first 30 seconds of the source, untouched, as sneaker requested. What I have been doing is cutting the first and second 10 seconds, recoding the second 10 seconds, and trying to rejoin them. I'm doing this because it is more efficient than trying to rejoin the full 3 segments which constitute the whole movie (beginning, edit, and ending), but should be the same in principle. I will also upload the sections I have been troubleshooting with, but I thought this would be a good sample for people because there won't be any confusion about whether I cut on non-keyframes, or people could make their own samples to test with, etc.

[Travillion]

First look at the sample: it uses OpenGOP and 4 slices per frame. (4 slices is very common on Blu-Ray)

OpenGOP just means the cut points will be trickier, correct? It will have to cut on IDR frames and not i frames for proper playback? But that shouldn't preclude rejoining files. Is the 4 slices per frame a setting that would need to be set in the recode to create a matching segment?

[poisondeathray]

Sorry for the confusion. This is the first 30 seconds of the source, untouched, as sneaker requested. What I have been doing is cutting the first and second 10 seconds, recoding the second 10 seconds, and trying to rejoin them. I'm doing this because it is more efficient than trying to rejoin the full 3 segments which constitute the whole movie (beginning, edit, and ending), but should be the same in principle. I will also upload the sections I have been troubleshooting with, but I thought this would be a good sample for people because there won't be any confusion about whether I cut on non-keyframes, or people could make their own samples to test with, etc.

But where did you make cuts exactly ? Because most of those are open GOP (not IDR frames or "I" frames , but rather "i" frames) . Recall I said the most common problems were : not cutting on IDR frames, and SAR

[Travillion]

Here are three additional files. testparts-001 and -002 are the first and second 10-s clips from the source file in the above post (test.mkv). They are cut with mkvmerge (mkvmerge -o testparts.mkv --split timestamps:00:00:10,00:00:20 input.mkv). Mkvmerge will have cut them at the closest keyframes to those timepoints, I don't know if they were i frames or IDR frames. BUT, they play back just fine by themselves and rejoin without any problems (no error messages and smooth playback; mkvmerge -o output.mkv testparts-001.mkv + testparts-002.mkv), so I'm assuming the cut point is not the problem.

However, once testparts-002 is recoded, it won't join with testparts-001 (private data error and black screen after join point). testparts-002film.mkv is that second part, imported into PP, exported as lossless avi (ut video), and then recoded in ffmpeg with medium preset, crf 17, and film tune. I can't find any encode settings that will allow these to rejoin once one of the files has been modified.

[Travillion]

First look at the sample: it uses OpenGOP and 4 slices per frame. (4 slices is very common on Blu-Ray)

OpenGOP just means the cut points will be trickier, correct? It will have to cut on IDR frames and not i frames for proper playback? But that shouldn't preclude rejoining files. Is the 4 slices per frame a setting that would need to be set in the recode to create a matching segment?

I see that x264 does have a --slices setting, the default value is no slices per frame x264 settings. Not sure if that would make a difference but I'll add it to my list of settings to try.