Issing re-joining mkv files split using mkvtoolnix

[sneaker]

But what use is a list of frame types on its own? You need to know about re-ordering. (And if it's not Blu-ray you need to know even more.)

[jagabo]

But what use is a list of frame types on its own?

The OP wants to know the frame number of an I frame so he knows where to cut. This will work just fine for closed GOPs. I don't know how ffmpeg handles the cutting of open GOPs.

[poisondeathray]

He's cutting with mkvmerge, not ffmpeg. Maybe he should try ffmpeg

v10 has some changes, might be worth a try repeating the cuts with the newer mkvmerge version

mkvmerge: AVC/h.264 parser: mkvmerge will now drop all frames before the first key frame as they cannot be decoded properly anyway. See #1908.

Furthermore, if I try to play each of the individual split files separately before re-joining them they play fine. This includes the first few seconds of each file that is skipped when I try to re-join the ones I want together.

So what did that mean exactly ? Were there "extra frames" when you cut ? What are those first few seconds referring to ?



Or, if cutting isn't the problem, maybe there is a problem with appending ? For example can you append the cut segments with solveigmm or videoredo ? If those fail then it suggests the problem was indeed with the cutting

[Odaik]

Thanks again guys - really appreciate all the help.

Sneaker I did think of that but unfortunately there is a small rounding error involved (best_effort_timestamp_time is not an exact multiple of 1/23.976) and after a large number of frames I don't think the resulting frame number is reliable. Jagabo, I tried that and it works a treat, thanks. I can get a rough timestamp from the film, divide by the frame duration and round to the nearest integer, and use the closest I frame with a number closest to the result.

I did end up repeating the whole splitting/re-joining process using mkv merge and using ffprobe to analyse the cuts and final product, and it showed something interesting. There are four scenes in the film I wanted to cut out and to do so I entered the start and end timecodes of these scenes in to the "splitting" section of the mkvmerge GUI. I add the 4 raw streams (2 sub, 1 audio, 1 vid) and a chapter file (through the "output" tab) and start the muxing process. This produces 9 mkv files. Parts 2,4,6 and 8 are the mkv files that contain the scenes I wish to cut out, so I delete these files. I then re-open mkvmerge, add the part 1 mkv, and append parts 3,5,7 and 9. This produces the final singular mkv file that has been giving me problems at the join points. There are four join points (part 1 to part 3, part 3 to part 5 etc)

Analysing each of the individual 9 parts as separate parts shows that each one of them starts with an I frame, and ends in a P frame regardless of how long it is. I then analysed the final singular mkv and came across two issues. Firstly, in the process of performing the frame analysis, the ffprobe cmd window returns an error stating:

[h264 @ 00000000011b8a80] co located POCs unavailable

By the end of the run:

Last message repeated 7 times

Looking at the frame analysis text file for the final product, I find a further clue as to what is going on. Viewing the mkv file in a player, the points in the film where the first two joins occur have no problem. The second two joins are where the video will pause for some time while the audio continues, and then skip ahead. Looking these timestamps up in the frame analysis text file, I can see that where the join occurs, there is a long gap between the timestamps of the last frame of one mkv file and the first one of the file that is appended to it. For example, at the point where the join between parts 5 and 7 occurs, the "best_effort_timestamp_time" value for the last frame from part 5 is 4349.520000, but the very next frame (the first from part 7) has a timestamp of 4361.373000. That's a gap of about 12 seconds, which is actually how long the video freezes for at the point when I try to play back the faulty mkv. Further, looking at the separate frame analysis text file for the part 7 mkv, the first displayed frame has a timestamp of 0.083000 instead of 0. It also has "coded_picture_number" = 4, but I cannot find coded pictures 1, 2, or 3 (either visually looking down the document or using the find tool). These seem to have been dropped completely. Identical issues identified in the analysis of the part 9 mkv, with a similar video freeze of 6 seconds in the final mkv where part 9 starts. The mkv files for parts 1, 3 and 5 don't have these issues (with the joins in the final mkv between parts 1 and 3, and between 3 and 5 being completely fine).

The problem seems to have been introduced when splitting with mkvmerge, and compounded upon re-joining the desired parts. Any idea what could be going on specifically? I'm just trying to figure out whether it's worth identifying exactly what mkvmerge is doing to cause the issue to see if a workaround could be done. Otherwise I may just have to give up and either find a tool that will do the job without any issues.

By the way, looking through the frame analysis files I can see many examples of a "B" frame immediately preceding an "I" frame. Is this definitive confirmation that the film uses Open GoP encoding, as there are many examples of frames in one GoP referencing the keyframe for the next one?

[Odaik]

Apologies, looks like there were a few posts since I refreshed the page and I missed these before responding. Will try and reply in a couple of hours.

[jagabo]

By the way, looking through the frame analysis files I can see many examples of a "B" frame immediately preceding an "I" frame. Is this definitive confirmation that the film uses Open GoP encoding, as there are many examples of frames in one GoP referencing the keyframe for the next one?

Are the coded frame numbers of the B frames greater than the coded frame number of the I frame? If so, I'm pretty sure that's a definitive indication of open GOPs.

[Odaik]

As far as I can tell, most of them are like that, yes. Quite a common pattern is two B frames consecutively, then an I frame. An example, in display order, is a B frame with coded number 149, a B frame with coded number 150, and an I frame with coded number 148. From what you've explained, I think this means it is an open GoP blu-ray.

With open GoP, is there any limit to how "far" ahead a B frame can reference information? I know it's unlikely because of the lack of re-usable elements from completely different scenes, but could a B frame technically refer to information in an I frame, say, an hour later in the movie?

[sneaker]

Yes, it's likely OpenGOP. Which mkvmerge cannot safely cut unless you only pick IDR-frames. Which I have been telling you since the beginning. Stop trying to make it happen.

[sneaker]

With open GoP, is there any limit to how "far" ahead a B frame can reference information? I know it's unlikely because of the lack of re-usable elements from completely different scenes, but could a B frame technically refer to information in an I frame, say, an hour later in the movie?

On Blu-ray H.264 references must never cross I frames and I frames must appear at least every other second. So: no.

[jagabo]

With open GoP, is there any limit to how "far" ahead a B frame can reference information?

The number of consecutive B frames allowed in the encoding. 3 in Blu-ray compatible encodes, as many as 16 is allowed in the h.264 spec.

[sneaker]

Ah, I missed the "ahead". Then it's indeed even shorter because of the 3 consecutive bframe limit. So ahead: 3 frames, back: max. 2 seconds.

[Odaik]

OK, bearing that in mind I went through the list of frame types produced using the command jagabo posted. I looked up the frame numbers roughly corresponding to the timecodes for the "successful" split/join points, and found that at those particular points the I frame is preceded by a P frame, not a B frame. The vast majority of times an I frame appears, the preceding frames are (...BBPBBPBBPBBI). Very occasionally, I find (...BBPBBPBBPBBPI) instead. For every point where a split/re-join hasn't introduced problems, the latter pattern is what I find at the corresponding timecode in the frame analysis.

Therefore, I thought that I could solve the issue by simply going to the frames that corresponded to the timecodes for the split points I am having trouble with, and looking for a few seconds (about 100 frames worth) in either direction to see if I could see the same pattern of frames as above (with a P instead of a B before the I frame). I would then take the frame number of that new keyframe (by looking at the line number in notepad), converting to a timecode, and using that instead in mkvmerge. I found examples within about 4 seconds off of my original cut point, which isn't bad.

My thought was that this particular pattern of frames probably preceded a proper IDR type keyframe, as in the first two out of the four cuts (which were fine) this was the pattern I saw in the text file. Unfortunately I'm still running into issues. I no longer get the video freezing, but there is some "blockiness" for two frames after the split point. I guess managing to cut at a real IDR frame the first couple of times was just a coincidence that I can't replicate for the remaining cut points.

Is there any way to definitively identify IDR type I frames from the text files ffprobe produces? Is there some command I can add to the .bat file? Alternatively, I could just use another program that searches for and recognises IDR frames to cut the video. I did try mkvmerge v10 by the way, but it did not solve the issue. I am trying to using ffmpeg, but can't get my head around the required code for the CLI yet.

[poisondeathray]

Is there any way to definitively identify IDR type I frames from the text files ffprobe produces?

If the pict_type is "I" and it says key_frame "1" it's an IDR frame
If the pict_type is "I" and it says key_frame "0" it's an "i" frame (non IDR)

[poisondeathray]

But I just ran some quick tests and ffprobe seems unreliable (or at least it doesn't entirely match results of commercial stream parser)

[sneaker]

On Blu-ray H.264 IDR frame should be if coded picture number matches display number. As you found out and was already said earlier: the re-ordering is important. Just looking at the pattern without thinking about the frame numbers to find out re-ordering you can't be sure e.g. BBPIBB is IDR.

[Odaik]

Is there any way to definitively identify IDR type I frames from the text files ffprobe produces?

If the pict_type is "I" and it says key_frame "1" it's an IDR frame
If the pict_type is "I" and it says key_frame "0" it's an "i" frame (non IDR)

From what I can tell, all the I type frames in the log are flagged as key_frame = 1. Is this just a bug or is it simply the case that ffprobe always classes both IDR and non-IDR "i" frames as key frames?

Do you have any suggestions for a commercial stream parser that I could use?

On Blu-ray H.264 IDR frame should be if coded picture number matches display number. As you found out and was already said earlier: the re-ordering is important. Just looking at the pattern without thinking about the frame numbers to find out re-ordering you can't be sure e.g. BBPIBB is IDR.

OK, I went through the full analysis text file produced by ffprobe, and also the one produced using the command jagabo posted. I identified the frames using the latter and what the display number was (just line number - 1, as frame count starts at 0 and line number starts at 1). I then looked up the I frame at the same timecode in the full analysis file, and checked the coded picture number. In all cases the I-frames at the points the film was cut had the same display and coded number. That's the case for both the split points where the split and the re-join occurred seamlessly, and the frames where re-joining introduced problems.

Could it be then that, given the above points, these are actually genuine IDR frames, and there is something wrong with the way mkvmerge is cutting everything for some reason? I know the problem starts at the splitting process, as the analysis of the split mkv files reveals the issue with faulty time codes and coded picture numbers starting at 4 etc.

I'm still trying to wrap my head around using ffmpeg. Unfortunately Avanti GUI doesn't seem to be that intuitive either. Does anyone know what command I have to run to simply split a given mkv file (inc video, audio, and 2 sub streams) at given points, without re-encoding?

[sneaker]

Upload a minute or so from the m2ts (e.g. cut first 150 MB using DGSplit or try tsmuxer) so we can see if we can reproduce the problem.

[poisondeathray]

Do you have any suggestions for a commercial stream parser that I could use?

If you were shying away from solveigmm video splitter or videoredo because of the price, you definitely don't want to go this route