Video stutters after trimming

[ndjamena]

No, I mean THERE ARE NO B-FRAMES IN THE VIDEO, he's set them to 0, yet the audio and video frames are technically out of order in the MKVMerge created MKV. Unless you can find a way to rule that out as a possibility, that's what I'm getting out of this thread.

[Selur]

Strange, but still if it's a valid stream the decoder should handle this properly.
Just for the fun of it he could remux the mkv to mp4 (if the formats are supported) and see if that helps.

[poisondeathray]

No, I mean THERE ARE NO B-FRAMES IN THE VIDEO, he's set them to 0, yet the audio and video frames are technically out of order in the MKVMerge created MKV. Unless you can find a way to rule that out as a possibility, that's what I'm getting out of this thread.

Huh? There are b-frames in both videos

[poisondeathray]

ffprobe reports this about "original.mkv"

[h264 @ 03280be0] Delayed frames seen. Reenabling low delay requires a codec flu
sh.

[ndjamena]

No, I mean THERE ARE NO B-FRAMES IN THE VIDEO, he's set them to 0, yet the audio and video frames are technically out of order in the MKVMerge created MKV. Unless you can find a way to rule that out as a possibility, that's what I'm getting out of this thread.

Huh? There are b-frames in both videos

cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / stitchable=1 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=34 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=34 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=2 / qpmax=69 / qpstep=4 / vbv_maxrate=20000 / vbv_bufsize=25000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00

That's the trimmed version.
The original doesn't have any settings attached to it. Maybe you looked further inside the file than I did.

[ndjamena]

Sorry, I was looking at the Remuxed version, the original is even worse:

| + Block group at 12768
| + Block (track number 1, 1 frame(s), timecode 0.375s = 00:00:00.375) at 12770
| + Frame with size 70
| + Reference block: -41.000000ms at 12846

And this is the next video frame:

| + Block group at 27170
| + Block (track number 1, 1 frame(s), timecode 0.876s = 00:00:00.876) at 27173
| + Frame with size 15922

There's a whole lot of audio frames in between. But that's wrong. Did you happen to append two files together to create this one?

And the frame after that:

| + Block group at 54365
| + Block (track number 1, 1 frame(s), timecode 1.251s = 00:00:01.251) at 54369
| + Frame with size 27299
| + Reference block: -375.000000ms at 81676


Long after that we get this:

| + Block group at 441999
| + Block (track number 1, 1 frame(s), timecode 0.417s = 00:00:00.417) at 442003
| + Frame with size 57555
| + Reference block: 501.000000ms at 499566

Are there b-frames or aren't there?

[poisondeathray]

No, I mean THERE ARE NO B-FRAMES IN THE VIDEO, he's set them to 0, yet the audio and video frames are technically out of order in the MKVMerge created MKV. Unless you can find a way to rule that out as a possibility, that's what I'm getting out of this thread.

Huh? There are b-frames in both videos

cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / stitchable=1 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=34 / keyint_min=3 / scenecut=40 / intra_refresh=0 / rc_lookahead=34 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=2 / qpmax=69 / qpstep=4 / vbv_maxrate=20000 / vbv_bufsize=25000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00

That's the trimmed version.
The original doesn't have any settings attached to it. Maybe you looked further inside the file than I did.

I examined at the actual file, not just the metadata

The small re-encoded section is one IDR frame and the rest P-frames, but the rest of the file still has b-frames

The original had a bunch of non key "i" frames and b-frame there

[ndjamena]

So what does the actual stream header say, that's what the player will be following?

[poisondeathray]

So what does the actual stream header say, that's what the player will be following?

In terms of what? Can you clarify what are you are asking?

[ndjamena]

I'm not sure, I've never seen a h.264 header. There must be something that prepares a decoder for b-frames.

[poisondeathray]

I don't know exactly how the decoder part of that equation, maybe selur knows or ask at doom9 - but there are slice headers that indicate slice type e.g. "P slice" or "B slice" etc...

However the decoder definitely does not pay any attention to the metadata

[ndjamena]

Well, it's using the header for the re-encoded portion, the presence of the x264 settings proves that, and for whatever reason x264 set the b-frames to zero. Should I think that's not relevant to the problem?

[poisondeathray]

Well, it's using the header for the re-encoded portion, the presence of the x264 settings proves that, and for whatever reason x264 set the b-frames to zero. Should I think that's not relevant to the problem?

Yes, that re-encoded section has zero b-frames

But this is just a x264 SEI string, it has no importance to decoder - it doesn't even look at it. You can use a hex editor and delete it or write bframes=8 and it make no difference

I don't know if using zero b-frames is relevant, but the presence of b-frames might cause problems in some situations due to bidirectional nature, or decoder lag (even back in xvid days), but the absence usually doesn't. If anything , using no b-frames or even all I-frames should cause fewer problems

I think problem might be related to what ffprobe detected about "delayed frames", in the "original" video. But recall he said even the "original.mkv" had problems compared to the original, original. So there might be some problems in how he prepared the sample "original.mkv" in the first place

[ndjamena]

I almost forgot that the FFMPEG version plays perfectly. I'd take the time to examine it more thoroughly, but I'm pretty sure Selur is on top of this.

[Selur]

Nope. Selur doesn't see a mistake in the way MKV Cutter is behaving. If the input is broken (doesn't play well) it's no surprise the output of a cut isn't better.
I could rewrite Mkv Cutter to do more thorough identification of the source, this would require to extract the raw stream and run the whole raw stream through h264_parse, but:
a. this would slow down the whole process
b. even if MKV Cutter would use b-frames, it would probably not help with the playback.
From the sound of it, the only way to get a file which properly plays would be to reencode the whole file.
-> If someone can tell me what Mkv Cutter did wrong I could look into it, but like poisondeathray mentioned ' using no b-frames or even all I-frames should cause fewer problems'.

Cu Selur

[ndjamena]

OK then. I'll look into it if I can be bothered. It's a learning process after all.

[ndjamena]

[matroska @ 00000000057f4020] Non-monotonous DTS in output stream 0:0; previous:
 1251, current: 1210; changing to 1251. This may result in incorrect timestamps
in the output file.
[matroska @ 00000000057f4020] Non-monotonous DTS in output stream 0:0; previous:
 1251, current: 1168; changing to 1251. This may result in incorrect timestamps
in the output file.
[matroska @ 00000000057f4020] Non-monotonous DTS in output stream 0:0; previous:
 1251, current: 1126; changing to 1251. This may result in incorrect timestamps
in the output file.

OK, I was right. FFMPEG was expecting all the video frames to be in order, when it found that they weren't, it changed the timestamps - oddly it made them all the same, but at least they didn't go backwards anymore.

But you may be interested to know, that remuxing the original file using FFMPEG doesn't bring up the same Non-monotonous DTS errors, so it may not be a problem with the original file. FFMPEG removed a 42millisecond delay in the video, but that's about it (the frames are still out of order, in the original and the FFMPEG remux).

Maybe it's a problem with X264. I believe PDR said the re-encoded potion amounted to one single I-frame. Is it possible X264 saw that and automatically disabled B-frames because it wasn't relevant? I admit I have no idea how to read an X264 header, so I may have missed something.

[Selur]

No. MKV Cutter doesn't use B-Frames with x264 since the headers of the input file it got didn't show that the content contained b-frames.
The whole talk about b-frames is just misleading and doesn't have anything to do with the problem.
The problem is that you got non monotonous DTS inside your input stream, which has nothing to do with b-frames, but indicate broken time stamps.
MKV Cutter doesn't try to fix time stamps or other headers it assumes the input is okay and therefore non reencoded parts will be kept as they are.

[ndjamena]

But don't the non monotonous DTS timestamps exist because the b-frames are supposed to be decoded AFTER the other frames and so are stored AFTER the other frames even though their timestamps are EARLIER?

AVIDemux opens the "original" file just fine, and there are definitely b-frames in there, which means the frames are SUPPOSED to be stored out of order in the MKV.

[Selur]

You are mixing stuff.
a. Yes, for b-frames the coding order differs from the display order, thus a container will contain the frames in coding order and the decoder has to decode them in display/decoding order.
b. But, this should not lead mixed time stamps. The errors posted should not occur unless a stream is broken.
MKV Cutter always takes a whole GOP, feeds it to the decoder and then reencodes all the frames of the GOP which should be kept.
-> Replacing a GOP with b-frames with a new and shorter GOP which doesn't have b-frames will not cause any problem.

The difference between coding and display ordered frames should not play any role, assuming the time stamps are okay. If FFMPEG reports non monotonous time stamps, it does not indicate b-frames but broken time stamps. FFmpeg is able to handle the difference between coding and display order just fine for years. When it reports a problem with the time stamps it's because they are broken in one way or the other.

[ndjamena]

Yes, I can see the "original" file is balked now. If I play the "original" file forward using AVIDemux it shows that the "middle" section contains 24 b-frames in a row, if I then step backwards frame by frame the frames alternate between P and B. If the order of the frames in the Trimmed MKV is the decoding order, then to display the frame at 459ms you'd need to decode at least 10 frames first, I'm no expert on codecs but I imagine you'd need to store those 10 frames too, either that or you're supposed to decode them twice. Since they're ordered in display order in the "Original" mkv, I'm going to go out on a limb and say those b-frames are actually p-frames and that the pony section of the video doesn't have any b-frames at all.

I'd like to conclude that the MadMan logo and the ponies come from two separate encodes with different settings that were appended, but I'd need to know where the video was sourced from to be sure.

(oddly, no matter how many times and no matter how many ways I remux the "original", I can't get the middle section to be ordered in any way other than display order, so why is the "trimmed" version in that odd b-frame reverse order?)

That was a waste of time.

[poisondeathray]

If I play the "original" file forward using AVIDemux it shows that the "middle" section contains 24 b-frames in a row, if I then step backwards frame by frame the frames alternate between P and B. If the order of the frames in the Trimmed MKV is the decoding order, then to display the frame at 459ms you'd need to decode at least 10 frames first, I'm no expert on codecs but I imagine you'd need to store those 10 frames too, either that or you're supposed to decode them twice. Since they're ordered in display order in the "Original" mkv, I'm going to go out on a limb and say those b-frames are actually p-frames and that the pony section of the video doesn't have any b-frames at all.

Yes they are P frames, avidemux doesn't handle this properly

I'd like to conclude that the MadMan logo and the ponies come from two separate encodes with different settings that were appended, but I'd need to know where the video was sourced from to be sure.

I would say probably, but not using mkvcutter (since selur confirmed it doesn't use b-frames). That section preceding the scenecut has a bunch of non IDR "i" frames

[Freodon]

I extracted the "original" sample with mkvCutter, I see it's messed up. How should I extract one to provide a non-bugged sample?

[Selur]

since selur confirmed it doesn't use b-frames)

Not in general, but in this case since the mediainfo analysis of the input doesn't indicate b-frames.

btw. easiest way to know what really is inside the source file, would be to:
1. extract the raw video stream (using in example mkvextract or ffmpeg, ...)
2. run the raw file through h264_parse

[poisondeathray]

since selur confirmed it doesn't use b-frames)

Not in general, but in this case since the mediainfo analysis of the input doesn't indicate b-frames.

It's good that you clarified that

btw. easiest way to know what really is inside the source file, would be to:
1. extract the raw video stream (using in example mkvextract or ffmpeg, ...)
2. run the raw file through h264_parse

Easiest free way. There are easier ways with various stream analyzers.

[ndjamena]

I extracted the "original" sample with mkvCutter, I see it's messed up. How should I extract one to provide a non-bugged sample?

If the source is bugged then you can't.

If you want to rule out mkvCutter as a problem, cut the earliest source you have access to with MKVMerge (assuming it's not an actual blu ray or something).

Looking at the thing through FFInfo seems to show that the non-IDR frames are caused by the fade-in at the beginning, which makes sense, and there are in fact b-frames in the middle section in a repeated P-B-P-B pattern, which kind of hints there the b-frames parameter should be 1. I still haven't found anything in the h264 headers that indicates b-frames (something to do with a delay before display maybe?) but it really does look to me like the programs are expecting there to be no b-frames at all in that middle section. Whether there are or not I can't tell yet, but 1 - 1 = 0 so it's possible there was a bug in the program that encoded the video, which means you really shouldn't be re-muxing your file with anything, ever, and really ought to try just re-encoding the whole thing and hope for the best.

[Freodon]

The earliest source of the video I have access to is an iTunes video un-DRMed with Brahms' "Requiem" program. I have no info on the actual method of doing this. Before I extracted the sample, there were no problems at all in the original video. There also don't seem to be problems if I cut something out from the middle - only if I cut from the beginning or end. This video is one of many from the series, and the problem doesn't appear in 80% of them (after trimming) - it seems random, and I don't know what to think about it.

[ndjamena]

No program is immune to bugs, even the ones Apple uses. Unless you have reason to believe a program you used has caused the issue (which is unlikely) there's really nothing you can do. It's occurred to me the frame number or pic order fields may be wrong and that may be causing the problem, but I don't really know and there's not much point in finding out anyway. It's not something that would be easily fixed in any case. You can't really know if the problem occurs with other videos, because it's not always obvious. Like I say, I can remux the file with FFMPEG and it plays back perfectly, despite the fact the FFMPEG pretty much obliterates half the time code information.

Again, my suggestion is the find the decoder that plays it back the best and re-encode the thing. The resulting video may be of slightly lower quality but as things stand the stream itself is damaged. That should put an end to this once and for all.

[Freodon]

Would it be more beneficial to re-encode before or after trimming?

[ndjamena]

During.