Message : B-pyramid level 2 detected. Shift DTS to 3 frames

[fdenis31]

Hey guys !

I'm trying to remux a bluray using tsmuxer (a h264 video + a dts track + subtitles).
But when i do that i see this message in tsmuxer log :
B-pyramid level 2 detected. Shift DTS to 3 frames

Any idea what it means ?

Regards

[Selur]

it's just an info that b-frames (https://en.wikipedia.org/wiki/Video_compression_picture_types) are used in a pyramid like fashion with two levels and to compensate the (Decode Time Stamps) haven been adjusted accordingly.
-> it's just an info and the b-pyramid should normally not cause a problem

[jman98]

DTS = Decode Time Stamps?

That is a really unfortunate abbreviation given that it also is the name of an audio format and most people will take DTS to mean the audio format.

[Selur]

yup, in this context dts should be "decode time stamp"
iirc the dts = "decode time stamp" abbreviation is older than the dts = 'digital theater sound' abbreviation

[digitalfreaknyc]

it's just an info that b-frames (https://en.wikipedia.org/wiki/Video_compression_picture_types) are used in a pyramid like fashion with two levels and to compensate the (Decode Time Stamps) haven been adjusted accordingly.
-> it's just an info and the b-pyramid should normally not cause a problem

In my case, it's screwing up my files. Is there any way to remove this option?

[Selur]

reencode the file with a software which doesn't have a problem with b-pyramids (normally nearly all software should support it) and reencode it with b-pyramid disabled.

[digitalfreaknyc]

reencode the file with a software which doesn't have a problem with b-pyramids (normally nearly all software should support it) and reencode it with b-pyramid disabled.

There's got to be a better option than that. I record all of my HD material with the Hauppauge and it apparently records with the b-level pyramid on all of its recordings. I don't have a problem at all when I play them back with my blu-ray players. However, the problem is when I use multiavcHD to author the discs. It uses tsmuxer which always screws with the b-level pyramid and makes ALL files recorded with the Hauppauge and edited with videoredo jump. It only happens with the first b pyramid. For some reason, all others are left alone.

[Selur]

Let me guess if you don't edit them with videoredo the issue doesn't arise, right?
SmartCutters sometimes mess up the headers, so depending if not using VideoRedo helps, I would recommend to ask the VideoRedo or multiAVCHD authors for advise.
(development of tsmuxer has been halted for some time now and nobody has contact to the author, so no real hope that tsmuxer will be fixed assuming this is a bug in tsmuxer)

[digitalfreaknyc]

Let me guess if you don't edit them with videoredo the issue doesn't arise, right?
SmartCutters sometimes mess up the headers, so depending if not using VideoRedo helps, I would recommend to ask the VideoRedo or multiAVCHD authors for advise.
(development of tsmuxer has been halted for some time now and nobody has contact to the author, so no real hope that tsmuxer will be fixed assuming this is a bug in tsmuxer)

Bingo! THANK YOU!

I'm actually talking to the authors of VRD but they're saying I should talk to tsmuxer authors. Of course, it's one blaming the other. However, I have to say that the VRD guys are super nice and willing to work with people when they have these problems. So you're saying that it's a problem with the headers? Is there a way to edit or deal with headers?

[Selur]

I'm actually talking to the authors of VRD but they're saying I should talk to tsmuxer authors.

Problem is the tsMuxeR author isn't really available.

So you're saying that it's a problem with the headers?

Yes, my guess is that the VDR folks do something that might be allowed by the H.264/transportstream specification, but which is rather uncommon and thus isn't supported properly by tsMuxeR.

s there a way to edit or deal with headers?

Without extensive knowledge about the H.264 and/or Transportstream specifications and a general idea what the VDR folks did that causes tsMuxeR to fails: Not, really since you wouldn't really know where to look at the stream using a h.264/transportstream analyser.
-> you can try if older/newer tsMuxeR version handle such streams better and ask if the multiAVCHD author can my be write a workaround for (by may be using ffmpeg in some parts of the processing) your problem.

[digitalfreaknyc]

I'm actually talking to the authors of VRD but they're saying I should talk to tsmuxer authors.

Problem is the tsMuxeR author isn't really available.

So you're saying that it's a problem with the headers?

Yes, my guess is that the VDR folks do something that might be allowed by the H.264/transportstream specification, but which is rather uncommon and thus isn't supported properly by tsMuxeR.

s there a way to edit or deal with headers?

Without extensive knowledge about the H.264 and/or Transportstream specifications and a general idea what the VDR folks did that causes tsMuxeR to fails: Not, really since you wouldn't really know where to look at the stream using a h.264/transportstream analyser.
-> you can try if older/newer tsMuxeR version handle such streams better and ask if the multiAVCHD author can my be write a workaround for (by may be using ffmpeg in some parts of the processing) your problem.

We think alike. I always use the most recent version of tsmuxer (which isn't that recent, unfortunately) and I just downloaded the oldest available version of tsmuxer and still had the same problem. The OTHER problem is that multiavchd isn't being developed anymore either. So, basically, my ONLY hope is VRD.

[digitalfreaknyc]

reencode the file with a software which doesn't have a problem with b-pyramids (normally nearly all software should support it) and reencode it with b-pyramid disabled.

I've tried two different software and they BOTH include the jump. Any more suggestions?

[poisondeathray]

If that is the case what made you think b-pyramid was the problem in the first place ?

[digitalfreaknyc]

If that is the case what made you think b-pyramid was the problem in the first place ?

Because that's what VRD developers are telling me:

it is totally tsmuxers fault. It's screwing up the time codes in it's output. I don't know why the VRD cuts are triggering it to do that, but it shouldn't. Let me explain what exactly is happening...

In VRD when you make a cut what we do is we recode from the cut point to the next IDR frame in the source. At that point we stop and start outputting the original frames. In your file there are 3 B frames that come before the IDR that actually belong to the IDR GOP. So we recode up to the P frame before that and stop, then output the rest of the original video starting with those 3 B frames. When you remux with tsmuxer it's changing the time codes on those 3 B frames so they overlap with the recoded frames which is what creates the skip. But the frames are actually still there.

The only thing I can think of that could cause this is that when we recode those few seconds of video at the start of the cut there is a chance that the recoded video will not match the original video's parameters exactly because the encoder we use does not expose every param in the H.264 spec. We attempt to match them as closely as possible, but it's not always possible for them to be exact. The spec allows for encoding param changes at IDR frames, so this should not be an issue, but it's possible tsmuxer is handling the transition incorrectly and that's why it's screwing up the time codes. Unfortunately there is nothing we can do to fix that. We get as close as we can with our encoder, and the spec allows this sort of change, so this is still a tsmuxer issue.

However, it turns out that EVERY multiplexer results in the same issue: tsmuxer, tsremux and tmpgenc. I have tried all 3 and get the same skip.

[poisondeathray]

I'm assuming you re-encoded, then used videoredo ?

If you were going to re-encode the entire thing ( with or without b-pyramid) it would work without videoredo . My point is, if you 're going to incur a re-encode, you might as well do it correctly in the first place

Videoredo is nice software, but this is obviously related to the edits you're making

So it' s not EVERY multiplexer, because the ones you listed are not necessarily blu-ray compliant . There are different levels of "strictness" for multiplexers . The ones you listed will not pass BD verification (they will happily make some BD's that may play on some players - but those BD's are not spec compliant according to Sony's verifier)

[digitalfreaknyc]

I'm assuming you re-encoded, then used videoredo ?

If you were going to re-encode the entire thing ( with or without b-pyramid) it would work without videoredo . My point is, if you 're going to incur a re-encode, you might as well do it correctly in the first place

Videoredo is nice software, but this is obviously related to the edits you're making

So it' s not EVERY multiplexer, because the ones you listed are not necessarily blu-ray compliant

Which encoders would you recommend? I tried TMPGENC VMW 5 and 6 and they both see the jump.

[poisondeathray]

Clarify what you did first . Did you take the "damaged" video and re-encode that ? Or did you take the original, original video and re-encode that ?

As soon as you touch videoredo, you're going to get a problem, so the key is to make the edits in the lossless domain before re-encoding . You don't edit after; you edit before you encode

The whole point of videoredo is so you DONT have to re-encode, but if you're going to re-encode because of these problems, then you might as well skip videoredo

[digitalfreaknyc]

That isn't an option. If I didn't use videoredo, I probably wouldn't have this issue, nor would I need to re-encode.

[poisondeathray]

That isn't an option. If I didn't use videoredo, I probably wouldn't have this issue, nor would I need to re-encode.

You lost me...

Didn't you re-encode with b-pyramid disabled ? And you reported the problem was still there ? Thus the logical conclusion is it's not related to b-pyramid ....

Do you understand what I'm saying? If you were going to resort to re-encoding, then the "option" is don't use videoredo for this... You do your edits in the uncompressed domain (e.g. with a video editor, avisynth, etc...). Then you re-encode that with blu-ray compliant settings . That will work 100% .

[digitalfreaknyc]

That isn't an option. If I didn't use videoredo, I probably wouldn't have this issue, nor would I need to re-encode.

You lost me...

Didn't you re-encode with b-pyramid disabled ? And you reported the problem was still there ? Thus the logical conclusion is it's not related to b-pyramid ....

Do you understand what I'm saying? If you were going to resort to re-encoding, then the "option" is don't use videoredo for this... You do your edits in the uncompressed domain (e.g. with a video editor, avisynth, etc...). Then you re-encode that with blu-ray compliant settings . That will work 100% .

What I'm saying is that when I open the file post-VRD in any re-authoring program that I own, it is missing the 3 frames (aka the jump is still there). Therefore, I wouldn't be able to re-encode because it's not reading the file correctly.

To your other point, I no longer have the original video files. After I edited, I deleted the original, figuring I wouldn't need them anymore since everything looked fine. This has been a problem for YEARS. it's not something new, unfortunately.

However, I know that the edited file, while damaged for playback, is not damaged for good because I can still play it back properly with Splash. That's why I'm hoping that someone more knowledgeable than I am will know what to do with it and somehow save my files. At this point, I'm willing to offer $$.

I've messaged Pegasys, since they're the only encoder whom I've purchased a program from (as opposed to the other freebies) who might be willing to look into this problem. I know that there must be a solution somewhere...but I just don't know where.

I have the file uploaded, if anyone wants to take a look at it.

[poisondeathray]

What I'm saying is that when I open the file post-VRD in any re-authoring program that I own, it is missing the 3 frames (aka the jump is still there). Therefore, I wouldn't be able to re-encode because it's not reading the file correctly.

To your other point, I no longer have the original video files. After I edited, I deleted the original, figuring I wouldn't need them anymore since everything looked fine. This has been a problem for YEARS. it's not something new, unfortunately.

However, I know that the original file, while damaged for playback, is not damaged for good because I can still play it back properly with Splash. That's why I'm hoping that someone more knowledgeable than I am will know what to do with it and somehow save my files. At this point, I'm willing to offer $$.

I've messaged Pegasys, since they're the only encoder whom I've purchased a program from (as opposed to the other freebies) who might be willing to look into this problem. I know that there must be a solution somewhere...but I just don't know where.

I have the file uploaded, if anyone wants to take a look at it.

Ok, now I get it

Hard lesson to learn...but you should always keep the originals , especially if they are semi-important

Now when you say "However, I know that the original file, while damaged for playback, is not damaged for good..." - that refers to the POST videoredo file, correct ? When you say "original file" - It's not the ORIGINAL, ORGINAL, file. You have to be more specific in your replies

So for that post videoredo file - do all software players fail, except splash ? Which others ones have you tried ?

If it's a small file, I can take a look at it. Or cut a section around it (not with videoredo ) and upload it

[digitalfreaknyc]

What I'm saying is that when I open the file post-VRD in any re-authoring program that I own, it is missing the 3 frames (aka the jump is still there). Therefore, I wouldn't be able to re-encode because it's not reading the file correctly.

To your other point, I no longer have the original video files. After I edited, I deleted the original, figuring I wouldn't need them anymore since everything looked fine. This has been a problem for YEARS. it's not something new, unfortunately.

However, I know that the original file, while damaged for playback, is not damaged for good because I can still play it back properly with Splash. That's why I'm hoping that someone more knowledgeable than I am will know what to do with it and somehow save my files. At this point, I'm willing to offer $$.

I've messaged Pegasys, since they're the only encoder whom I've purchased a program from (as opposed to the other freebies) who might be willing to look into this problem. I know that there must be a solution somewhere...but I just don't know where.

I have the file uploaded, if anyone wants to take a look at it.

Ok, now I get it

Hard lesson to learn...but you should always keep the originals , especially if they are semi-important

Now when you say "However, I know that the original file, while damaged for playback, is not damaged for good..." - that refers to the POST videoredo file, correct ? When you say "original file" - It's not the ORIGINAL, ORGINAL, file. You have to be more specific in your replies

So for that post videoredo file - do all software players fail, except splash ? Which others ones have you tried ?

If it's a small file, I can take a look at it. Or cut a section around it (not with videoredo ) and upload it

You're a quick responder! Yeah I changed the words "original file" to "edited file." So, correct= the post-VRD file.

I'm more worried about the post-VRD files. Otherwise, I can just re-encode everything without a problem. It's only once it's been edited that it's an issue. If you'd like to take a look at what a pre-VRD and post-VRD look like, have a go. They're small. However, I will say that they are not of the same thing. They're two different recordings.

Pre-VRD (Raw Hauppauge):
https://mega.nz/#!30dyTKYT!q0xz8QCvXp4Y5LvsdhPN5TW_xVesQrGne7cyCxA3kXM

Post-VRD (but PRE-muxing):
https://mega.nz/#!n91jzJ4b!vwgB1JloSMZ49wk-q51tak84NdHUwYw8Ye_4xRVMr80

[digitalfreaknyc]

Oh...and to answer your other question: I've only tried VLC and splash since they're the only ones I use and are the most reliable (plus the various encoders).

With regards to the post-VRD, if you run it through any muxer, you'll see the jump about 2-3 seconds in when the DJ is clapping.

[poisondeathray]

Yes, there are some problems with that video .

I see jumps in some players only

I don't see a jump when the DJ is clapping, if you re-encode it using directshow/avisynth with any player. I think splash is a directshow player, so that probably is a good way to re-encode if it seems immune to the problems

I do see a jump and hear a jump in Ellen's opening dialog. If you demux the audio and play audio only you will hear glitch and jump in that part - is this what you were really referring to? The DJ isn't the problem, this is the more serious problem because it will cause A/V sync issues depending on the method of processing

Do you have another sample, maybe slightly longer ?

[digitalfreaknyc]

Yes, there are some problems with that video .

I see jumps in some players only

I don't see a jump when the DJ is clapping, if you re-encode it using directshow/avisynth with any player. I think splash is a directshow player, so that probably is a good way to re-encode if it seems immune to the problems

I do see a jump and hear a jump in Ellen's opening dialog. If you demux the audio and play audio only you will hear glitch and jump in that part - is this what you were really referring to? The DJ isn't the problem, this is the more serious problem because it will cause A/V sync issues depending on the method of processing

Do you have another sample, maybe slightly longer ?

Wait...did you watch the entire clip? It's not one real clip. VRD told me to take the longer clip that I was having problems with and cut out the middle...so it's not supposed to sound proper.

Also, did you remux it before you watched it? If you remux it with tsmuxer, you'll see the 3 frames missing when the DJ claps. It suddenly jumps.

[poisondeathray]

Ok that makes more sense that you edited out a middle section. For a minute there I thought it was COMPLETELY messed up

Yes, I can replicate the DJ glitch when remuxing with tsmuxer - but only in some players. Also not when re-encoding with directshow. If you re-encode with various other methods, I don't see the DJ glitch, but it will go out of sync probably (clip isn't long enough to tell) - hence the request for another sample.

[digitalfreaknyc]

Ok that makes more sense that you edited out a middle section.

Yes, I can replicate the glitch when remuxing with tsmuxer - but only in some players. Also not when re-encoding with directshow. If you re-encode with various other methods, I don't see the DJ glitch, but it will go out of sync probably (clip isn't long enough to tell) - hence the request for another sample.

Interesting. I'm loving you this evening. THANK YOU for helping me out with this!

So are all the players that don't have a problem using directshow?

Unfortunately, this particular file (in its original unedited-yet-post-edited form) is over 2 gigs. I have smaller files with the jump, though. I will PM you.

[poisondeathray]

Yes, it seems that directshow is able to avoid the problem in that DJ section , either with ffdshow or lav

I don't need a large file, just a slightly longer clip to test sync, maybe another clip that doesn't have another section cut out. That is actually what I'd be more worried about - sync issues.

[poisondeathray]

And the VRD explanation is not quite correct - the "i" frame near the end of the DJ part is a non key frame (non IDR, or "i" (small case) frame) . It's the open GOP that might cause problems and headaches with some programs / players /decoders . Many BD players will choke on this anyways so you probably want to re-encode (the short version of the explantion: open gop is allowed in BD, but only a specific kind of open gop that is dependent on GOP length defined by on coded order, not display order). The non IDR frames "i" frames are present in the native HDPVR sample as well. That explains why so many people seem to have problems with them in various editing softwares

[digitalfreaknyc]

And the VRD explanation is not quite correct - the "i" frame near the end of the DJ part is a non key frame (non IDR, or "i" (small case) frame) . It's the open GOP that might cause problems and headaches with some programs / players /decoders . Many BD players will choke on this anyways so you probably want to re-encode (the short version of the explantion: open gop is allowed in BD, but only a specific kind of open gop that is dependent on GOP length defined by on coded order, not display order). The non IDR frames "i" frames are present in the native HDPVR sample as well. That explains why so many people seem to have problems with them in various editing softwares

Well, neither clip that I posted is remotely what I'd author to a finished disc anyway. It's really just a test. The only problem I'm having is with that damn jump/skipping of 3 frames. The other weird thing is that I don't hear a skip in the audio so I'm wondering if there's a sync problem in everything as well. I've pm-ed you.