How to rearrange I,B,P frames in a given video ?

[db2t4]

Hi All,

I am a newbie in video and video frames , Hence this question goes to all the video experts :

I have a .mp4 file and i was able to successfully see the arrangement of (I,B,P) frames in a given video using ffprobe tool . Now my question is ..... if my original video has the following frames arranged ( please consider the numbers as just a reference i want to use) for example : {I0B0B1P0B2B3P1B4B5I1} is it possible for me to arrange the frames in the following manner : {I0P0P1I1B0B1B2B3B4B5} ? if so could anyone guide me on how do i proceed ? and is there any tool i can use to do so ?

As a newbie i am not sure where to look into for the right resources.

Thanks in advance for the help provided!

[El Heggunte]

As a newbie i am not sure where to look into for the right resources.

You may start by reading this: https://en.wikipedia.org/wiki/Video_compression_picture_types

Now my question is ..... if my original video has the following frames arranged ( please consider the numbers as just a reference i want to use) for example : {I0B0B1P0B2B3P1B4B5I1} is it possible for me to arrange the frames in the following manner : {I0P0P1I1B0B1B2B3B4B5} ?

But WHY would you want to do that

I suppose that it's not impossible to do it --- BUT the job would have to be incredibly well-done,
in order to NOT confuse (or even worse, crash) the decoders

[Cornucopia]

Each frame has a delivery read-order timestamp and a presentation timestamp. That's because with IBP type pictures, the B picture type can reference a subsequent I or P frame. But a decoder can't know ahead of time what those reference frames are going to be like, so that B frame cannot be fully decoded until that subsequent reference frame is decoded first.
That is one reason why there is a frame-delay buffer.
However, if you were to willy-nilly start changing around the read order, you would put a burden on the buffer...possibly to the point of underflow/overflow and breaking it. Don't do it.

Scott

[db2t4]

@El Heggunte and Cornucopia :

First of all thank you for answering my doubts .

The reason i need to reorder the frame is to do some research on what is the impact on temporal quality of the video when i reorder the frames and send it .
Also, how do i recognize read-order timestamp and a presentation timestamp in a given sample of frame i have obtained from ffprobe output?
And recently while i was researching i came across something about Reference B frame which can reference an I frame or a P frame ...my question here is how do i recognize which of the B frames is a reference B frame in ffprobe ouput since right now my output shows pict_type = I/B/P ?

Also is there some command in ffmpeg i can use for rearranging the frames ?

Output of a single frame i have obtained from ffprobe:

[FRAME]
media_type=video
stream_index=0
key_frame=0
pkt_pts=153088
pkt_pts_time=11.960000
pkt_dts=153088
pkt_dts_time=11.960000
best_effort_timestamp=153088
best_effort_timestamp_time=11.960000
pkt_duration=512
pkt_duration_time=0.040000
pkt_pos=760800
pkt_size=523
width=352
height=288
pix_fmt=yuv420p
sample_aspect_ratio=1:1
pict_type=B
coded_picture_number=299
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
[/FRAME]

( reference link : http://www.streaminglearningcenter.com/articles/producing-h264-video-for-flash-an-over...ew.html?page=4)

[pandy]

DTS is Decoding Time Stamp and it tels when frame should be decoded (order of frames in bitstream)
PTS is Presentation Time Stamp and it tells decoder when decoded frame need to be displayed (as such it keep order decoded frames equal to video).

It will be good if you start reading H.264 standard https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.264-201602-I!!PDF-E&type=items to understand what you trying to do - as you insist to operate at bitstream syntax level you must create something capable to parse bitstream and modify interesting for you fields.

Good Luck.

[poisondeathray]

The reason i need to reorder the frame is to do some research on what is the impact on temporal quality of the video when i reorder the frames and send it .

Re-ordering the frames requires re-encoding. Thus the quality of every frame will decrease if you use lossy encoding from the current video if it is used as the source. If you use lossless encoding , the filesize will increase many times and will not be a "streamable" format suitable for a browser

If you have the original , original source (the one before your current video) , then you might consider looking at better encoding options . It is possible to specify frametype distribution with a qpfile for example. But rarely will the user make "better" decisions over an encoder like x264; the qpfile is only used in specific situations to override the encoder's decisions (e.g. you might want to place IDR seekpoints at specific locations)

[Cornucopia]

Real simple: if you change the order away from what is allowable by the decoder, it breaks and quality goes to shit. What is allowable depends on its buffer size.
IOW, within the allowable range all acts like normal, full quality. Outside the range, you'll get breakups, stutters, freezing, blank screen, garbage.

What do you intend to accomplish?

Scott

[db2t4]

@pandy and posiondeathray : Thank you

So i just want to make sure my understanding is right : pkt_size=523 in the above format is in bits/ bytes ?

Well my professors want me to work on a video segment which is already encoded and wants me to do the rearrangement of frames as a pre-processing part before i can stream it and then he wants to calculate the distortion produced and impact of temporal quality .

Thank you for reference material i will read it .

[pandy]

@pandy and posiondeathray : Thank you

So i just want to make sure my understanding is right : pkt_size=523 in the above format is in bits/ bytes ?

Well my professors want me to work on a video segment which is already encoded and wants me to do the rearrangement of frames as a pre-processing part before i can stream it and then he wants to calculate the distortion produced and impact of temporal quality .

Thank you for reference material i will read it .

Bytes, i can imagine what are you trying to achieve but it may be required to calculate not only DTS but also some other fields - you will touch lowest level of codec and codec syntax understanding is mandatory to succeed.