FFmpeg cutting/joining findings

[Budman1]

There are literally thousands of internet hits Suggesting how to cut videos using FFmpeg and almost all of them have comments afterward that the methods do not work. Since I am working on a utility to cut and join videos at Key Frames and perhaps re-encode cut portions that are not at key frames, I have done extensive testing with the one of the latest FFmpeg FFprobe FFplay versions. I use FFprobe on all the results as well as MediaInfo. They may not be accurate but as far as I know these have worked well in the past.

The only variations I have found is that FFprobe used to represent a cut preceding an I-Frame as a minus number but now starts the cut at 0 (since -ss Input now frame accurate) and show frame 0 as a plus number. Also the 64 bit MediaInfo CLI later versions lost the ability to output some information. Both of these were not a major problem as far as my results showed.

I am sure there will be many comments and I welcome them since there is much that I wish to learn even though I have read and tested as much as I can. I have also tested such as AVIDemux and others and they achieve the approximately the same results as below although the offset is difference in the final cut's frame 0 PTS.

I still do not know if a frame 0 PTS of say 2+ seconds causes a problem when joining.

I used the following video as the main reference but used other formats for testing just to confirm the actions of cutting.

FINDINGS
=============================================
My computer is Windows 7 Hex core Mid speed processors with the latest .NET installed.
Closed GOP 250 29.97 FPS MP4 AVC/264 AAC 318.785452(I-Frame)PBBB----/250/-------327.127127------/250/-----335.468802
=============================================

ffmpeg -ss 327.127127 -i cgop.mp4 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = FFProbe will not retrieve any results
8.342 seconds video plays incomplete/blamk frame only entire 8.342 seconds video

ffmpeg -i cgop.mp4 -ss 327.127127 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = FFProbe reports ptsTime 8.341008 at point of NEXT I-frame at 335.468802
16.580 seconds video plays audio with audio for 16 seconds and freeze frame from
next I-Frame (335.468802) for 8 seconds and then 8 seconds of 335.468802 GOP

ffmpeg -ss 327.093760 -i cgop.mp4 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = The 'P' frame immediately in front of 327.127127 I-Frame as it should and PTS time is 0.000000
8+ seconds video plays audio (undiscernible) and freeze frame video ('P' Frame)for 8+ seconds

ffmpeg -i cgop.mp4 -ss 327.093760 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = 8.375042 The NEXT I-Frame following 327.127127
16+ seconds video plays audio (16 seconds) and video 8 second freeze frame then normal 335.468802 8+ seconds

ffmpeg -ss 327.060394 -i cgop.mp4 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = 0.000000 but is the 2nd P-Frame (PPI) in front of the 327.127127 I-Frame
8+ seconds video plays 2 frames only (not 8 seconds) Eith PP or PI Frames (cannot tell)

================================================== ============
IMPORTANT:

ffmpeg -i cgop.mp4 -ss 327.060394 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = 0.065983 correct GOP for 250 frames 8.342 length audio 8.313 length
8+ seconds video plays correctly with no audio sync or freeze/duplicate frames discernible

THIS video can be set to Frame 0 PTS by running unknown algorithym that resets time: (but is it necessary?)

ffmpeg -ss 0.050016 -i cgop.mp4 -vframes 250 -c copy cgop_ss_result0.mp4

================================================== ============

ffmpeg -ss 326.926927 -i cgop.mp4 -vframes 250 -c copy cgop_ss_result.mp4

Frame 0 = 0.000000 but is the correct 6th frame (preceding the 327.127127 I frame.
FFprobe reports only those 6 frames
8+ seconds video plays 6 frames only (not 8 seconds) PBPPPP (I) not played.

================================================== ============
FURTHER SUMMARY
FFmpeg is supposed to reset PTS Time to zero if used as input and appears to do so
by cutting at the I-Frame following the reset point when used as input option
and causes the Frame 0 to be offset from seek point but the output is stopped after the
I-Frame following the cut point.

FFmpeg -ss option used as output option seems to seek to the NEXT I-Frame after cut point,
offsetting that Frame 0 PTS the amount and then cut for the correct amount of time with:

ffmpeg -i cgop.mp4 -ss 327.060394 -vframes 250 -c copy cgop_ss_result.mp4

or

ffmpeg -i cgop.mp4 -ss 327.060394 -t 8.308383 -c copy cgop_ss_result.mp4 (249*0.033367=8.308383)

No amount of seeking with both input and ouput option results in correct/playable video.

SUMMARY:
using FFmpeg with -ss as output option and either -vframes xxx or -t (frames_requested -1 * PTS duration time)
will give a playable video with frame 0 not at 0.000000. The amount varies by video being cut
and I cannot find what or why it results in different amounts. Joining may be a question since I have not
joined a lot of segments with offset PTS Time to see if sync problems arise.

I usually reset the frame 0 PTS to 0.000000 with my 'silly' method before joining segments
but, again, I usually only join 2-3 segments so no sync problems are detectable by me.

========================================
If anyone's findings are different, or if anyone has an explanation please let me know. As I mentioned, I need all the information I can
get in order to create a working program that addresses the 1000's of comments of FFmpeg cuttting failures.

So far I have been able to create segments that appear to be the correct length and play correctly with no sync problems.?????


[Attachment 46008 - Click to enlarge]
86 + 72 + 76 = 234
0 0.000000 1 I
1 0.042000 0 B
2 0.084000 0 B
3 0.126000 0 B
/---/
229 9.552000 0 B
230 9.593000 0 B
231 9.635000 0 P
232 9.677000 0 B
233 9.719000 0 P

[pandy]

Hello Budman1 - watching very close all your findings and appreciate hard work. Silly question - did you ever tried to modify ffmpeg behaviour by using within your command line '-strict -2' a.k.a. '-strict experimental' ?

[Budman1]

@pandy... Didn't try -strict option because I thought it was an old option for earlier AAC audio. I did however give it a try and the results were the same. Apparently -ss as an input or output works very similar (Well they are both exact now). They start the video at the NEXT Key Frame but give everything up to that point a negative Timestamp (earlier FFprobe versions) or start timestamp at 0 at the cut point so the video starts at a positive number (later FFprobe versions)

In my tests, the best option was to use as an output option 2 frame (more will also work) and the cut will be at the FOLLOWING I-Frame but with a slight positive timestamp of some unknown duration. In my tests one file was 0.65983 while another different size and FPS gave other results such as 0.122784 ???? This is the duration that we all need re-encoded so it can be joined with the least loss during cutting and joining. That's the next project/ LOL

[pandy]

@pandy... Didn't try -strict option because I thought it was an old option for earlier AAC audio.

Strict enable ffmpeg to use some non compliant/untested/experimental options - it affects almost every aspect of ffmpeg processing - for example yuv4pipe can emit broader codecs spectrum at the output.

[Budman1]

Update:
Looks like the leading preroll, even on non-AVC/AAC videos causes delay. Tests below, all done with copy, no re-encoding, were as follows:

1. I used a Gunshot video (MKV) of short duration and joined it to itself for a total of 8 segments. Left the +DTS of Frame 0
2. I converted the MKV to MP4 and joined the MP4 to itself for a total of 8 segments. Left the +DTS of Frame 0
3. I converted the MKV to MPE$, reset the Frame 0 DTS to 0.00000 and truncated the 133 frames to 132 to remove the longer Audio

RESULTS:
1.
Gunshot.mkv= frame 0= 0.003000, 133 frames, Video=vp9= 4.500 seconds, audio =vorbis=4.483 (2 frames delay to start)
8(x)Joined.mkv =frame 0= 0.003000, 1064 frames, Video=vp9= 35.824 seconds, audio =vorbis=35.888
2 frames delay start, 13 frames delay finish


2.
(fFFmpeg -i Gunshot.mkv -c copy Gunshot.mp4)
Gunshot.mp4= frame 0= 0.003000, 133 frames, Video=vp9= 4.433 seconds, audio =vorbis=4.500 (2 frames delay to start)
8(x)Joined.mp4 =frame 0= 0.003000, 1064 frames, Video=vp9= 35.934 seconds, audio =vorbis=36.000
1+ frames delay start, 13+ frames delay finish

Segment 1 (Below) and Segment 8 for attempts 1. and 2.

[Attachment 46040 - Click to enlarge]
[Attachment 46041 - Click to enlarge]


3.
Start with MP4/Set Frame 0 = 0.00000 truncate to 132 frames to remove excess audio (Algorythm-via FFmpeg)
Gunshot.mp4= frame 0= 0.000000, 132 frames, Video=vp9= 4.400 seconds, audio =vorbis=4.381 (1 frames delay to start)
8(x)Joined.mp4 =frame 0= 0.000000, 1056 frames, Video=vp9= 35.201 seconds, audio =vorbis=35.181
less than 1 frames delay start, less than 1 frames delay finish

Segment 1 and segment 8 from attempt with reZero Frame 0 DTS

[Attachment 46042 - Click to enlarge]
[Attachment 46043 - Click to enlarge]

I will attempt other formats and codec to see if the result hold true, but this might account for why many cut/join operations fail with generic FFmpeg scripts.
I have also found a way to interrogate an AVC/264 file for encoding formats CABAC, GOP,etc when MediaInfo does not show this. Hopefully this would allow the re-encoding of cut sections with the same results. Still testing.