Need to cut out video sections at segment borders, how?

[BosseB]

I am downloading streaming video on a Linux Ubuntu 20.04.6 LTS server.
This is done using a bash script calling ffmpeg, the script is executed from an at schedule.

The script re-encodes (might not be the correct word) the video format on the fly so I get a 480p mp4 file.

Later I want to edit the video by extracting sections from it and pasting them together into a shorter video.
To do this I use a player utility I have written which plays the video and I can set cut points at integer second positions.
These are then converted to a series of ffmpeg commands to extract parts of the video using the start time and duration (in seconds) of each section. This gives me a number of short files.
Then the short files are combined into a single one using the concat filter in a final ffmpeg command.

The problem I am facing is that when playing the end result it has trouble doing a smooth transition between the shorter sections.
Sometimes a rather annoying pause in the video is shown or else a part of the cutaway video is still shown with audio misplaced.
Sometimes all works OK and sometimes the problem persists for a few seconds.

I have deduced that the issue is caused by the way the cut times are located with respect to the video content, in particular the segment borders in the video file...

The ffmpeg commands I use in the various steps are like this:

1. The download command from the stream:

ffmpeg -hide_banner -referer \"${VIDEOURL}\" -i \"${M3U8URL}\" -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -c:a aac -t 3600 output.mp4

2. The command that creates the extracted video file based on start and duration times in seconds (x is a sequence number):
Here the start time is 138 seconds and the duration is 519 seconds:

ffmpeg -hide_banner -i inputfile.mp4 -ss 138 -t 519 -c:v copy -c:a copy cut_x.mp4

3. The command that pastes together the different cuts uses a text file "joinfile" containing a list of the individual files:

file cut_1.mp4
file cut_2.mp4
...
file cut_n.mp4

4. The command used to paste together the cuts itself:

ffmpeg -hide_banner -f concat -i joinfile -c copy finaloutputfile.mp4

I have understood by reading various webpages that my problem is that the cut points do not correspond to a border between two video "segments" inside the file, so the first part of each video cut is a truncated part of the previous segment.
But I have found no way to fix this...

So my question is if there is a way to:

A) adjust the download script such that it encodes the output video to have the segment borders on integer second times?

or

B) Adjust the set cut times to coincide with the closest segment start time inside the source video, essentially going from integer seconds to floating point seconds.

or

C) On creation of the cut videos make sure they start on a segment border inside the source video

I have downloaded this documentation pdf file where I have found a chapter that deals with splitting a video into several segments, but unfortunately it is a Windows batch script, which I cannot translate to the Linux bash scripts I use...
The text in the pdf looks like this:

2.59 Split a video in multiple segments
A video can be split in multiple segments with the segment muxer. All segments will have the same length, except the last one.

set "IN=my_video.mov"    :: Input video
set "L=10"               :: Segment length in seconds
seu "OUT=out%%2d.mov"    :: Output filename
ffmpeg -i %IN% -f segment -segment_time %L% -c copy %OUT%
pause

This batch fill extracts a segment with known start and end frame numbers:

set "start=100"       :: First frame number 
set "end=200"         :: Last frame number 
set /a startms=%start%*1001/30    :: This calculation is for framerate 30000/1001 = 29.97
set /a endms=(%end%+1)*1001/30    :: Note that in the batch file only integer arithmetic is possible!
                                  :: It's important to do first the multiplication and then the division
ffmpeg -i in.mp4 -ss %startms%ms -to %endms%ms -c copy -y out.mp4
pause

Note: The above command line with "-c copy" works only for intraframe codecs, meaning that all frames are I-frames. For interframe codecs you must
remove "-c copy", but then then the video will be re-encoded and the process is much slower.

I wanted to test the commands from the pdf document but got stuck with the strange Winows batch format...

Anyway, my main questions above remain...

PS:
I have found a workaround by extracting and combining the cuts in a single ffmpeg command but it re-encodes the content and it takes way too long to do.
But the result is a video with smooth transitions.
DS

[butterw]

follow-up from: https://forum.videohelp.com/threads/406241-Cutting-away-sections-from-a-video-using-ff...hout-re-encode
If you don't want to re-encode you have to figure out where the keyframes are. IMO if the source has been encoded with fixed duration keyframes (ex: 2 seconds) it will be easier to find them.
ffmpeg concat options also need to be correct, check previous threads on the subject.

If reencoding is more practical, fixed-function hardware encoding on modern gpu (or quicksync on intel igpu) is faster/more energy efficient than cpu encoding.

[BosseB]

If you don't want to re-encode you have to figure out where the keyframes are. IMO if the source has been encoded with fixed duration keyframes (ex: 2 seconds) it will be easier to find them.

There is one place where re-encoding could fit and that is during stream download since it is re-encoding then anyway and there is plenty of time at that point.
I showed the download script command above.

How can it be modified so that the encoded output is also using fixed duration keyframes?

If that could be done then I believe the whole issue will be solved.
What are "normal" keyframe durations? You suggest 2 s but what would happen if I use 1 s instead? Will the video file output expand in size?

Given that the editor's cut commands are on 1 s intervals then if the downloaded video is set to 1 s intervals then there will always be a start on each cut point, right?

EDIT
In the pdf document I searched for the term "keyframe" and found in section 4.2 page 561 the following:

4.2 Find the keyframe timestamps
ffprobe -select_streams V:0 -show_frames -skip_frame nokey -show_entries frame=best_effort_timestamp_time input.mp4

So I made a short 600 seconds long video download to test on (from he same source I use).

Then I ran the command above with the downloaded video as input piping the output into a file, and got one looking like this:

[FRAME]
best_effort_timestamp_time=0.000000
[SIDE_DATA]
side_data_type=H.26[45] User Data Unregistered SEI message
[/SIDE_DATA]
[/FRAME]
[FRAME]
best_effort_timestamp_time=2.135467
[/FRAME]
[FRAME]
best_effort_timestamp_time=4.137467
[/FRAME]
... a total of 185 such FRAME 3-line items...
[FRAME]
best_effort_timestamp_time=599.632367
[/FRAME]

So in a 600 second video there are 185 frames each then occupying 3.24324324... seconds.

QUESTION:
How can I modify this download script command to force it to use 1 second keyframes?
That is given the command looks like this:

ffmpeg -hide_banner -referer \"${VIDEOURL}\" -i \"${M3U8URL}\" -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -c:a aac -t 3600 output.mp4

EDIT 2:
I also used the command in section 4.1 to count the number of frames. Turns out that it goes away for 17 seconds, and then outputs the data:

ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 input.mp4
17983

17983/600 = 29.9718888
So the video frame rate seems to be a hair slower than 30 fps.

I would like the video to become 30 fps and with keyframes every 1 second , i.e. 30 frames between each keyframe.
Is that possible and how can I do that again given the download command shown above?

[mrbass]

Get familiar with how LosslessCut does it as you can have it even show you the ffmpeg command it uses then apply that to your script.

[BosseB]

Get familiar with how LosslessCut does it as you can have it even show you the ffmpeg command it uses then apply that to your script.

Thanks,
I have just downloaded the LosslessCut program to my Windows 10 laptop after reading through some of the info on the VideoHelp and GitHub pages (there is a lot...).

A quick question:
In my own video editor one of the most often used functions is a skip where I have 5 forward and 5 backward skip buttons with fixed times and also two with configurable times. I use them to skip a known time forward or backward since I know the most often used times of ads etc.
Is there some kind of LosslessCut function like this?

Also in order to pause the video I use a click anywhere in the video screen to freeze right there, then use the small increment move to adjust to the exact position of the cut.
Is something like that available in LosslessCut as well?
Using the small pause button makes it less precise because one must put the mouse in that small space...
I can't see it but it might need to be enabled/configured? Or is not there....

Anyway, thanks, I will have to look into it more so I can run it.

One caveat:
All of my videos live on my Ubuntu Server and I operate on them using ffmpeg scripts which are supplied arguments from my video editor.
And the Linux box is a server with no GUI available, it is just plain command line (via PuTTY SSH).
With LosslessCut it seems like the cut is done by LosslessCut itself (by calling the ffmpeg executable on Windows) so it then needs to do its thing via a Samba share to the video store, right?

[jagabo]

How can it be modified so that the encoded output is also using fixed duration keyframes?

I don't think it will completely fix the problem but you can add

-x264-params keyint=25:scenecut=0:open-gop=0

to your download/encoding command line. Use a keyint value that matches the frame rate of your video (25 fps in this example). scenecut=0 disables keyframes at scene cuts (really shot cuts). I believe open-gop defaults to 0 (closed GOPs) but it doesn't hurt to specify it here.

What are "normal" keyframe durations? You suggest 2 s but what would happen if I use 1 s instead? Will the video file output expand in size?

The default GOP size in x264 is 250 frames. Broadcast TV is more like 2 seconds. DVD is about 1/2 second. Since you're using bitrate based encoding, very short GOPs will keep the same size but reduce picture quality. If you were to use crf encoding you would get the same quality but larger files.

Given that the editor's cut commands are on 1 s intervals then if the downloaded video is set to 1 s intervals then there will always be a start on each cut point, right?

You have to make sure that scenecut is also disabled. Otherwise a new GOP will start whenever there's a shot change. min-keyint defaults to 23 frames but is automatically set to about 1/10 the keyint value unless you specify otherwise.

[BosseB]

Get familiar with how LosslessCut does it as you can have it even show you the ffmpeg command it uses then apply that to your script.

Thanks,
I have just downloaded the LosslessCut program to my Windows 10 laptop after reading through some of the info on the VideoHelp and GitHub pages (there is a lot...).

First test shows:
Using a 1 hour video downloaded from a news outlet with commercials.

I set the boundaries of 4 cuts such that it was on black frames before and after the cut itself.
Then exported it to the combined output file.

When I check the output file it turns out that all cut points contain artifacts from the commercial I tried to cut away...
So it does not solve the problem I am trying to address.

Notice:
The test video is the kind that is not quite running at 30 fps so this might be the problem also here.
Could it be solved by ffmpeg adjusting the frame rate while downloading?

[butterw]

HLS video streaming sites use a .m3u8 playlist of .ts files. Typically the .ts fragments are about 2-4s long, but one second would work. The source files would need to be converted to an integer framerate (ex: 30fps).

[BosseB]

HLS video streaming sites use a .m3u8 playlist of .ts files. Typically the .ts fragments are about 2-4s long, but one second would work. The source files would need to be converted to an integer frame rate (ex: 30fps).

My question is really:
Is it possible to configure the ffmpeg download command (shown above in 2 places) such that it will convert the frame rate to 30 fps (slightly faster than the stream) or if that is not possible to 25 fps, which is slower than the stream?
Then it should be possible too to put 30 or 25 such frames in a segment resulting in 1 second segments. Again: with what arguments?

But in any case I am not knowledgeable enough to figure out the required ffmpeg arguments that would make this happen on download.
So I need help from a forum member....

[jagabo]

-r 30 will convert the video to 30 fps by duplicating 1 frame out of every 1000 (29.97 fps to 30.00 fps) -- retaining the original run time. Combined with what I gave you earlier, -x264-params keyint=30:scenecut=0:open-gop=0, will give you a 30 fps file with a keyframe every second, 30 frames. So the full command line would be:

ffmpeg -hide_banner -referer \"${VIDEOURL}\" -i \"${M3U8URL}\" -r 30 -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -x264-params keyint=30:scenecut=0:open-gop=0 -c:a aac -t 3600 output.mp4

But even that won't work properly. There is a bug in ffmpeg (in copy mode) where you need to specify about two frames before the keyframe to get a cut that starts at a keyframe. Say you know the segment you want starts at 900 seconds. And you know there's a keyframe at 900 seconds. You have to specify a -ss time two frames earlier.

[BosseB]

-r 30 will convert the video to 30 fps by duplicating 1 frame out of every 1000 (29.97 fps to 30.00 fps) -- retaining the original run time. Combined with what I gave you earlier,

-x264-params keyint=30:scenecut=0:open-gop=0

will give you a 30 fps file with a keyframe every second, 30 frames. So the full command line would be:

ffmpeg -hide_banner -referer \"${VIDEOURL}\" -i \"${M3U8URL}\" -r 30 -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -x264-params keyint=30:scenecut=0:open-gop=0 -c:a aac -t 3600 output.mp4

But even that won't work properly. There is a bug in ffmpeg (in copy mode) where you need to specify about two frames before the keyframe to get a cut that starts at a keyframe. Say you know the segment you want starts at 900 seconds. And you know there's a keyframe at 900 seconds. You have to specify a -ss time two frames earlier.

OK thanks, I see....

Since the cut commands from the editor are at 1 s resolution and with the new download coding there will be keyframes at even seconds, I assume that by making the cut points a floating point number and subtracting 2/30 from the wanted start point will do the trick.
So the -ss param example would become 900 - 2/30 = 899.933

I have done some manual testing after running a download using the modified script code and the conclusion is:
1) The download produces a video with 1.0 s frames (checked)
2) If I cut this using the whole second values from the editor I get transition artifacts (as you stated)
3) If I cut using the values but after subtracting 1/15 from them I get the wanted behavior without artifacts (checked)

So now I have modified the download script to use the settings to make a 30 fps video with frames at 1s intervals.
I also need to reprogram my video editor to subtract the 1/15 seconds from the start times of the cuts.
This will take a bit longer since it involves bringing up my Ubuntu programming environment on a laptop normally not used.

THANKS FOR YOUR ASSISTANCE!
Much appreciated!

[jagabo]

One other thing I found: If you encode without b-frames you can specify the exact time of the I frame -- no need to calculate two frames earlier. Of course, encoding without b-frames doesn't get as good compression. So that's not an ideal solution.

[BosseB]

So now I have modified the download script to use the settings to make a 30 fps video with frames at 1s intervals.
I also need to reprogram my video editor to subtract the 1/15 seconds from the start times of the cuts.
This will take a bit longer since it involves bringing up my Ubuntu programming environment on a laptop normally not used.

Did not take that long, I found a key place where I could do the change on two lines of code and I am now all done!
The code already had the cut as an integer so I could decrement it and then when converting to the command string just paste ".933" at the end.

Works like magic compared to the re-encode I had to do previously.
Extracting 10 cuts and pasting them together in 3-4 seconds rather than 8 minutes!