Need help writing an ffmpeg script to batch split many video files by size

[manav]

Hi,
I am pretty noob with ffmpeg. I have more than a thousand .flv files, the size of each of which varies from 1GB to 11GB, downloaded using Streamlink. I was wondering if anyone here can help me write a .bat script that will split all of those video files into parts/pieces, each of which should not be more than 500 MB, without re-encoding, and keeping the original filenames with just a suffix added (part1, part2...). It would be better if the script can also delete the original files after processing them, though, not necessary.

For example, a 4000 MB file, named "example.flv", after splitting, results in 8 files, named "example_part1.flv", "example_part2.flv", and so on, each of which is 500 MB.

Below are the details of one of the videos, using MediaInfo:

General
ID                                       : 1 (0x1)
Complete name                            : G:\Streamlink\102906.flv
Format                                   : MPEG-TS
File size                                : 4.61 GiB
Duration                                 : 1 h 28 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 7 437 kb/s
FileExtension_Invalid                    : ts m2t m2s m4t m4s tmf ts tp trp ty

Video
ID                                       : 256 (0x100)
Menu ID                                  : 1 (0x1)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.2
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 1 frame
Format settings, GOP                     : M=1, N=30
Codec ID                                 : 27
Duration                                 : 1 h 28 min
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 60.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive

Audio
ID                                       : 257 (0x101)
Menu ID                                  : 1 (0x1)
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format version                           : Version 4
Format profile                           : LC
Muxing mode                              : ADTS
Codec ID                                 : 15
Duration                                 : 1 h 28 min
Bit rate mode                            : Variable
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -5 ms

All the videos have the same format/codec but varying frame rates and resolutions. Any help would be greatly appreciated.

[butterw]

I don't know anything about flv, but it looks like your files could be put in a mp4 container ?

Try muxing one of your files to mp4 to see if it plays ok (or mkv with mkvtoolnix if it doesn't work).

Also how do you want to cut, as playable clips ?

[manav]

I don't know anything about flv, but it looks like your files could be put in a mp4 container ?

Try muxing one of your files to mp4 to see if it plays ok (or mkv with mkvtoolnix if it doesn't work).

Hello,
Thanks for the response.
I used avidemux to open up one of the small clips. I set the Video Output and Audio Output to "Copy" and changed the Output Format to "MP4 muxer" and saved it, which took less than a second. There doesn't seem to be any problem with the playback. The duration of both the files is the same, however, the mp4 file has a reduced file size of 94.6 MB, compared to the 97.9 MB on the flv.

Also how do you want to cut, as playable clips ?

I didn't clearly understand your question. I just wanted to split all of the videos into parts of not more than 500 MB. And all parts should be separate clips that can be playable independently. Also, I want all of the videos to process in one go, as there are way too many of them. I found a similar thread on a different forum which probably has a solution to my problem, but I, being an ffmpeg newbie, was unable to figure out how to modify those commands/script to my requirement. That thread can be found here.
Thanks

[butterw]

What's your OS ?

What you want is as follows:
- input: 4000 MB file, named "example.flv"
- output: "example_part1.mp4", "example_part2.mp4", and so on, each of which is <=500 MB.

1. The operation will be lossless (no recompression required, just remux)
2. For the clips to be playable you must cut on keyframes which is the default with ffmpeg.
3. Cutting based on duration may however be more common with ffmpeg ?

[manav]

What's your OS ?

I am mainly on Windows 10, and also have access to macOS High Sierra (thanks to the Hackintosh community).

What you want is as follows:
- input: 4000 MB file, named "example.flv"
- output: "example_part1.mp4", "example_part2.mp4", and so on, each of which is <=500 MB.

Yes, that is what I want.

2. For the clips to be playable you must cut on keyframes which is the default with ffmpeg.

I have no problems with that as long as the files are at most 512 MB in size, which is why I have kept 500 MB the target.

3. Cutting based on duration may however be more common with ffmpeg ?

The problem is that there are more than a thousand videos, each of which vary heavily in duration and size, and every 1 out of 8 clips vary in frame rate, resolution and bitrate. All I want is to split them up in parts of less than 500 MB.

[butterw]

- Windows batch file is .bat.

- I have not split based on filesize with ffmpeg. Someone here might know how to do it, or not.

- As a backup solution, mkvtoolnix does have an option to split on filesize. mkvmerge is the command line tool.
The output is mkv, but you could transform mkv to any format with ffmpeg.

- take your time to do the conversion and check the result before you start deleting.
Most of your files might convert fine, but you could still have issues with others if the format is different for any reason.

[abolibibelot]

Did you try something like this ?

FOR %F in (*.flv) DO mp4box -add "%F" -split-size 512000 -new "%~nF.mp4"

With mkvmerge if I'm not mistaken it should be something like :

FOR %F in (*.flv) DO mkvmerge --output "%~nF.mkv" "%F" --split size:500M

These commands should work from the command prompt (cmd). To run them as batch scripts (.bat) each "percent" sign (%) must be doubled (%%).

As it's been mentioned those files are not truly "FLV", but MPEG-TS.

The duration of both the files is the same, however, the mp4 file has a reduced file size of 94.6 MB, compared to the 97.9 MB on the flv.

It is normal that the size is slightly reduced when converting from MPEG-TS to a container with less "overhead" like MP4 or MKV (a MKV is usually slightly smaller than MP4 with the same content).

[manav]

- Windows batch file is .bat.

- I have not split based on filesize with ffmpeg. Someone here might know how to do it, or not.

- As a backup solution, mkvtoolnix does have an option to split on filesize. mkvmerge is the command line tool.
The output is mkv, but you could transform mkv to any format with ffmpeg.

- take your time to do the conversion and check the result before you start deleting.
Most of your files might convert fine, but you could still have issues with others if the format is different for any reason.

No problem. Thanks for your precious time and all of the information you've provided.

Did you try something like this ?

FOR %F in (*.flv) DO mp4box -add "%F" -split-size 512000 -new "%~nF.mp4"

With mkvmerge if I'm not mistaken it should be something like :

FOR %F in (*.flv) DO mkvmerge --output "%~nF.mkv" "%F" --split size:500M

These commands should work from the command prompt (cmd). To run them as batch scripts (.bat) each "percent" sign (%) must be doubled (%%).

As it's been mentioned those files are not truly "FLV", but MPEG-TS.

The duration of both the files is the same, however, the mp4 file has a reduced file size of 94.6 MB, compared to the 97.9 MB on the flv.

It is normal that the size is slightly reduced when converting from MPEG-TS to a container with less "overhead" like MP4 or MKV (a MKV is usually slightly smaller than MP4 with the same content).

Thanks, I'll test these commands and report back in a while.

[manav]

Okay, so here are the results. Tested with a single flv in the folder.
Input file size: 4.61GB

With

FOR %F in (*.flv) DO mp4box -add "%F" -split-size 512000 -new "%~nF.mp4"

in ffmpeg
Time taken: Around 15 minutes
Output: A single mp4 file, the size of which was 4.46 GB, with "001" added as suffix.
CMD window Screenshots:
While processing: https://imgur.com/sNht0xh
The last line which is something like

Import: |==============| (100/100)

kept blinking on the screen for 15 minutes.
After Processing: https://imgur.com/AZSCKY4
Conclusion: Didn't work as expected.

With

FOR %F in (*.flv) DO mkvmerge --output "%~nF.mkv" "%F" --split size:500M

in mkvmerge/mkvtoolnix
Time taken: About 2 minutes
Output: 10 mkv files, with the last one being 73.2 MB in size, and the other 9 being 500 MB. Total size of all the files was 4.46 GB, same as the previous mp4 output with ffmpeg which was just a single file.
CMD window Screenshot after processing:https://imgur.com/ZTK0CfR
Conclusion: Working perfectly as I wanted.

I later tested out the mkvmerge command with multiple flv's in the folder, and it worked perfectly. Thanks a ton, abolibibelot. You're a legend!

[abolibibelot]

in ffmpeg

same as the previous mp4 output with ffmpeg which was just a single file

You mean, mp4box. It may be possible to do this with ffmpeg but I don't know how.

Conclusion: Didn't work as expected.

Not sure, mp4box often has trouble dealing with input files which are not already in MP4 container. I just tested with a MP4 file, it worked as intended (with a 435MB input file and -split-size 51200 it generated 8 files with sizes ranging from 50400KB to 50784KB and a 9th with a size of 40932KB).

in mkvmerge/mkvtoolnix
...
Conclusion: Working perfectly as I wanted.
...
Thanks a ton, abolibibelot. You're a legend!

Well, at least I did sumpting useful yesterday !
It may be a tad excessive for a little tip, but I could use some cheering up these days...

[manav]

You mean, mp4box.

Yeah, lol. As I have already told, I am noob with this stuff. I don't even have ffmpeg properly installed. Instead, I had just put the three .exe files (that come inside the ffmpeg folder) into the folder where my videos were stored in, and I was shift+right-clicking to run command prompt directly. I thought that first command is for ffmpeg only, and thus, was referring to the same.

Currently, I have put the mkvtoolnix files inside the videos' folder, along with the .bat file containing the command, and everything is working like a charm. Splitting all of those videos is a cake-walk now. I am constantly downloading GBs of videos/streams daily, so this was very much needed. In addition to that, output files being mkv is also proving to be beneficial because of a little lower size, and also because it has made it easy to group the files in file explorer. Attached a screenshot of the folder.

[abolibibelot]

I don't even have ffmpeg properly installed.

Command line programs don't need to be “installed”, strictly speaking, but it's more convenient to have them in the so-called Windows “PATH”.
https://www.howtogeek.com/118594/how-to-edit-your-system-path-for-easy-command-line-access/
On my system I included a single “CLI” directory in the “PATH”, and then I put in that directory a hard link of all frequently used executables.
Using hard links instead of simply copying the files reduces unnecessary clutter on the system partition (considering that some of these files can be quite large, for instance ffmpeg which is now about 60MB – and it's becoming quite an issue by the way since many programs which rely on ffmpeg each add a copy of ffmpeg.exe to their own directory, and so I might have at least a dozen copies of ffmpeg.exe on my system, which amounts to at least 700MB...).
LinkShellExtension makes it easy to create them.
https://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html
Otherwise, it's still possible to launch a CLI program by typing its complete path in the command. It's just more convenient to type only the name of the command / executable, when using a particular command regularly.

FOR %F in (*.flv) DO "C:\wherever in the godforsaken mess\of this unholy computer\that damn program may be\mkvmerge.exe" --output "%~nF.mkv" "%F" --split size:500M

Yet another possibilty, opening the executable's path in the command prompt before running the command with “CD”, then specifying the directory where the input files are located with “/R” :

CD "C:\wherever in the godforsaken mess\of this unholy computer\that damn program may be"
FOR /R "G:\Streamlink" %F in (*.flv) DO mkvmerge --output "%~nF.mkv" "%F" --split size:500M
# note : /R stands for "recursive", so this command would also process any subfolder inside "Streamlink"

Instead, I had just put the three .exe files (that come inside the ffmpeg folder) into the folder where my videos were stored in

To run ffmpeg, only the ffmpeg.exe file would be required. But if the mp4box command worked (even though it failed) that means that you already have the mp4box executable somewhere in your system's “PATH”. (Possibly inside “Program Files”.)