Merge / Joining Multiple Videos with Different Resolutions / FPS Using FFMP

[VideoHelper9061]

I tried running LosslessCut and the Apple Shortcuts-based "Join Media" command (with a-shell mini) to try merge a bunch of 10-45sec second video clips taken on an iphone 13 together, but the lossless, non-encoding route is creating a 2-3 hour file with frozen frames vs. the combined video should be only \~2 minutes long. I have all the .mov files listed in the below file (some are 1280x720 horizontal, some are 720x1280 vertical, some are 30fps, some are 29.99, some are 30.03fps etc.)

What command should i be running with ffmpeg that can join these files together with the least amount of quality loss (i assume some amount of encoding will need to get done since LosslessCut was failing with the freeze frame corruption issue).

I tried running

ffmpeg -safe 0 -f concat -i files_to_combine.txt merged2.MOV

but that didn't work since it gave me a corrupted file (only the first video of the 6 plays then it freezes frame after that.

files_to_combine.txt

file ./1IMG_3.mov
file ./2IMG_8.mov
file ./3IMG_8.mov
file ./4IMG_8.mov
file ./5IMG_3.mov
file ./6IMG_3.mov

[ProWo]

Try clever Ffmpeg-GUI.
Load you first chunk, click main, click join, drag & drop all other files into the grid and click join.
All chunks must have the same codecs.

[VideoHelper9061]

Try clever Ffmpeg-GUI.
Load you first chunk, click main, click join, drag & drop all other files into the grid and click join.
All chunks must have the same codecs.

it looks like 2 of my 6 files have different ffprobe outputs which is creating issues with different codecs to join. I was able to join 2/3/4.mov together using the below command. It looks like my stream orders don't match but codecs / tbn values (600) do match. any idea what to do to be able to make these compatible to join? If they cannot be joined without encoding, i don't mind encoding them together (do note that 1/5/6 are horizontal 720p while 234.mov is vertical 720p)

ffmpeg -safe 0 -f concat -i files_to_combine.txt -movflags use_metadata_tags -map_metadata 0 -vcodec copy -acodec copy 234.MOV

But now i want to combine 1.mov, 234.mov, 5.mov, 6.mov but 1/5 and 234/6.mov have different sets of stream orders.

1.mov

Duration: 00:00:43.40, start: 0.000000, bitrate: 1283 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720, 1207 kb/s, 30 fps, 30 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2024-02-19T17:30:17.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : H.264
  Stream #0:1[0x2](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:30:17.000000Z
        handler_name    : Core Media Metadata
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:30:17.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 69 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:30:17.000000Z
        handler_name    : Core Media Metadata
Unsupported codec with id 0 for input stream 1
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 0 for input stream 3

234.mov

Duration: 00:00:45.75, start: 0.000000, bitrate: 1379 kb/s
  Stream #0:0[0x1]: Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720, 1210 kb/s, 30 fps, 30 tbr, 19200 tbn (default)
      Metadata:
        handler_name    : Core Media Video
        vendor_id       : FFMP
        encoder         : H.264
      Side data:
        displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2]: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 162 kb/s (default)
      Metadata:
        handler_name    : Core Media Audio

vendor_id : [0][0][0][0]

6.mov

Duration: 00:00:32.00, start: 0.000000, bitrate: 1402 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 96 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:52:27.000000Z
        handler_name    : Core Media Audio
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720, 1230 kb/s, 30.03 fps, 30 tbr, 600 tbn (default)
      Metadata:
        creation_time   : 2024-02-19T17:52:27.000000Z
        handler_name    : Core Media Video
        vendor_id       : [0][0][0][0]
        encoder         : H.264
  Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:52:27.000000Z
        handler_name    : Core Media Metadata
  Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:52:27.000000Z
        handler_name    : Core Media Metadata
  Stream #0:4[0x5](und): Data: none (mebx / 0x7862656D), 69 kb/s (default)
      Metadata:
        creation_time   : 2024-02-19T17:52:27.000000Z
        handler_name    : Core Media Metadata
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 0 for input stream 3
Unsupported codec with id 0 for input stream 4

[Sharc]

As has been said, all files should be of the same format (codecs, framesize etc) for concatenation.
Save below script as 'normalize.cmd', put it onto your desktop and drag and drop your files alltogether onto the icon on your desktop. The files will be reencoded to a common (normalized) format, ready for concatenation:

:next
if "%~1"=="" goto done

ffmpeg -y -i "%~1" -c:a aac -c:v libx264 -preset slow -crf 20 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mov"

shift
goto next
:done

pause

[hydra3333]

Thanks ! I must look that up !
I wish I knew that 12 months ago when I was trying to create a DVD slideshow creator which accepted any format/size of images and any format/size of videos.

VFR and auto-rotation according to metadata during normalization (before resizing) are still going to be an issue I guess.
Thinking of phones where people do everything possible

(some are 1280x720 horizontal, some are 720x1280 vertical, some are 30fps, some are 29.99, some are 30.03fps etc.)

I've seen it established, for samsung's at least, that VFR can safely be thought of as CFR at the nominal framerate eg 30fps ... dunno if iphones do VFR and if they do whether it's real or wobbly cfr.

[Sharc]

... auto-rotation according to metadata during normalization (before resizing) are still going to be an issue I guess.
Thinking of phones where people do everything possible

Shouldn't be a problem if the source format and the target format (container) support the rotation flag, like .mp4 for example.

[Cornucopia]

Thanks ! I must look that up !
I wish I knew that 12 months ago when I was trying to create a DVD slideshow creator which accepted any format/size of images and any format/size of videos.

VFR and auto-rotation according to metadata during normalization (before resizing) are still going to be an issue I guess.
Thinking of phones where people do everything possible

(some are 1280x720 horizontal, some are 720x1280 vertical, some are 30fps, some are 29.99, some are 30.03fps etc.)

I've seen it established, for samsung's at least, that VFR can safely be thought of as CFR at the nominal framerate eg 30fps ... dunno if iphones do VFR and if they do whether it's real or wobbly cfr.

There are many variables here:
1. Make/model of the phone
2. Version of the OS
3. App used taking pix/vid recording
4. Settings for the cam in the OS and app
5. Amount of available light in the scene <<<This is the most important one!

If your scene isn't bright enough, the cam MUST lower the framerate in order to achieve sufficient light to the sensor. That's just the way phones are (currently) built. So, the more variable the light is, the more variable the framerate is.


Scott

[hydra3333]

... auto-rotation according to metadata during normalization (before resizing) are still going to be an issue I guess.
Thinking of phones where people do everything possible

Shouldn't be a problem if the source format and the target format (container) support the rotation flag, like .mp4 for example.

Just wondering, perhaps this code ignores the metadata rotation angle ( ... and also perhaps the aspect ratio applies to the rotated image rather than the raw image ? unsure what gets stretched and shrunk and if it really matters) ?

-filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black"

Love someone to test it, perhaps I'll get around to it sometime, I'm sure I have rotated .mp4s somewhere to play with

[Sharc]

... auto-rotation according to metadata during normalization (before resizing) are still going to be an issue I guess.
Thinking of phones where people do everything possible

Shouldn't be a problem if the source format and the target format (container) support the rotation flag, like .mp4 for example.

Just wondering, perhaps this code ignores the metadata rotation angle ( ... and also perhaps the aspect ratio applies to the rotated image rather than the raw image ? unsure what gets stretched and shrunk and if it really matters) ?

-filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black"

Love someone to test it, perhaps I'll get around to it sometime, I'm sure I have rotated .mp4s somewhere to play with

Nope. I use it myself for mobile phone clips in portrait and landscape, upside down etc. as well as with WhatsAp videos from many sources. Try it yourself instead of just speculating.
Of course if the source has no orientation flag the script doesn't know what to do. In this case one would need to force the orientation in the script.

[hydra3333]

[QUOTE=Sharc;2728695][QUOTE=hydra3333;2728692][QUOTE=Sharc;2728652]

Nope. I use it myself for mobile phone clips in portrait and landscape, upside down etc. as well as with WhatsAp videos from many sources. Try it yourself instead of just speculating.
Of course if the source has no orientation flag the script doesn't know what to do. In this case one would need to force the orientation in the script.

Beaut.
Did some stuff in python, with 3rd party packages, and encountered a fair few issues along those lines which is why I asked.
Looking forward to trying this out, it'll be really really handy.

[VideoHelper9061]

As has been said, all files should be of the same format (codecs, framesize etc) for concatenation.
Save below script as 'normalize.cmd', put it onto your desktop and drag and drop your files alltogether onto the icon on your desktop. The files will be reencoded to a common (normalized) format, ready for concatenation:

:next
if "%~1"=="" goto done

ffmpeg -y -i "%~1" -c:a aac -c:v libx264 -preset slow -crf 20 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mov"

shift
goto next
:done

pause

this worked perfectly for an encoded then joined set of videos

quick question - do you have the CMD command code so that i can drop all the "1/2/3/4/5/6.MOV_normalized" files into the CMD to join it instead of running the ffmpeg command manually from command line with a separate .txt file referencing the name of them?

[Sharc]

As has been said, all files should be of the same format (codecs, framesize etc) for concatenation.
Save below script as 'normalize.cmd', put it onto your desktop and drag and drop your files alltogether onto the icon on your desktop. The files will be reencoded to a common (normalized) format, ready for concatenation:

:next
if "%~1"=="" goto done

ffmpeg -y -i "%~1" -c:a aac -c:v libx264 -preset slow -crf 20 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mov"

shift
goto next
:done

pause

this worked perfectly for an encoded then joined set of videos

quick question - do you have the CMD command code so that i can drop all the "1/2/3/4/5/6.MOV_normalized" files into the CMD to join it instead of running the ffmpeg command manually from command line with a separate .txt file referencing the name of them?

Try this. All 3 phases are included. Phase 4 deletes the intermediate normalized files. Skip if you want to keep these.
You can change the container to .mov if needed

REM === 1) Normalization ===
:next
if "%~1"=="" goto done
ffmpeg -y -i "%~1" -c:a aac -c:v libx264 -preset slow -crf 20 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mp4"
shift
goto next
:done

REM === 2) Creating the files list ===
(for %%i in (*normalized.mp4) do @echo file '%%i') > mylist.txt

REM === 3) Joining the files ===
ffmpeg -f concat -safe 0 -i mylist.txt -c copy all_joined.mp4

REM === 4) Delete the normalized files
del *normalized*    

pause

Note: Use on progressive video only.

[VideoHelper9061]

As has been said, all files should be of the same format (codecs, framesize etc) for concatenation.
Save below script as 'normalize.cmd', put it onto your desktop and drag and drop your files alltogether onto the icon on your desktop. The files will be reencoded to a common (normalized) format, ready for concatenation:

:next
if "%~1"=="" goto done

ffmpeg -y -i "%~1" -c:a aac -c:v libx264 -preset slow -crf 20 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mov"

shift
goto next
:done

pause

this worked perfectly for an encoded then joined set of videos

quick question - do you have the CMD command code so that i can drop all the "1/2/3/4/5/6.MOV_normalized" files into the CMD to join it instead of running the ffmpeg command manually from command line with a separate .txt file referencing the name of them?

Try this. All 3 phases are included. Phase 4 deletes the intermediate normalized files. Skip if you want to keep these.
You can change the container to .mov if needed

REM === 1) Normalization ===
:next
if "%~1"=="" goto done
ffmpeg -y -i "%~1" -c:a aac -c:v libx264 -preset slow -crf 20 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mp4"
shift
goto next
:done

REM === 2) Creating the files list ===
(for %%i in (*normalized.mp4) do @echo file '%%i') > mylist.txt

REM === 3) Joining the files ===
ffmpeg -f concat -safe 0 -i mylist.txt -c copy all_joined.mp4

REM === 4) Delete the normalized files
del *normalized*    

pause

Note: Use on progressive video only.

I ran this code and it stripped all the audio out of the master combined file. here's the ffprobe output

Duration: 00:02:15.51, start: 0.000000, bitrate: 1654 kb/s
  Stream #0:0[0x1]: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720, 1651 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : Core Media Video
        vendor_id       : FFMP
        encoder         : Lavc60.41.100 libx264

[Sharc]

No idea. Do the individual normalized files have the audio?
Try again with replacing all occurrences of .mp4 by .mov in the script. Also add -sn to the ffmpeg commandline (no subtitles)
Something must be awkward with your files.

Or upload your files so I may try.

[VideoHelper9061]

No idea. Do the individual normalized files have the audio?
Try again with replacing all occurrences of .mp4 by .mov in the script. Also add -sn to the ffmpeg commandline (no subtitles)
Something must be awkward with your files.

Or upload your files so I may try.

I tried both .mov and .mp4. some of the files have audio, some do not (not sure reason why 2 of the 6 don't have audio). I'll PM you the files for you to try.

FWIW - if i do just your initial step of combining (which creates 6 files i.e. 1.mov_normalized.mov, etc.), the audio is retained in that. so i'm not sure if audio is being stripped at step REM 2/3/4?

[Sharc]

The problem is the missing audio track of files 1.mov and 5.mov. The script works perfectly for the other files.
A stream remapping may possibly help with the script, or better include the missing audio track.

Edit: from the ffmpeg / concat docs:
https://ffmpeg.org/ffmpeg-filters.html#concat

All segments must have the same number of streams of each type, and that will also be the number of streams at output.

So remapping wouldn't help. One would have to add the missing audio stream to the files 1.mov and 5.mov. Here's how you can add an audio stream to the video. The script then works perfectly with the modified files. See your PM.

REM Adding an audio stream to the video stream
ffmpeg -i "%~1" -i "%~2" -map 0:v -map 1:a -c copy -shortest "%~1_with audio.mov"

Another solution is to put the files onto the timeline of an NLE. No problems here with Shotcut for example, see your PM.

[VideoHelper9061]

The problem is the missing audio track of files 1.mov and 5.mov. The script works perfectly for the other files.
A stream remapping may possibly help with the script, or better include the missing audio track.

Edit: from the ffmpeg / concat docs:
https://ffmpeg.org/ffmpeg-filters.html#concat

All segments must have the same number of streams of each type, and that will also be the number of streams at output.

So remapping wouldn't help. One would have to add the missing audio stream to the files 1.mov and 5.mov. Here's how you can add an audio stream to the video. The script then works perfectly with the modified files. See your PM.

REM Adding an audio stream to the video stream
ffmpeg -i "%~1" -i "%~2" -map 0:v -map 1:a -c copy -shortest "%~1_with audio.mov"

Another solution is to put the files onto the timeline of an NLE. No problems here with Shotcut for example, see your PM.

Thank you very much for the help here. On the adding an audio stream - can a "blank" audio be added just to fill in the audio piece? I assume "%~2" is referencing an audio file here, what would i need to edit in the below CMD command be to drag and drop a file that just adds a blank audio to it (if it has no audio) before i run it through the master 4-step command above?

REM === adding an audio stream to the video stream ===
:next
if "%~1"=="" goto done

ffmpeg -i "%~1" -i "%~2" -map 0:v -map 1:a -c copy -shortest "%~1_with audio.mov"

shift
goto next
:done

pause

[Sharc]

Here the commandline for adding a silent audio stream to the video

REM Adding a silent stream
ffmpeg -i "%~1" -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 -c:v copy -shortest "%~1_with_silent_audio.mov"

[Sharc]

So here the full script which includes the audio fix.

REM ====== 1) Normalization ======
:next
if "%~1"=="" goto done
ffmpeg -i "%~1" -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 -c:v copy -shortest "%~1_audiofix.mov"
ffmpeg -y -i "%~1_audiofix.mov" -c:a aac -c:v libx264 -preset slow -crf 18 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mov"
shift
goto next
:done

REM ====== 2) Creating the files list ======
(for %%i in (*normalized.mov) do @echo file '%%i') > mylist.txt

REM ====== 3) Joining the files ======
ffmpeg -f concat -safe 0 -i mylist.txt -c copy all_joined.mov

REM ====== 4) Delete the temporary files ======
del *audiofix*
del *normalized*

pause

[VideoHelper9061]

So here the full script which includes the audio fix.

REM ====== 1) Normalization ======
:next
if "%~1"=="" goto done
ffmpeg -i "%~1" -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 -c:v copy -shortest "%~1_audiofix.mov"
ffmpeg -y -i "%~1_audiofix.mov" -c:a aac -c:v libx264 -preset slow -crf 18 -filter:v "scale=w=1280:h=720:force_original_aspect_ratio=decrease,pad=1280/sar:720:-1:-1:color=black" "%~1_normalized.mov"
shift
goto next
:done

REM ====== 2) Creating the files list ======
(for %%i in (*normalized.mov) do @echo file '%%i') > mylist.txt

REM ====== 3) Joining the files ======
ffmpeg -f concat -safe 0 -i mylist.txt -c copy all_joined.mov

REM ====== 4) Delete the temporary files ======
del *audiofix*
del *normalized*

pause

dope - this worked perfectly!! Last question how do i copy all the metadata from one of the original files (say 1.mov) so that i can keep the capture date, camera used, etc. on the new all_joined video?

[Sharc]

Last question how do i copy all the metadata from one of the original files (say 1.mov) so that i can keep the capture date, camera used, etc. on the new all_joined video?

Try something like:

ffmpeg -i "%~1" -f ffmetadata metadatafile.txt

Edit:
.... and then analyze ....

[Attachment 77892 - Click to enlarge]

[VideoHelper9061]

Last question how do i copy all the metadata from one of the original files (say 1.mov) so that i can keep the capture date, camera used, etc. on the new all_joined video?

Try something like:

ffmpeg -i "%~1" -f ffmetadata metadatafile.txt

Edit:
.... and then analyze ....

I outputted 1.mov's metadata to data.txt, how do i know overwrite all_joined.mov's metadata with the data.txt metadata?

[Sharc]

I outputted 1.mov's metadata to data.txt, how do i know overwrite all_joined.mov's metadata with the data.txt metadata?

Try this

ffmpeg -i 1.mov -i all_joined.mov -map 1 -map_metadata 0 -c copy -movflags use_metadata_tags "all_joned_with_metadatacopy.mov"