Outputting Frame Sequence Images to Create A Single Image

[imaginethat]

Hello & Happy 2022!

I would like to create a single image file to accompany certain video files I have, the video to image sequence function takes a still image every so many seconds/frames of video then saves those frames in separate files. I want to take all those images combined into a single one so I can easily view that one image before opening the video file to locate a particular spot I may want to skip to.

Thanks!

[imaginethat]

I've seen it done before basically turning taking the original high quality frame sequence images turning into a single large thumbnail directory image basically.

[jagabo]

With ffmpeg:

ffmpeg -i intput.mp4 -vf "thumbnail=37,scale=128:72,tile=6x6" output.jpg

The thumbnail value should be a little more than the total number of tiles. scale is the size of each thumbnail, tile is the array size, X:Y. From an old FedEx ad:


[Attachment 62829 - Click to enlarge]


Some media players have this built in. Like MPCHC. File -> Save Thumbnails.

[poisondeathray]

There are several thumbnail applications here
https://www.videohelp.com/software/sections/screenshots

[imaginethat]

There are several thumbnail applications here
https://www.videohelp.com/software/sections/screenshots

Exactly what I was after thank you very much, GDS Video Thumbnailer very simple to use and the different available presets are handy, only question I have if anyone knows how to move the "File Info Fields" to the bottom instead of the top. Thanks again

[imaginethat]

actually have another follow-up question for anyone who with experience using GDS Video Thumbnailer, I can't find a tutorial online but it seems the when I select more then 5x4 20 frames the thumbnails are squished into the same height jpg file, is there a way to generate 36+ frame thumbnail file that will increase the height of the jpg so the frames are not squished/cut off? also less important but would be nice to put the "file info field" text at the bottom instead of top of jpg. Thanks again!

[Budman1]

Some useful ffmpeg scripts to create tiled image and video:

ffmpeg -i myvideo.avi -vf fps=1/60 img%03d.jpg
ffmpeg -i input -filter_complex "select='not(mod(n,300))',scale=120:-1,tile=layout=3x2" -vframes 1 -q:v 2 output.jpg

all in one, every 30 seconds
ffmpeg -ss 00:00:05 -i cgop.mp4 -frames 1 -vf "select=not(mod(n\,900)),scale=160:-1,tile=4x3" tile.png

so many segments 4=total seconds / number of segments
ffmpeg -i yosemite.mp4 -vf select='lt(mod(t,4),2)',setpts=N/FRAME_RATE/TB -af aselect='lt(mod(t,120),2)',asetpts=N/SR/TB  out17.mp4

Adjust as necessary
---------------

[imaginethat]

Some useful ffmpeg scripts to create tiled image and video:

ffmpeg -i myvideo.avi -vf fps=1/60 img%03d.jpg
ffmpeg -i input -filter_complex "select='not(mod(n,300))',scale=120:-1,tile=layout=3x2" -vframes 1 -q:v 2 output.jpg

all in one, every 30 seconds
ffmpeg -ss 00:00:05 -i cgop.mp4 -frames 1 -vf "select=not(mod(n\,900)),scale=160:-1,tile=4x3" tile.png

so many segments 4=total seconds / number of segments
ffmpeg -i yosemite.mp4 -vf select='lt(mod(t,4),2)',setpts=N/FRAME_RATE/TB -af aselect='lt(mod(t,120),2)',asetpts=N/SR/TB  out17.mp4

Adjust as necessary
---------------

thank you for this, like most I'm better with UI software then linux command base, but this is a good chance to try to wrap my head around ffmpeg command lines, I've finally got it installed i've been trying to look through ffmpeg documentation guide to understand each of the functions you use in the lines above but if you could be kind enough to help would appreciate it.

I have a folder of .mp4 videos different lengths varying from a few minutes to a few hours I want to create for each video extract the best possible quality (100% ideally) single image file with every 30 seconds of video on a 3 column tile by however many rows it needs based on the particular video files run time. Also ideal but not a requirement is to include original video file metadata info text on the bottom of that thumbnail image. Thanks!

[Budman1]

You mention metadata. Which metadata would that be? Thanks

[imaginethat]

You mention metadata. Which metadata would that be? Thanks

First priority is producing a high quality image that will be properly measured out squares/tiles of 3 (I may prefer 4-5 for some) rows by 'X' columns depending on the length of the video / interval of frame shots per minute I choose and the height of the outputted image file not be limited causing it to squish 50 tiles in same height as 20 as is what GDS is doing. So I want the width of the outputted image file to be uniform if I choose 4 tile row instead of 3 the tiles will be smaller then height based on frame shot interval selected divided by total video length, hopefully you understand what I wrote lol

I will elaborate on my use case end game that I think would be extremely useful to a large number of people in organizing their personal video/photos that were never properly organized. I have photos/videos that contain a particular person such as my mother or father which I cannot even find because of the tens of thousands of other unorganized photos mixed in. I'd like to run a facial recognition software on my photos/videos folder to find particular people that I can mark in the metadata similar to how Facebook implants/identifies particular people in photos and the location/date the photo/video was taken etc then I can do more precise search of my photo/videos folder containing particular people (ie can create a database of known faces/names and run a script that identifies file to contain only 1 or multiple known people and marks them as such for easy searching).

The program GDS Video Thumbnailer allows you to choose to include info about the original video file; Duration, Name, Path, Size, Frame Rate, Frame Size, Video Info, Audio Info, my only issue is GDS puts this metadata text at the top of the image I would much prefer on bottom if I were to use, also ideally the option to include even more metadata fields if available such as film/photo date, location, names of known people included etc.

A program like this would be a game changer for people consolidating/organizing family photos/videos if you could combine all your files into a folder create a face/name database of known people and then run a script bam, decades of memories organized and easily searched.

Thanks

[DB83]

Try Image Grabber


https://www.videohelp.com/software/ImageGrabber


That allows you to place the file description in any of the 4 corners of the picture frame.

[jagabo]

I've come up with a drag/drop batch file that produces an array of specified columns at the original resolution. The number of rows is automatically set to an appropriate number for the length of the video and the chosen interval. It uses ffprobe to get the required stats:

@echo off

REM user variables.  Set these to the desired values.
REM cols is the number of columns for the image array.
REM t_interval is the time interval in seconds.

set /A cols = 3
set /A t_interval = 30

REM get video stats:

ffprobe.exe -v quiet -show_entries stream=r_frame_rate -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p fps= < %1.txt
ffprobe.exe -v quiet -show_entries stream=duration -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p duration= < %1.txt
ffprobe.exe -v quiet -show_entries stream=width -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p width= < %1.txt
ffprobe.exe -v quiet -show_entries stream=height -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p height= < %1.txt
del %1.txt

REM At this point the environment variables cols, t_interval, fps, duration, width, and height are available.
REM
REM Note 1 fps is in numerator/denominator format -- convert to an integer with set /A frate = fps.  Only works for N/1
REM Note 2: calculate the number of rows required.
REM note 3: calculate the frame count interval: user t_interval * frate

set /A frate = fps
set /A rows = duration / t_interval / cols
set /A interval = t_interval * frate

REM for debugging
REM echo duration = %duration%
REM echo fps = %fps%
REM echo frate = %frate%
REM echo width = %width%
REM echo height = %height%
REM echo cols = %cols%
REM echo rows = %rows%
REM echo interval = %interval%
REM pause

@echo on

ffmpeg -i %1 -vf "thumbnail=%interval%,scale=%width%:%height%,tile=%cols%x%rows%" -frames:v 1 "%~dpnx1.jpg"

pause

It has a few known problems at this point.

1) ffprobe returns the frame rate in numerator/denominator format -- 25/1 for example. I use "set /A frate = fps" to convert that to a single integer. But all it does is discard the "/denominator" portion. So a rate like 24000/1001 becomes 24000. This needs to be fixed. You can probably use MediaInfoCLI which gets the frame rate as a floating point number like 23.976.

2) ffprobe doesn't return a duration for MKV files. So this doesn't work for MKV. I've tested with mp4, mpg, avi, ts -- all of those worked (as long as #1 wasn't a problem). MediaInfoCLI can get the runtime of MKV files in milliseconds, easily converted to seconds.

3) I don't know what will happen if you feed a long video with a large frame size. A 2 hour long 3840x2650 video will produce a 3x80 array. That's a 11520x204800 image. I don't know if that will work.

[Budman1]

@jagabo
I tried your script before substituting mediainfoCLI but i get a memory error from an interval of 299700. Any idea why?
Thanks

[jagabo]

MediaInfo reports frame rates in milliseconds. You need to divide by 1000 to get seconds.

[Budman1]

I have not used mediainfo yet. Just tried the original script. I misspoke before. The thumbnail=299700 causing memory error

[jagabo]

I tried your script before substituting mediainfoCLI but i get a memory error from an interval of 299700. Any idea why?

You mean you used ffprobe to get the r_frame_rate value? It returns the fps as a numerator/denominator. If the denominator is not 1 then the later calculation of frate gives a wrong value (it doesn't calculate numerator/denominator, it just stops parsing at a non-numeric character, so you just get frate = numerator). Since interval is calculated from frame interval will also be wrong.

The batch file needs some work before it's ready to use in the wild.

[Budman1]

Ahhhhhh. Yes it is 30000/1001. Ill keep working and substitute mediainfo after i have it working. Mediainfo also report duration in ms or HH:MM:SS format but that i can handle. Thanks for help

[jagabo]

I worked out a fix for the numerator/denominator problem. And I added another fix for a rows=0 calculation:

@echo off

REM user variables.  Set these to the desired values.
REM cols is the number of columns for the image array.
REM t_interval is the time interval in seconds.

set /A cols = 3
set /A t_interval = 30

REM get video stats:

ffprobe.exe -v quiet -show_entries stream=r_frame_rate -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p fps= < %1.txt
ffprobe.exe -v quiet -show_entries stream=duration -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p duration= < %1.txt
ffprobe.exe -v quiet -show_entries stream=width -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p width= < %1.txt
ffprobe.exe -v quiet -show_entries stream=height -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p height= < %1.txt
del %1.txt

REM At this point the environment variables cols, t_interval, fps, duration, width, and height are available.
REM
REM Note 1 fps is in numerator/denominator format -- convert to an integer with set /A frate = fps.
REM Note 2: caculate the number of rows.
REM note 3: calculate the frame count interval from user t_interval * frate

set /A frate = %fps%
set /A rows = duration / t_interval / cols
if %rows% EQU 0 set /A rows = 1
set /A interval = t_interval * frate

REM for debugging
echo duration = %duration%
echo fps = %fps%
echo frate = %frate%
echo width = %width%
echo height = %height%
echo cols = %cols%
echo rows = %rows%
echo interval = %interval%
pause

@echo on

ffmpeg -i %1 -vf "thumbnail=%interval%,scale=%width%:%height%,tile=%cols%x%rows%" -frames:v 1 "%~dpnx1.jpg"

pause

You'll still need a fix for the MKV problem.

[Budman1]

I did a preliminary test using mediainfo and it works for mkv. If i may, I'll use you script and modify it for mediainfo for mkv also since media onfo report all necessas necessary metadatameta.
Thank you again for an excellent written script

[Budman1]

OK the script below works with MKV as well as mp4. I just added MediaInfoCLI line for Duration. All the other FFPROBE lines work fine. MediaInfo reports duration in Milliseconds or Standard HH:MM:SS as well as HH:MM:SS.mmm but no seconds so I added a step to convert to seconds.

I also set t_interval lower since the video I used is short and I wanted several rows instead of 1. It works with the original 30 but gives only 1 row.
Disregard the distortion in some of the frames belows as the test video was created with Picture in Picture as well as transitions every so often. The ugly frames are just showing partial transitioned new video.

IMPORTANT:
My script change shows the MediaInfoCLI as just MEDIAINFO since I changed the name for easier standard use. You may have to use whatever the name is on the current version of MediaInfoCli

@echo off

REM user variables.  Set these to the desired values.
REM cols is the number of columns for the image array.
REM t_interval is the time interval in seconds.

set /A cols = 3
set /A t_interval = 5

REM get video stats:

ffprobe.exe -v quiet -show_entries stream=r_frame_rate -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p fps= < %1.txt
"MediaInfo.exe" "--Inform=Video;%%Duration%%" %1 >%1.txt
set /p Tempduration= < %1.txt
echo %Tempduration%
set /A duration=%Tempduration/1000
echo duration is %duration%
ffprobe.exe -v quiet -show_entries stream=width -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p width= < %1.txt
ffprobe.exe -v quiet -show_entries stream=height -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt
set /p height= < %1.txt
del %1.txt

REM At this point the environment variables cols, t_interval, fps, duration, width, and height are available.
REM
REM Note 1 fps is in numerator/denominator format -- convert to an integer with set /A frate = fps.
REM Note 2: caculate the number of rows.
REM note 3: calculate the frame count interval from user t_interval * frate

set /A frate = %fps%
set /A rows = duration / t_interval / cols
if %rows% EQU 0 set /A rows = 1
set /A interval = t_interval * frate

REM for debugging
echo duration = %duration%
echo fps = %fps%
echo frate = %frate%
echo width = %width%
echo height = %height%
echo cols = %cols%
echo rows = %rows%
echo interval = %interval%
pause

@echo on

ffmpeg -i %1 -vf "thumbnail=%interval%,scale=%width%:%height%,tile=%cols%x%rows%" -frames:v 1 "%~dpnx1.jpg"

pause

[Attachment 62877 - Click to enlarge]

[Budman1]

OOOPS... Tempduration line should be set /A duration=Tempduration/1000 not set /A duration=%Tempduration/1000 but works either way on my machine.

[jagabo]

Just some musings...

It would probably useful to add a scale parameter to reduce the thumbnail size for large videos. I tried the batch file on a 1080p movie. It ran for about 5 minutes then gave the error:

[mjpeg @ 000001f7b3ffdb80] [IMGUTILS @ 00000092131fe520] Picture size 5760x57120 is invalid
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

I don't know what the JPEG size limit is. Maybe 32K?

One could probably build a subtitle file of metadata with ffprobe or mediainfocli. That could be overlaid onto the image.

The thumbnail filter selects the "most representative" frame from each segment. I don't know how that's determined.
https://ffmpeg.org/ffmpeg-filters.html#thumbnail

Colors may not be right. I think ffmpeg defaults to rec.601 when converting to RGB images -- so rec.709 videos (most HD) will have slightly wrong colors. HDR video isn't tone mapped.

[Budman1]

It may also be advantageous to use only scene xhanges to prevent possible repeats of images since it would be a better representation of each video. The limit could be set for the number of images so they don't overrun the screen size. Most viewer will shrink the imaes to screen size anyway which makes images unrecognizable.

The size could be set to a certain minimum to further prevent errors. Not sure why unlimited rows were wanted but... Hmmmmm if you add the metadata overlay, sounds like a GUI project. Hmmmm...

[jagabo]

By the way, ffprobe can output the duration of MKV files. For example:

ffprobe.exe -v quiet -show_entries stream -select_streams v:0 %1

outputs:

[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=High
codec_type=video
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=854
height=480
coded_width=854
coded_height=480
closed_captions=0
has_b_frames=2
sample_aspect_ratio=10:11
display_aspect_ratio=427:264
pix_fmt=yuv420p
level=30
color_range=tv
color_space=smpte170m
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
refs=1
is_avc=true
nal_length_size=4
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/1000
start_pts=0
start_time=0.000000
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:language=eng
TAG:HANDLER_NAME=VideoHandler
TAG:VENDOR_ID=[0][0][0][0]
TAG:DURATION=00:03:31.200000000
[/STREAM]

You can see TAG:DURATION at the end of the list in HH:MM:SS.decimal format. But I don't know how to get it to output just that one value.

[Budman1]

Only way i know to get seconds is parse and do the math(*3600,*60,etc)
Something like:
echo off
for /F "tokens=1-4 delims=/ " %%i in ('%date%') do (
set dow=%%i
set /A hours=%%j*3600
set /A minutes=%%k*60
set seconds=%%l
set /A totalseconds=%%j+%%k+%%l
)

Can you do ffprobe show_stream show_entries GAPURATION to get just duration?

[jagabo]

Can you do ffprobe show_stream show_entries GAPURATION to get just duration?

I think I figured it out. Instead of:

ffprobe.exe -v quiet -show_entries stream=duration -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt

use:

ffprobe.exe -v quiet -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 -select_streams v:0 %1 > %1.txt

So far that has worked for all files I've tried (MP4, MKV, TS, MPG, AVI).

[jagabo]

Next might be to automatically calculate the rows/cols to generate roughly equal values:

segments = duration / t_interval
rows = sqrt(segments)
cols = segments / rows

[Budman1]

There might still be a lot of segments? How about. Limiting to specific size tile that fits and of meaningful size. 3x8 and 4x8 works nice on mine so something like:
Interval(fps)=total segments /total file length

[Budman1]

Scene changes works great, just need to set the change amount. Could be done with manual input or scene changes count.
Anyway looks like this:

[Attachment 62890 - Click to enlarge]

[imaginethat]

Next might be to automatically calculate the rows/cols to generate roughly equal values:

segments = duration / t_interval
rows = sqrt(segments)
cols = segments / rows

This is great, I am happy to find others which have the technical knowledge to accomplish this interested, I think would be very useful to many people.

The idea is to produce a single large high quality image file (within reason) uniform in width and length based on video length/frame interval chosen to accompany each video file, this way I can keep reference folder and don't need to open and scan through the video files to exactly find the place I'm looking for.

I know its sad but I'm brand new to ffmpeg and have attention issues read the instructions a few times now I'm better with UI software then text based, but I will get there especially with the aid of a completed code/script I appreciate you updating on progress, I want to batch produce my entire video collection and clean up and cold store some of the less important large videos.

Thanks again and hope everyones new year is going well!