ffmpeg avc from .jpgs of arbitrary dimensions, maintaining aspect ratio

[hydra3333]

Cool !

Another issue was using frame lengths for transitions or referencing lengths of some sort in frames was not a good idea, because if fpsnum/fpsden changes, it is all wrong then. So for transition lengths, image lengths and keeping reference for audio lengths, there is time in milliseconds used as a reference. Also pydub works in milliseconds, so it is just all working.

OK. I used frame numbers after conversion of all incoming clips (a) VFR to CFR to target FPS (b) off-target FPS to target FPS, for the same reason. Then ms = (frames/target_fps)*1000 and it all falls into place since there's only one fps. I'm still "iffy" with the VFR to CFR methods, I need to look into it more.

Don't rely on ChatGPT to generate working code. I had a rather extensive play. It generates good ideas but its code has lots of subtle and unsubtle bugs and red herrings, although some of it could work if you're lucky. For example with pydub, it generated incorrect calculations for frame based insertions time after time, incorrect use of functions (did not do what it thought they did), used incorrect parameters in functions it found which did not exist, etc etc. Very often I'd specify a change and it made it while also forgetting an essential change it had made only minutes earlier. I asked it to simulate some calculations and it did try, but in once case got them very wrong and even when pointed out had trouble fixing them bit by tiny bit.
As an ideas generator, cool; for generating working code, well nice try but no cigar. Don't let it loose as yet for coding nuclear power plan scada software

Another issue was with vapoursynth bas source, it could have a problem with some weird audios in some old camcordes etc., but ffmpeg would just load it. So using pydub for videopaths, if there is a problem it falls back to make wave audio using ffmpeg.

Nice, I'm keen to use see that. I had thought pydub used ffmpeg under its covers. Getting excited to see how you go about it.
I use a pre-prepared large-ish background audio file of free instrumentals etc appended together into a .aac which pydub should open and "overlay" small audios onto, and hopefully should create .aac (not tested that bit).

Another thing to watch for was to make sure audio and video times for a segment are same exact length. It extends silent parts to some audio tracks (that yours video surveillance footage I guess) because audio was much shorter. This way it would not go out of sync (video and audio for the total length).

Yes, for a segment I trimmed/padded it with silence as required, before "overlaying" it into the background with pydub since I'd thought the overlay method kept the underlying video duration etc as-is; I need to check that ... and also happily see your code.

... make it on the need bases, gradually ... So audio can be run at least after that, then muxed together.

For video, I kludged chunks of 200 or so pics/clips (getting circa 23fps) into temp intermediate ffv1 files (oh the disk space used !) and later "-f concat input" transcode (taking care of pts/dts issues ay the same time) those part-slideshow videos together with ffmpeg into the final h.264 video. Haven't checked whether this maintains color metadata etc.
Audio sounds similar but different, to me.
I don't vapoursynth audio at all; given the video parsing first we know it's frame insertion point in its chunk which we extrapolate an insertion point aligning with the final video, based on its frame numbers in chunks since it's target_fps is fixed.

[JN-]

Hello.

I have a large number of old family .jpg pics with arbitrary filenames (although ordered) which I'd like to turn into a fast video slideshow at say 1080p (most TVs have this) with 1s/2s frame display duration ... or even dabble with 2160p.

The thing is the .jpg's will have arbitrary dimensions (different cameras, settings) and so need to be resized (with some "quality" resizer) whilst maintaining aspect ratio during the process. Famous last words subject to change: speed isn't necessarily an issue.

Unfortunately, I am also clueless about what (if anything) to do about colourspace conversions for this case.
All I know about the .jpg files is that they are jpegs , some old some new, some "landscape" some "portrait"
I guess the result would need to be Rec.709, but how to ensure it safely gets there prior to encoding is a question.

So, could some kind souls please provide suggestions on how to use ffmpeg to
1. do the "quality" resizing from arbitrarily dimensioned .jpg inputs whilst maintaining aspect ratios
2. do any necessary colourspace conversions to ensure input to the h264_nvenc encoder is Rec.709 (or suggest a better alternative)

Thanks !

Context:
have seen https://trac.ffmpeg.org/wiki/Slideshow
have an nvidia "2060 Super" with an AMD 3900X cpu
have an ffmpeg build which accepts vapoursynth (or even avisynth) input, if that helps, although I'd prefer
prefer to use nvidia's h264_nvenc gpu encoding, probably with parameters including something like this (once I figure out how to force every frame to be an i-frame)

-c:v h264_nvenc -pix_fmt nv12 -preset p7 -multipass fullres -forced-idr 1 -g 1 -coder:v cabac -spatial-aq 1 -temporal-aq 1 -dpb_size 0 -bf:v 0 -b_ref_mode:v 0 -rc:v vbr -cq:v 0-b:v %bitrate_target% -minrate:v %bitrate_min% -maxrate:v %bitrate_max% -bufsize %bitrate_target% -profile:v high -level 5.2 -movflags +faststart+write_colr

Have you considered using an nle instead of ffmpeg to create the slide shows for each of the sets of folders of video clips and stills. Where the frame rates for video clips are too far apart to combine in a single slide show (25 to 30, 30 to 25 fps you may get away with) then use more than a single slide show per folder.

Within the nle you can use use pan, scan, cropping as required of stills to output to a slideshow video file. Add music as required to the stills sections. Most modern nle’s should handle vfr to cfr, or do it before import. Of course it means a fairly non automated process, more hands on, it depends on how much time you have to do it all.

[_Al_]

OOoooh, don't take our toy away ....
Yes, sure nle is fast and quick, not fully automatized though, what comes to mind, what is handled in this processing as oppose to nle:
-automate custom transition image-video, video-image, image-image,video-video,
-automatic 601 to709 colorspace conversion if output is 709 or vice versa
-specify upscaler or downscaler (any code if available in python), that means whatever based on input
-deinterlacers could be QTGMC, again depending on source, right now for SD QTGMC is used, higher resolution uses Bwdif but if QTGMC is slow or knowing sources are mostly bad, deinterlace could be select based on video extension, that is a good idea also
-nle tends to hiccup with weird audios, maybe videos too, here using ffmpeg for audio and specify a selected source video plugin for an extension would work more reliably (think early or some cheap camera videos)

nle can input music with a comfort, yes, no problem, that could be done on audio output as well though, having audio exported as wav,
manual pan and cropping, that is editing, which is all different category, thats editing a video

[hydra3333]

Have you considered using an nle instead of ffmpeg to create the slide shows for each of the sets of folders of video clips and stills. Where the frame rates for video clips are too far apart to combine in a single slide show (25 to 30, 30 to 25 fps you may get away with) then use more than a single slide show per folder.

Within the nle you can use use pan, scan, cropping as required of stills to output to a slideshow video file. Add music as required to the stills sections. Most modern nle’s should handle vfr to cfr, or do it before import. Of course it means a fairly non automated process, more hands on, it depends on how much time you have to do it all.

Thank you, yes I have, and tried one (Nero).
With over 28,000 files with a variety for formats/codecs in hundreds of folders ... not practically doable.
A thought, when trying to merge video clip's of different framerates and force them to assume the same framerate without "transcoding", yes that is doable (I tried it), however audio will become out of sync ... eg phone cameras are a pain in that they generally produce VFR, sometimes also with odd framerates like 16.1 or 31.4 (I'm in PAL 25fps land).
I'm not sure what a free NLE would do with VFR and a variety of CFR framerates, I suspect less than agreeable things

Thanks for the thought, though.

[JN-]

Your welcome.

[hydra3333]

OK.

I usually run stuff in a (free) Windows Sandbox that comes with Windows 10/11, since it is dead easy.

Provided one has enabled it in Windows settings, the web tells me that you can stoke one up easily and when you close it then everything is automatically deleted permanently.
One could have a read-write host folder mapped and copy everything you want to keep into that before closing the sandbox.
Be prepared to have (lots of) space used on your C: drive if/when you run one, I'm not sure how to tell WSB to run on another disk, only how to map a folder.

Anyway, for posterity and just in case someone else may find it useful to have an example to work from for other stuff,

Here is the 2 files you need.
You will need to edit both files, as they contain matching folder mappings, host <--> sandbox ... you can delete or modify bits in them to suit yourself.
There are some dependencies within the files which you must delete or change (or comment out) as well.

The 2 files do everything to start it:
AI_sandbox_v01.wsb - double click on this to stoke up a Sandbox with the specified settings and host <--> sandbox folder mappings
AI_Setup_v01.bat - this is auto-run once the sandbox is fired up, it creates desktop icons and handy files, etc
Both of these file currently have a HARD dependency of being located on the host in C:\SOFTWARE\WindowsSandbox

AI_Setup_v01.bat also creates 2 .bat files on the sandbox desktop:

000-INIT.bat
once the sandbox finished firing up, double-click this to copy files etc.
111-SAVE_AI.bat
once one has finished playing, if you had write enabled the correct mapped folder in the .wsb then this will save stuff to it.

There are dependencies in AI_Setup_v01.bat which you can comment out, and mappings in AI_sandbox_v01.wsb which you can delete since .wsb crashes on comments.
C:\SOFTWARE must exist
C:\SOFTWARE\WindowsSandbox must exist and contain the above 2 files
C:\SOFTWARE\NPP must exist and contain notepad++ installation, i.e. notepad++.exe
And the other folder mappings and stuff you see in both those 2 files.

Here's the content of those files.
1. AI_sandbox_v01.wsb

<Configuration>
  <VGpu>Enable</VGpu>
  <Networking>Default</Networking>
  <PrinterRedirection>Enable</PrinterRedirection>
  <ClipboardRedirection>Default</ClipboardRedirection>
  <MappedFolders>
    <MappedFolder>
      <HostFolder>C:\software</HostFolder>
      <SandboxFolder>C:\host_software</SandboxFolder>
      <ReadOnly>true</ReadOnly>
    </MappedFolder>
    <MappedFolder>
      <HostFolder>C:\software\WindowsSandbox</HostFolder>
      <SandboxFolder>C:\Users\WDAGUtilityAccount\Desktop\WindowsSandbox</SandboxFolder>
      <ReadOnly>true</ReadOnly>
    </MappedFolder>
    <MappedFolder>
      <HostFolder>D:\ssTEST\_AI_2023.06.05\AI</HostFolder>
      <SandboxFolder>C:\HOST_AI</SandboxFolder>
      <ReadOnly>true</ReadOnly>
    </MappedFolder>
    <MappedFolder>
      <HostFolder>D:\ssTEST\_AI_2023.06.05\000-tasmania-renamed</HostFolder>
      <SandboxFolder>C:\HOST_000-tasmania-renamed</SandboxFolder>
      <ReadOnly>true</ReadOnly>
    </MappedFolder>
  </MappedFolders>
  <LogonCommand>
    <Command>C:\host_software\WindowsSandbox\AI_Setup_v01.bat</Command>
  </LogonCommand>
</Configuration>

2. AI_Setup_v01.bat

@echo on
mkdir C:\AI
mkdir C:\000-tasmania-renamed
mkdir C:\TEMP

REM Create a Initialization bat to run after the sandbox has fired up, for the user to double-click on in the sandbox
set "f=%USERPROFILE%\Desktop\000-INIT.bat"
DEL "%f%">NUL 2>&1
echo @ECHO ON >> "%f%"
echo mkdir C:\AI >> "%f%"
echo mkdir C:\000-tasmania-renamed\ >> "%f%"
echo mkdir C:\TEMP >> "%f%"
echo CD C:\TEMP >> "%f%"
REM echo copy /Y C:\host_software\wget\wget.exe C:\AI\ >> "%f%"
echo xcopy C:\HOST_AI\*.* C:\AI\ /e /v /f /h /r /y >> "%f%"
echo xcopy C:\HOST_000-tasmania-renamed\*.* C:\000-tasmania-renamed\ /e /v /f /h /r /y >> "%f%"
echo pause >> "%f%"
echo goto :eof >> "%f%"

REM Create a .bat to save any .py .vpy .bat results of runnng AI back in the sandbox system onto the host system
REM won't work for the moment, as we set C:\AI to readonly 
set "f=%USERPROFILE%\Desktop\111-SAVE_AI.bat"
DEL "%f%">NUL 2>&1
echo @ECHO ON >> "%f%"
echo REM THE FOLLOWING IS TO SAVE AI .bat and .vpy back to the non-sandbox host>> "%f%"
echo copy /Y C:\AI\*.bat  C:\HOST_AI\ >> "%f%"
echo copy /Y C:\AI\*.py   C:\HOST_AI\ >> "%f%"
echo copy /Y C:\AI\*.vpy  C:\HOST_AI\ >> "%f%"
echo pause >> "%f%"
echo goto :eof >> "%f%"

REM Create a link on the sandbox desktop which is mapped to the read-only C:\SOFTWARE folder on the host system
REM which contains our set of downloaded software that we use all the time. ie make it available in the sandbox.
REM We rely on this for stuff like notepad++ ... it C:\SOFTWARE or any other mapped folders do not exist on the host, the sandbox won't start
set create_shortcut_SCRIPT="%TEMP%\create_HOST_software_desktop_link_%RANDOM%-%RANDOM%-%RANDOM%-%RANDOM%.vbs"
del %create_shortcut_SCRIPT%
echo Set oWS = WScript.CreateObject("WScript.Shell") >> %create_shortcut_SCRIPT%
echo sLinkFile = "%USERPROFILE%\Desktop\HOST software.lnk" >> %create_shortcut_SCRIPT%
echo Set oLink = oWS.CreateShortcut(sLinkFile) >> %create_shortcut_SCRIPT%
echo oLink.TargetPath = "C:\host_software\" >> %create_shortcut_SCRIPT%
echo oLink.Arguments = "" >> %create_shortcut_SCRIPT%
echo oLink.Description = "HOST software" >> %create_shortcut_SCRIPT%
echo 'oLink.HotKey = "" >> %create_shortcut_SCRIPT%
echo 'oLink.IconLocation = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WindowStyle = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WorkingDirectory = "" >> %create_shortcut_SCRIPT%
echo oLink.Save >> %create_shortcut_SCRIPT%
cscript /nologo %create_shortcut_SCRIPT%
del %create_shortcut_SCRIPT%

REM Create a link on the sandbox desktop which is mapped to our readonly D:\ssTEST\_AI_2023.06.05\AI folder on the host system
REM This is where we originally downloaded and extracted AI's lovely software on the host
set create_shortcut_SCRIPT="%TEMP%\create_HOST_AI_desktop_link_%RANDOM%-%RANDOM%-%RANDOM%-%RANDOM%.vbs"
del %create_shortcut_SCRIPT%
echo Set oWS = WScript.CreateObject("WScript.Shell") >> %create_shortcut_SCRIPT%
echo sLinkFile = "%USERPROFILE%\Desktop\HOST AI.lnk" >> %create_shortcut_SCRIPT%
echo Set oLink = oWS.CreateShortcut(sLinkFile) >> %create_shortcut_SCRIPT%
echo oLink.TargetPath = "C:\HOST_AI\" >> %create_shortcut_SCRIPT%
echo oLink.Arguments = "" >> %create_shortcut_SCRIPT%
echo oLink.Description = "HOST AI" >> %create_shortcut_SCRIPT%
echo 'oLink.HotKey = "" >> %create_shortcut_SCRIPT%
echo 'oLink.IconLocation = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WindowStyle = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WorkingDirectory = "" >> %create_shortcut_SCRIPT%
echo oLink.Save >> %create_shortcut_SCRIPT%
cscript /nologo %create_shortcut_SCRIPT%
del %create_shortcut_SCRIPT%

REM Create a link on the sandbox desktop which is mapped to our read-only D:\ssTEST\_AI_2023.06.05\000-tasmania-renamed folder on the host system
REM This is where our original images/videos are
set create_shortcut_SCRIPT="%TEMP%\create_HOST_000-tasmania-renamed_desktop_link_%RANDOM%-%RANDOM%-%RANDOM%-%RANDOM%.vbs"
del %create_shortcut_SCRIPT%
echo Set oWS = WScript.CreateObject("WScript.Shell") >> %create_shortcut_SCRIPT%
echo sLinkFile = "%USERPROFILE%\Desktop\HOST_000-tasmania-renamed.lnk" >> %create_shortcut_SCRIPT%
echo Set oLink = oWS.CreateShortcut(sLinkFile) >> %create_shortcut_SCRIPT%
echo oLink.TargetPath = "C:\HOST_000-tasmania-renamed\" >> %create_shortcut_SCRIPT%
echo oLink.Arguments = "" >> %create_shortcut_SCRIPT%
echo oLink.Description = "HOST_000-tasmania-renamed" >> %create_shortcut_SCRIPT%
echo 'oLink.HotKey = "" >> %create_shortcut_SCRIPT%
echo 'oLink.IconLocation = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WindowStyle = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WorkingDirectory = "" >> %create_shortcut_SCRIPT%
echo oLink.Save >> %create_shortcut_SCRIPT%
cscript /nologo %create_shortcut_SCRIPT%
del %create_shortcut_SCRIPT%

REM Create a link on the sandbox desktop which is mapped to C:\AI in the local sandbox system
REM This is where our sandbox copies of original folders of AI software is
set create_shortcut_SCRIPT="%TEMP%\create_AI_desktop_link_%RANDOM%-%RANDOM%-%RANDOM%-%RANDOM%.vbs"
del %create_shortcut_SCRIPT%
echo Set oWS = WScript.CreateObject("WScript.Shell") >> %create_shortcut_SCRIPT%
echo sLinkFile = "%USERPROFILE%\Desktop\AI.lnk" >> %create_shortcut_SCRIPT%
echo Set oLink = oWS.CreateShortcut(sLinkFile) >> %create_shortcut_SCRIPT%
echo oLink.TargetPath = "C:\AI\" >> %create_shortcut_SCRIPT%
echo oLink.Arguments = "" >> %create_shortcut_SCRIPT%
echo oLink.Description = "AI" >> %create_shortcut_SCRIPT%
echo 'oLink.HotKey = "" >> %create_shortcut_SCRIPT%
echo 'oLink.IconLocation = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WindowStyle = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WorkingDirectory = "" >> %create_shortcut_SCRIPT%
echo oLink.Save >> %create_shortcut_SCRIPT%
cscript /nologo %create_shortcut_SCRIPT%
del %create_shortcut_SCRIPT%

REM Create a link on the sandbox desktop which is mapped to "C:\000-tasmania-renamed in the local sandbox system
REM This is where our sandbox copies of original folders of images are
set create_shortcut_SCRIPT="%TEMP%\create_000-tasmania-renamed_desktop_link_%RANDOM%-%RANDOM%-%RANDOM%-%RANDOM%.vbs"
del %create_shortcut_SCRIPT%
echo Set oWS = WScript.CreateObject("WScript.Shell") >> %create_shortcut_SCRIPT%
echo sLinkFile = "%USERPROFILE%\Desktop\000-tasmania-renamed.lnk" >> %create_shortcut_SCRIPT%
echo Set oLink = oWS.CreateShortcut(sLinkFile) >> %create_shortcut_SCRIPT%
echo oLink.TargetPath = "C:\000-tasmania-renamed\" >> %create_shortcut_SCRIPT%
echo oLink.Arguments = "" >> %create_shortcut_SCRIPT%
echo oLink.Description = "000-tasmania-renamed" >> %create_shortcut_SCRIPT%
echo 'oLink.HotKey = "" >> %create_shortcut_SCRIPT%
echo 'oLink.IconLocation = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WindowStyle = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WorkingDirectory = "" >> %create_shortcut_SCRIPT%
echo oLink.Save >> %create_shortcut_SCRIPT%
cscript /nologo %create_shortcut_SCRIPT%
del %create_shortcut_SCRIPT%

REM Create a link on the sandbox desktop which is mapped to a local TEMP folder in the local sandbox system
set create_shortcut_SCRIPT="%TEMP%\create_TEMP_desktop_link_%RANDOM%-%RANDOM%-%RANDOM%-%RANDOM%.vbs"
del %create_shortcut_SCRIPT%
echo Set oWS = WScript.CreateObject("WScript.Shell") >> %create_shortcut_SCRIPT%
echo sLinkFile = "%USERPROFILE%\Desktop\TEMP.lnk" >> %create_shortcut_SCRIPT%
echo Set oLink = oWS.CreateShortcut(sLinkFile) >> %create_shortcut_SCRIPT%
echo oLink.TargetPath = "C:\TEMP\" >> %create_shortcut_SCRIPT%
echo oLink.Arguments = "" >> %create_shortcut_SCRIPT%
echo oLink.Description = "TEMP" >> %create_shortcut_SCRIPT%
echo 'oLink.HotKey = "" >> %create_shortcut_SCRIPT%
echo 'oLink.IconLocation = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WindowStyle = "" >> %create_shortcut_SCRIPT%
echo 'oLink.WorkingDirectory = "" >> %create_shortcut_SCRIPT%
echo oLink.Save >> %create_shortcut_SCRIPT%
cscript /nologo %create_shortcut_SCRIPT%
del %create_shortcut_SCRIPT%

REM change explorer View settings to be as we like them
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V AlwaysShowMenus /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V SeparateProcess /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V NavPaneExpandToCurrentFolder /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V NavPaneShowAllFolders /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V HideFileExt /T REG_DWORD /D 00000000 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V Hidden /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V ShowSuperHidden /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V ShowEncryptCompressedColor /T REG_DWORD /D 00000001 /F
REG Add HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced /V ShowStatusBar /T REG_DWORD /D 00000001 /F

REM add "Edit with Notepad++" to right click context pop-up
REM ... Notepad++ is located in the host's SOFTWARE\NPP folder
set fnpp="C:\TEMP\EDIT_WITH_NPP.REG"
del %fnpp%
ECHO Windows Registry Editor Version 5.00                     > %fnpp%
ECHO.                                                        >> %fnpp%
ECHO [HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced] >> %fnpp%
ECHO "UseCompactMode"=dword:00000001 >> %fnpp%
ECHO [HKEY_CLASSES_ROOT\*\shell\Edit with Notepad++]         >> %fnpp%
REM ECHO "Icon"="C:\\SOFTWARE\\NPP\\notepad++.exe"
ECHO "Icon"="C:\\HOST_SOFTWARE\\NPP\\notepad++.exe"               >> %fnpp%
ECHO [HKEY_CLASSES_ROOT\*\shell\Edit with Notepad++\command] >> %fnpp%
REM ECHO @="\"C:\\SOFTWARE\\NPP\\notepad++.exe\" \"%%1\""
ECHO @="\"C:\\HOST_SOFTWARE\\NPP\\notepad++.exe\" \"%%1\""        >> %fnpp%
regedit /s %fnpp%

REM restart explorer so that eveything appears including on the desktop
taskkill /f /im explorer.exe
start explorer.exe

REM start an explorer window at the specified directory
explorer.exe "C:\AI\"

REM start an explorer window at the specified directory
explorer.exe "C:\000-tasmania-renamed\"

goto :eof