ffmpeg framcount differs from real numbers

[Fabian Schmidt]

Dear Forum!

I am currently using ffmpeg in order to produce screenshots of each new scene in archival footage. The name of the screenshot is supposed to include the framcount and I want to make a screenshot 3 frames after the cut so my commandline looks like this:

ffmpeg -i WESTERBORK_2019_progres.mov -filter:v "select=if(gt(scene\,0.13)\,st(1\,n)-n\,if(ld(1)\,eq(n\,ld(1)+3))),showinfo" -vsync vfr Westerbork_Film_%05d.jpg

Even though this worked the last two films with my current material I get a stange output. The timecode in the name is roughly 4% higher that the one I burnt in to the videofile (for orientation). Examples:


Westerbork_Film_6479.jpg (showing a frame count of 6739)

https://ibb.co/9qgvqzS

Westerbork_Film_2171.jpg: (showing a frame count of 2259)

https://ibb.co/jWsSynJ

I exported some seconds from the beginning as mp4 and uploaded it here: https://we.tl/t-ktK7C1NVVM

This happens with every version (I remuxed the file in mkv toolnix and speded it down to 16 fps and to 25fps) I made. So, even though I doubt it will help, here’s the media info of the file. It’s a silent film; the file is an export from Davinci resolve where I added the frame count:


General
Complete name : WESTERBORK_2019_progres.mov
Format : MPEG-4
Format profile : QuickTime
Codec ID : qt 0000.02 (qt )
File size : 8.90 GiB
Duration : 1 h 12 min
Overall bit rate mode : Variable
Overall bit rate : 17.6 Mb/s
Encoded date : UTC 2021-06-04 18:51:26
Tagged date : UTC 2021-06-04 18:51:26
Writing application : Blackmagic Design DaVinci Resolve

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 1 h 12 min
Bit rate mode : Variable
Bit rate : 17.3 Mb/s
Maximum bit rate : 768 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.333
Stream size : 8.74 GiB (98%)
Encoded date : UTC 2021-06-04 18:51:26
Tagged date : UTC 2021-06-04 18:51:26
Color range : Limited
Color primaries : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 1 h 12 min
Bit rate mode : Constant
Bit rate : 320 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 166 MiB (2%)
Default : Yes
Alternate group : 1
Encoded date : UTC 2021-06-04 18:51:26
Tagged date : UTC 2021-06-04 18:51:26

Other
ID : 3
Type : Time code
Format : QuickTime TC
Duration : 1 h 12 min
Frame rate : 25.000 FPS
Time code of first frame : 01:00:00:00
Time code, striped : Yes
Language : English
Default : No
Encoded date : UTC 2021-06-04 18:51:26
Tagged date : UTC 2021-06-04 18:51:26


Looking forward to hear any tips or comments. Also one more question to the experts: Sometimes ffmepg puts out the frame count in the name with 000 in front and sometimes not. I guess it has something to do with the %05d or %d in the end. Where can I find an explanation how this works?

Kind regards
Fabian

[Selur]

Hope this helps:
in short: The syntax %0Xd specifies to use a decimal number composed of X digits padded with zeroes to express the sequence number.

a bit longer:
X is the minimum number of characters to be printed.
If the value to be printed is shorter than this number, the result is padded with zero. The value is not truncated even if the result is larger.

%05d -> pad with zero to archive a length of 5
%d -> no zero padding
%7d -> pad with blank to archive a length of 7

Where can I find an explanation how this works?

it's the same syntax as printf in C programming language, so looking for printf documentation should work.

Cu Selur

[Fabian Schmidt]

Thanks for clarifying!

Now I only need an answer to why the framcount is larger than it actually should be

[Selur]

does

ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 "PATH to Input file"

report the right number?
Only thing I find strange is

...
Duration : 1 h 12 min
..
Bit rate : 17.3 Mb/s
Maximum bit rate : 768 kb/s
..
Stream size : 8.74 GiB (98%)

the maximum bit rate.

My first guess would be that the problem lies with the QuickTime TC stream,..

[Fabian Schmidt]

ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 /path/to/filename

puts out: 108790 (it counts the entirety of frames in that movie, right?), which is a little lower than the number of frames that the burnt in framecount shows.

Can I change the "quicktime" aspect, and if, how?

[Selur]

it counts the entirety of frames in that movie, right?

yes

which is a little lower than the number of frames that the burnt in framecount shows.

the count should be one number below it assuming the frame numbers start at 0, not more.

Can I change the "quicktime" aspect, and if, how?

remuxing the file with mkvtoolnix to an mkv and ignoring the time code stream is on option to get rid of the stream.

[Fabian Schmidt]

remuxing the file with mkvtoolnix to an mkv and ignoring the time code stream is on option to get rid of the stream.

how can I achieve this, I mean ignoring the time code stream?

[Selur]

if you open the file with mkvtoolnix it should show the stream, simply disabling and saving the new file should work,..

[Fabian Schmidt]

if you open the file with mkvtoolnix it should show the stream, simply disabling and saving the new file should work,..

All of this didn't work (there's no such stream that could be disabled). I also reencoded the video once more to mp4 - still ffmpeg framecount differs from the real tc.

Would anyone perhaps kindly take a look at the file? I could upload a snippet somewhere.

[emcodem]

If you can upload it, please do so but only a snipped might be too less, except the snippet contains one point where a frame gets lost for you. I'd start by checking whats the first missing frame anyways and try to extract a very small portion of the original file that contains the problematic spot using -ss, -t and codec copy.

What about the -vsync vfr option, isn't that supposed to throw away frames and thus could potentially lead to your numbers mismatch? Why do you supply this option in anyways?
I'd also recommend to use -loglevel debug and check if it says anything about dropping there.

Another also: you might also want to check out the #ffmpeg on libera irc chat.

[Fabian Schmidt]

I don't think that any frames are dropped. The reason must be something else. Here's a piece of the file that produces the same results.

https://we.tl/t-03TKBlJlWC

and here again my current commandline:

ffmpeg -i WESTERBORK_2019_progres.mp4 -filter:v "select=if(gt(scene\,0.25)\,st(1\,n)-n\,if(ld(1)\,eq(n\,ld(1)+3))),showinfo" -vsync 0 -frame_pts 1 Westerbork_%05d.jpg

[emcodem]

From feeling everything works as expected here, but the watermarked white counter bottom left in the original video just do not count up consecutively.
e.g. frame number 15 is missing completele in the original video, i did not see it using ffmpeg decoder nor chromes builtin html5 decoder.
Using -vsync drop, at least the filenames of the first 14 pictures match the count in the video

[jagabo]

The numbers burned into the frames are missing every 26th number, starting with 15, 41, 67, 93... There are a total of 1477 frames, 0 to 1476. So it makes sense that the last frame has the number 1536 burned into it but is really frame number 1476. ffprobe shows the timecodes increase monotonically at 40 ms each. So it makes sense that the last frame (1476) has a timestamp of 59.04 seconds (1476 * 0.04 = 59.04).

[Fabian Schmidt]

how stupid. thanks for the heads up. will make a new framecount.

[jagabo]

By the way, the video appears to be missing lots of frames too. I'd estimate something like 20 to 30 percent of the original film frames are missing.

[Fabian Schmidt]

By the way, the video appears to be missing lots of frames too. I'd estimate something like 20 to 30 percent of the original film frames are missing.

How did you detect possibly missing frames? 20 - 30% missing within one shot would lead to massive stuttering and jumping of the image. I don't see any of that. It's 16mm footage shot with 16 frames per second btw.

[jagabo]

Open the video in an editor and step through frame by frame. You'll see obvious jumps in motion between some frames. For example, there's a jump between frames 15 and 16, 18 and 19, 32 and 33, 35 and 36... Look at the panning shot that starts around frame 750. It's obviously missing many frames. Some of that may just be jerky panning but I'm pretty sure much of it missing frames.

[Fabian Schmidt]

Open the video in an editor and step through frame by frame. You'll see obvious jumps in motion between some frames. For example, there's a jump between frames 15 and 16, 18 and 19, 32 and 33, 35 and 36... Look at the panning shot that starts around frame 750. It's obviously missing many frames. Some of that may just be jerky panning but I'm pretty sure much of it missing frames.

OK, I compared this one to an older scan I have and indeed some frames are missing. Have asked for a new file from the archive without duplicate frames. Thanks again.

[jagabo]

I wouldn't worry about duplicate frames in your source. That's much easier to deal with.

[Fabian Schmidt]

The original file cam with every second frame duplicated. I tried to get rid of these duplicates with decimate=cycle=2 but it seems to have gone wrong.

[Fabian Schmidt]

I wouldn't worry about duplicate frames in your source. That's much easier to deal with.

I am very interested to find out how it could be achieved to get rid of the dupe frames in the original without loosing any frames as it seemed to have happened during my first attempt (also to learn what went wrong). Here's a (short and small) sample of the file:

https://we.tl/t-b4IZENVBAQ

[poisondeathray]

The original file cam with every second frame duplicated. I tried to get rid of these duplicates with decimate=cycle=2 but it seems to have gone wrong.

I wouldn't worry about duplicate frames in your source. That's much easier to deal with.

I am very interested to find out how it could be achieved to get rid of the dupe frames in the original without loosing any frames as it seemed to have happened during my first attempt (also to learn what went wrong). Here's a (short and small) sample of the file:

https://we.tl/t-b4IZENVBAQ

But on this sample it's 1 in 3, not every second frame.

[Fabian Schmidt]

122344566 is the pattern, I thought that is described well as: every second frams is duplicate. But I may mistaken here. So, how can I get rid of the duplicates?

[poisondeathray]

Every group of 3 frames has 1 duplicate. So your decimation cycle is 3. (-vf decimate=cycle=3) . That will leave you a fps of 16.666... with unique frames

(You can use something like mpv and use the "." (period key) to frame step)

[jagabo]

This apears to have worked perfectly:

ffmpeg -i WESTERBOK_final_edit_restoration_25fps_sample.mp4 -vf decimate=cycle=3 outpu.mkv

I didn't check every single frame but a spot check showed no duplicates and no jerks.

TDecimate(Cycle=3, CycleR=1) in AviSynth gave exactly the same result.

Of course, that leaves you with a 16.67 fps video.

[Fabian Schmidt]

thank you very much. Will check it out right away!

[Fabian Schmidt]

This apears to have worked perfectly:

ffmpeg -i WESTERBOK_final_edit_restoration_25fps_sample.mp4 -vf decimate=cycle=3 outpu.mkv

I didn't check every single frame but a spot check showed no duplicates and no jerks.

TDecimate(Cycle=3, CycleR=1) in AviSynth gave exactly the same result.

Of course, that leaves you with a 16.67 fps video.

OK, that worked well, now I want to add (=burn in) a framecount in DaVinci resolve, but I run into problems now with 16,66666 fps. Also, I need a mp4 or mov to import into DaVinci. I already altered the command line towards mp4, but the fps problem is still there. Do I have to re-encode the mp4 with 25 fps in order to be able to add a framecount in DaVinci? Or is there another solution?

[jagabo]

You can add frame numbers while decimating with ffmpeg:

ffmpeg -i input.mp4 -vf decimate=cycle=3,drawtext=fontfile=C\\:/Windows/fonts/arial.ttf:fontsize=30:fontcolor=white:text=''%{frame_num}':bordercolor=black:borderw=2:x=20:y=20 output.mp4

[Fabian Schmidt]

You can add frame numbers while decimating with ffmpeg:

ffmpeg -i input.mp4 -vf decimate=cycle=3,drawtext=fontfile=C\\:/Windows/fonts/arial.ttf:fontsize=30:fontcolor=white:text=''%{frame_num}':bordercolor=black:borderw=2:x=20:y=20 output.mp4

no matter what I try the output is always

quote>

OK, I found the reason. Deleted the "'" after "frame_num}". Thanks again!

[jagabo]

Oops, sorry about the typo.