On x264's Handling regards Consecutive Duplicate Frames

[gdgsdg123]

Rough Preview of the Sample

The sample serves as an adequate example for a demonstration of the subject:
Consecutive duplicate frames for 3:32.56 --> 5:04 (the end), usual content for the rest of the video.


The term "last frame" below means: A single frame from those consecutive duplicates during 3:32.56 --> 5:04 in the above sample.





Below are some file sizes pertaining to the demo, which may shed some light on the subject:
(Tool: FFmpeg 4.1.4)

#Lossless (rgb24), keyint=infinite
30126520439 Bytes:

ffmpeg -i INPUT_rgb24 -an -c:v libx264rgb -preset placebo -qp 0 -x264-params "keyint=infinite:no-deblock=1" "Lossless_keyint=infinite.mkv"

30126201980 Bytes:

ffmpeg -i INPUT_rgb24_TRIM -an -c:v libx264rgb -preset placebo -qp 0 -x264-params "keyint=infinite:no-deblock=1" "TRIM_Lossless_keyint=infinite.mkv"

30126520439-30126201980 = 318459 Bytes


#Lossless (rgb24), keyint=15
#Note: Missing the last frame in the TRIM.
31362559339 Bytes:

ffmpeg -i INPUT_rgb24 -an -c:v libx264rgb -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" "Lossless_keyint=15.mkv"

30730918362 Bytes:

ffmpeg -i !INPUT_rgb24_TRIM -an -c:v libx264rgb -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" "!TRIM_Lossless_keyint=15.mkv"

31362559339-(30730918362+7683697) = 623957280 Bytes



#PNG (last frame)
7683697 Bytes:

ffmpeg -ss 3:32.56 -i INPUT_rgb24_TRIM -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+accurate_rnd+full_chroma_int "PNG.png"

(workaround as of `-sseof 0` failed to work...)



#Near-Lossless (yuv420p), keyint=150
1272631389 Bytes:

ffmpeg -i INPUT_rgb24 -an -c:v libx264 -preset placebo -crf 19 -x264-params "level=5.1:keyint=150:psy-rd=2,1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "Near-Lossless_keyint=150.mp4"

1229081917 Bytes:

ffmpeg -i INPUT_rgb24_TRIM -an -c:v libx264 -preset placebo -crf 19 -x264-params "level=5.1:keyint=150:psy-rd=2,1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "TRIM_Near-Lossless_keyint=150.mp4"

1272631389-1229081917 = 43549472 Bytes


#Near-Lossless (yuv420p), keyint=15
1637464899 Bytes:

ffmpeg -i INPUT_rgb24 -an -c:v libx264 -preset placebo -crf 19 -x264-params "level=5.1:keyint=15:psy-rd=2,1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "Near-Lossless_keyint=15.mp4"

1377592853 Bytes:

ffmpeg -i INPUT_rgb24_TRIM -an -c:v libx264 -preset placebo -crf 19 -x264-params "level=5.1:keyint=15:psy-rd=2,1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "TRIM_Near-Lossless_keyint=15.mp4"

1637464899-1377592853 = 259872046 Bytes



Note

The "last frame"s are at 15 FPS in the Lossless, but at 30 FPS in the Near-Lossless... (thanks to my inappropriate handling, imposing `-vsync vfr` should will fix this... haven't tried though)
Though it might not make any significant difference at all... (VFR, CFR can be essentially the same thing... with only the timestamps differ)

But who knows?..
Sadly... that's not the case. (which means, that cfr/vfr in the case will have an obvious impact on the output size of materials with such characteristics)





Conclusion

Absurd, isn't it?
Bewildered too at first... But then soon sort of understood its design language behind this seeming absurdity.
(To be continued...)







Off-Topic

And if you wouldn't mind... please do me a favor:
How to trick the mpdecimate to drop those exactly identical frames but not the nearly identical ones?

...To save me a few minutes' blind testing (as you see in the docs... meh).
Like this:

-vf "mpdecimate=0:0:0"

-> have you tried setting low, high and frac to zero? Sounds to me like the filter then should drop all consecutive bit identical frames and only those.

You're so right! ...Except that `frac` doesn't matter at all. (in this case... well, technically it does, though likely unnoticeable)

According to the source, and an answer on Stack Exchange:

As mpdecimate explains, one 8x8 block where the sum of differences exceeds `hi` will result in an output frame (no drop). Same when more than `frac` fraction of the 8x8 blocks have more than `lo` difference. So if there's a blinking cursor, that will trigger `hi` every time unless you crank it all the way up and just depend on `lo`.

Further Off-Topic

Be careful with the `-vsync cfr` option (...and take care that the option can be inexplicitly enabled), it may alter the playback experience by duplicating the wrong frames.



When using `mpdecimate` excessively (like having the whole 3:32.56 --> 5:04 in the aforementioned sample as a single frame...), the output tends to report a wrong duration, while the PTS seemed to remain correct. (which means the playback progress could look like: 00:05:04.00 / 00:03:32.96)

Note: I added (must add?) a single "Black Screen" frame (or whatever frames) here right after the last frame just for the testing.

* For the "duration" of frames may not actually exist...



Take extra caution when having the framerate below 15 FPS in any part of the video.
Anything below 15 FPS seems to have serious playback issues in quite an amount of players... (problematic seeking, frozen video, etc.)

* More accurately, VLC-like players. (and likely a VLC exclusive long time known issue...)

Reference:
https://forum.videolan.org/viewtopic.php?t=107770
https://forum.videolan.org/viewtopic.php?t=123887
https://stackoverflow.com/a/39213083





Similar behavior also in VP9.

Check the output of `youtube-dl`:

C:\>youtube-dl -F https://www.youtube.com/watch?v=8fyTbI4TIS0
[youtube] 8fyTbI4TIS0: Downloading webpage
[youtube] 8fyTbI4TIS0: Downloading video info webpage
[info] Available formats for 8fyTbI4TIS0:
format code  extension  resolution note
249          webm       audio only DASH audio   64k , opus @ 50k, 1.61MiB
250          webm       audio only DASH audio   83k , opus @ 70k, 2.11MiB
140          m4a        audio only DASH audio  130k , m4a_dash container, mp4a.40.2@128k, 3.65MiB
251          webm       audio only DASH audio  162k , opus @160k, 4.16MiB
160          mp4        256x108    144p   31k , avc1.4d400b, 15fps, video only, 567.07KiB
278          webm       256x108    144p   45k , webm container, vp9, 15fps, video only, 696.57KiB
133          mp4        426x180    240p   68k , avc1.4d400c, 15fps, video only, 1.21MiB
242          webm       426x180    240p   83k , vp9, 15fps, video only, 1.07MiB
243          webm       640x270    360p  122k , vp9, 15fps, video only, 1.59MiB
134          mp4        640x270    360p  160k , avc1.4d4015, 15fps, video only, 2.22MiB
244          webm       854x360    480p  186k , vp9, 15fps, video only, 2.71MiB
135          mp4        854x360    480p  276k , avc1.4d4016, 15fps, video only, 3.82MiB
247          webm       1280x540   720p  319k , vp9, 15fps, video only, 4.21MiB
136          mp4        1280x540   720p  523k , avc1.4d401f, 15fps, video only, 6.80MiB
248          webm       1920x810   1080p  655k , vp9, 15fps, video only, 8.57MiB
271          webm       2560x1080  1440p  900k , vp9, 15fps, video only, 14.08MiB
137          mp4        1920x810   1080p 1091k , avc1.640028, 15fps, video only, 11.08MiB
313          webm       3840x1620  2160p 1749k , vp9, 15fps, video only, 28.65MiB
272          webm       5120x2160  2880p 5291k , vp9, 15fps, video only, 51.15MiB
18           mp4        640x270    medium  199k , avc1.42001E, mp4a.40.2@ 96k (44100Hz), 5.63MiB (best)

The original file (what I threw at YouTube):

ffmpeg -loop 1 -i "Entrepreneurship.png" -i "1.flac" -c:a copy -c:v libx264 -r 15 -vframes 3548 -preset placebo -qp 0 -x264-params "level=5.1:keyint=infinite:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "1.mkv"

The output ("1.mkv") is of 23.37 MiB, in which 18.88 MiB is the FLAC audio stream... The video stream is of only 4.48 MiB.



Strange... it ends in audio truncation near the EOF (End of File).

C:\>ffprobe "1.mkv"
ffprobe version 4.1.4 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 9.1.1 (GCC) 20190716
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, matroska,webm, from '1.mkv':
  Metadata:
    ENCODER         : Lavf58.20.100
  Duration: 00:03:56.53, start: 0.000000, bitrate: 822 kb/s
    Stream #0:0: Video: h264 (High 4:4:4 Predictive), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 5120x2160, 15 fps, 15 tbr, 1k tbn, 30 tbc (default)
    Metadata:
      ENCODER         : Lavc58.35.100 libx264
      DURATION        : 00:03:56.534000000
    Stream #0:1: Audio: flac, 48000 Hz, stereo, s16 (default)
    Metadata:
      DURATION        : 00:03:54.912000000

C:\>ffprobe -hide_banner "1.flac"
Input #0, flac, from '1.flac':
  Metadata:
    ENCODER         : Lavf58.20.100
  Duration: 00:03:56.57, start: 0.000000, bitrate: 669 kb/s
    Stream #0:0: Audio: flac, 48000 Hz, stereo, s16

Has to be done in a separated manner:
(...in order to have things worked normally)

ffmpeg -loop 1 -i "Entrepreneurship.png" -c:v libx264 -r 15 -vframes 3548 -preset placebo -qp 0 -x264-params "level=5.1:keyint=infinite:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "TEMP_v.mkv"
ffmpeg -i "TEMP_v.mkv" -i "1.flac" -c copy "1.mkv"

And similar things also for FFmpeg 4.2...

Ticket of the Issue



* FYI, the best command for preparing music uploading for YouTube is:
(taken the above case as example)

ffmpeg -i "Entrepreneurship.png" -i "1.flac" -c:a copy -c:v libx264 -r 1/236.571542 -preset placebo -qp 0 -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int+bitexact -vsync vfr -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 -flags +bitexact -fflags +bitexact -map_metadata -1 "1.mkv"

[Selur]

How to trick the mpdecimate to drop those exactly identical frames but not the nearly identical ones?

looking at the definition you linked to:

Drop frames that do not differ greatly from the previous frame in order to reduce frame rate.

The main use of this filter is for very-low-bitrate encoding (e.g. streaming over dialup modem), but it could in theory be used for fixing movies that were inverse-telecined incorrectly.

A description of the accepted options follows.
max
Set the maximum number of consecutive frames which can be dropped (if positive), or the minimum interval between dropped frames (if negative). If the value is 0, the frame is dropped disregarding the number of previous sequentially dropped frames.
Default value is 0.
hi
lo
frac
Set the dropping threshold values.
Values for hi and lo are for 8x8 pixel blocks and represent actual pixel value differences, so a threshold of 64 corresponds to 1 unit of difference for each pixel, or the same spread out differently over the block.
A frame is a candidate for dropping if no 8x8 blocks differ by more than a threshold of hi, and if no more than frac blocks (1 meaning the whole image) differ by more than a threshold of lo.
Default value for hi is 64*12, default value for lo is 64*5, and default value for frac is 0.33.

-> have you tried setting low, high and frac to zero? Sounds to me like the filter then should drop all consecutive bit identical frames and only those.

[johnmeyer]

While I don't understand what you are trying to do, the usual tool I use for finding and then eliminating duplicate fields or duplicate frames is the TFM/TDecimate combination. It has a myriad of settings that let you find and then eliminate duplicates. However, it is generally best suited for situations where the duplicates come at fairly regular intervals, such as the dups found in telecined material. However, I have used it for many other situations where the pattern of duplicates is obscure or non-existent.

An alternative is to simply construct your own duplicate detection filter using built-in AVISynth functions, such as the YDifference functions, or some of the functions found in RTStats and similar libraries. This is actually quite easy to do IF your duplicates are, in fact, 100.0000% identical. When this is the case, even the simple YDifference function will return 0.000, whereas any other set of adjacent frames will return a non-zero result. You then either output the frame numbers of the duplicate frames and feed those to a decimation filter, often done as a second step, or do more work and try to decimate them in one pass.

Just search this forum for "YDifferenceFromPrevious" or "YDifferenceToNext". If you search on my user name you'll find dozens of scripts that show how to use this in conjunction with conditional logic to take some action when this metric differs greatly from the metrics from adjacent frames; when it blows up; or when, as in this case, goes to a mathematically pure zero.

[gdgsdg123]

-> have you tried setting low, high and frac to zero? Sounds to me like the filter then should drop all consecutive bit identical frames and only those.

You're so right! ...Except that `frac` doesn't matter at all. (in this case... well, technically it does, though likely unnoticeable)

According to the source, and an answer on Stack Exchange:

As mpdecimate explains, one 8x8 block where the sum of differences exceeds `hi` will result in an output frame (no drop). Same when more than `frac` fraction of the 8x8 blocks have more than `lo` difference. So if there's a blinking cursor, that will trigger `hi` every time unless you crank it all the way up and just depend on `lo`.