Removing consecutive duplicate frames

[marius2du]

Hello,

I have a 60 fps source video in which some sections appear to be with duplicate frames.
I am using the following command to "cleanup" the parts with duplicate frames and to keep audio in sync:

ffmpeg -i input.mp4 -vf mpdecimate -vsync vfr out.mp4

When I run resulting video, the player will speed up at the section where duplicate frames where removed.
What am I doing wrong ?
Thank you.

[Selur]

Probably nothing, mpdecimate is a video only filter, it does not touch the audio at all.
There are projects like https://github.com/KenjiTakahashi/mpdecimate_trim which try to keep the audio sync, but afaik. there is no option for this directly in ffmpeg.

Cu Selur

[jagabo]

Maybe your player doesn't handle variable frame rate properly?

[marius2du]

Maybe your player doesn't handle variable frame rate properly?

I am using Media Player Classic or VLC. Same thing happens with both - video starts playing faster at the point where duplicate frames were removed.

[rgr]

Hello,

I have a 60 fps source video in which some sections appear to be with duplicate frames.
I am using the following command to "cleanup" the parts with duplicate frames and to keep audio in sync:

ffmpeg -i input.mp4 -vf mpdecimate -vsync vfr out.mp4

When I run resulting video, the player will speed up at the section where duplicate frames where removed.
What am I doing wrong ?
Thank you.

Can you send the link? (privately possible)
I want to test my VFR to CFR conversion script with duplicate frame removal.

[marius2du]

@rgr Please check your inbox. I have sent you both, the source file and the resulting file with duplicate frames being removed.

[jagabo]

That ffmpeg command line produces a variable frame rate video. It shouldn't play any smoother than the original because the timecodes play the remaining frames at exactly the same time as in the original file. So for example, one second worth of totally black frames might be reduced to a single black frame. But that single black frame will be displayed for one second, just like in the original video.

Some players (especially editors) can't handle VFR properly and will play the video at the average frame rate of the final video. That one black frame will be displayed for the same duration as all the other frames. That is probably what you're seeing.

[rgr]

@rgr Please check your inbox. I have sent you both, the source file and the resulting file with duplicate frames being removed.

This is a CFR file, not a VFR file.
This is probably not the original file; someone has already converted it.
Part of the clip is 60fps, part is 30fps with duplicate frames.
Nevertheless, I'll try converting it and let you know.

By removing duplicate frames, you've sped up the video because there are now fewer of them. The video is artificially sped up at these points and therefore finishes sooner than the audio.

[rgr]

@rgr Please check your inbox. I have sent you both, the source file and the resulting file with duplicate frames being removed.

Check inbox. Duplicate frames were interpolated from adjacent ones.

[rgr]

That ffmpeg command line produces a variable frame rate video. It shouldn't play any smoother than the original because the timecodes play the remaining frames at exactly the same time as in the original file. So for example, one second worth of totally black frames might be reduced to a single black frame. But that single black frame will be displayed for one second, just like in the original video.

Some players (especially editors) can't handle VFR properly and will play the video at the average frame rate of the final video. That one black frame will be displayed for the same duration as all the other frames. That is probably what you're seeing.

Well, something didn't work because CFR came out.

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.2
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference fra : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 9 s 944 ms
Bit rate : 6 455 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 59.940 (60000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.052
Stream size : 7.65 MiB (97%)
Writing library : x264 core 144 r2525 40bb568
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 12 s 791 ms
Duration_LastFrame : -9 ms
Bit rate mode : Constant
Bit rate : 128 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 200 KiB (2%)
Default : Yes
Alternate group : 1

Edit:
But out of curiosity I checked it with ffmpeg8 and it turned out OK. Of course, this doesn't actually improve anything.

Duration : 12 s 730 ms
Source duration : 12 s 763 ms
Frame rate mode : Variable

[jagabo]

Well, something didn't work because CFR came out.

I think you left out the "-vsync vfr" on the ffmpeg command line. Without it, I believe mpdecimate outputs VFR but ffmpeg converts it back to CFR by adding back duplicate frames.

[rgr]

Well, something didn't work because CFR came out.

I think you left out the "-vsync vfr" on the ffmpeg command line. Without it, I believe mpdecimate outputs VFR but ffmpeg converts it back to CFR by adding back duplicate frames.

I evaluated the output file provided by the author. My attempt is in the post above (Edit section).

[marius2du]

That ffmpeg command line produces a variable frame rate video. It shouldn't play any smoother than the original because the timecodes play the remaining frames at exactly the same time as in the original file. So for example, one second worth of totally black frames might be reduced to a single black frame. But that single black frame will be displayed for one second, just like in the original video.

Some players (especially editors) can't handle VFR properly and will play the video at the average frame rate of the final video. That one black frame will be displayed for the same duration as all the other frames. That is probably what you're seeing.

Well, something didn't work because CFR came out.

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.2
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference fra : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 9 s 944 ms
Bit rate : 6 455 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 59.940 (60000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.052
Stream size : 7.65 MiB (97%)
Writing library : x264 core 144 r2525 40bb568
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 12 s 791 ms
Duration_LastFrame : -9 ms
Bit rate mode : Constant
Bit rate : 128 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 200 KiB (2%)
Default : Yes
Alternate group : 1

Edit:
But out of curiosity I checked it with ffmpeg8 and it turned out OK. Of course, this doesn't actually improve anything.

Duration : 12 s 730 ms
Source duration : 12 s 763 ms
Frame rate mode : Variable

Interestingly, MediaInfo reports for me VFR. See below:

Video
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.2
Format settings, CABAC           : Yes
Format settings, ReFrames        : 4 frames
Codec ID                         : avc1
Duration                         : 9s 943ms
Bit rate mode                    : Variable
Bit rate                         : 6 455 Kbps
Width                            : 1 920 pixels
Height                           : 1 080 pixels
Display aspect ratio             : 16/9
Frame rate mode                  : Variable
Frame rate                       : 59.940 fps
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.052
Stream size                      : 7.65 MiB (97%)
Writing library                  : x264 core 144 r2525 40bb568
Encoding settings                : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Audio
Format                           : AAC
Format/Info                      : Advanced Audio Codec
Format version                   : Version 4
Format profile                   : LC
Format settings, SBR             : No
Codec ID                         : 40
Duration                         : 12s 790ms
Bit rate mode                    : Constant
Bit rate                         : 128 Kbps
Channel(s)                       : 2 channels
Channel positions                : L R
Sampling rate                    : 48.0 KHz
Resolution                       : 16 bits
Stream size                      : 200 KiB (2%)

[jagabo]

Interestingly, MediaInfo reports for me VFR. See below:

Video
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.2
Format settings, CABAC           : Yes
Format settings, ReFrames        : 4 frames
Codec ID                         : avc1
Duration                         : 9s 943ms
Bit rate mode                    : Variable
Bit rate                         : 6 455 Kbps
Width                            : 1 920 pixels
Height                           : 1 080 pixels
Display aspect ratio             : 16/9
Frame rate mode                  : Variable
Frame rate                       : 59.940 fps
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.052
Stream size                      : 7.65 MiB (97%)
Writing library                  : x264 core 144 r2525 40bb568
Encoding settings                : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Audio
Format                           : AAC
Format/Info                      : Advanced Audio Codec
Format version                   : Version 4
Format profile                   : LC
Format settings, SBR             : No
Codec ID                         : 40
Duration                         : 12s 790ms
Bit rate mode                    : Constant
Bit rate                         : 128 Kbps
Channel(s)                       : 2 channels
Channel positions                : L R
Sampling rate                    : 48.0 KHz
Resolution                       : 16 bits
Stream size                      : 200 KiB (2%)

That's a different matter. MP4 uses two integers to represent the frame rate, a numerator and a denominator. Often the denominator is 90,000. 59.94 (60000/1001) fps can't be represented exactly using that denominator -- it would require timestamps that are a multiple of 1501.5/90000. So even though the source is a constant frame rate of 60000/1001 the time codes are variable, alternating between 1501/90000 and 1502/90000 seconds, giving an average of 1501.5/90000.