Why are ffprobe and mediainfo reporting different framerates for same file?

[syrist]

I typically use both ffprobe and mediainfo to get video info on a bunch of my files. I noticed that on a few videos in particular, they'll give me different results. Here's just 1 of those files:

ffprobe (json):

  "r_frame_rate": "30/1",
"avg_frame_rate": "30/1",

mediainfo:

Frame rate : 24.000 FPS

I suspect it has something to do with file using variable framerate but still...

Here's a link to a short 10 second mkv to see for youself: https://www82.zippyshare.com/v/JOJm40KZ/file.html

The problem is I need to re-encode the video and don't know if I should encode it to 30 fps or 24 fps. Any ideas or suggestions?

[_Al_]

Vapoursynths ffms2.Source which is avisynth equivalent of FFVideoSource seams to load it and index it as 24 fps

if frame rate is selected higher it gives some repeated frames

but, it is animation , so anything is possible

[syrist]

The same thing is happening on other animated shows released by ALiGN. I use ffms2 all the time but I provide it proper fpsnum and fpsden values so my encodes are fine. You think maybe the group didn't use ffms2 properly?

[jagabo]

The video is variable frame rate. A full ffprobe analysis shows it alternates between displaying frames for 1/30 second and 1/20 second, making the average frame rate 24 fps.

I think your ffprobe command line looked only at the first frame.

[_Al_]

Never dealing with vfr, never had to, but I am curious about it regarding how source plugins handle this vs. timecodes.

clip = core.ffms2.Source(source_path, cachefile=ffindex_file, timecodes=r'test.mkv.txt')
If I print timecodes, it alternates frame lenght 0.033 , 0.05, 0.033, 0.05 so about 24x to 1 second:

# timecode format v2
0.00
33.00
83.00
117.00
167.00
200.00
250.00
284.00
334.00
367.00
417.00
450.00
501.00
534.00
584.00
617.00
667.00
701.00
751.00
784.00
834.00
868.00
918.00
951.00
1001.00
.
.
.
12137.00
12170.00
12221.00
12254.00

Where is that timecode coming from? Is it generated from mkv or by ffms2?

[syrist]

The video is variable frame rate. A full ffprobe analysis shows it alternates between displaying frames for 1/30 second and 1/20 second, making the average frame rate 24 fps.

I think your ffprobe command line looked only at the first frame.

This is my command to generate the json data:

ffprobe test.mkv -v error -show_format -show_streams -print_format json > test.json

and this is the json data:

{
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "High",
            "codec_type": "video",
            "codec_time_base": "1/60",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
            "width": 1280,
            "height": 720,
            "coded_width": 1280,
            "coded_height": 720,
            "has_b_frames": 0,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 31,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "topleft",
            "field_order": "progressive",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "r_frame_rate": "30/1",
            "avg_frame_rate": "30/1",
            "time_base": "1/1000",
            "start_pts": 0,
            "start_time": "0.000000",
            "bits_per_raw_sample": "8",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {
                "BPS-eng": "2586112",
                "DURATION-eng": "00:00:12.287000000",
                "NUMBER_OF_FRAMES-eng": "295",
                "NUMBER_OF_BYTES-eng": "3971945",
                "_STATISTICS_WRITING_APP-eng": "mkvmerge v46.0.0 ('No Deeper Escape') 64-bit",
                "_STATISTICS_WRITING_DATE_UTC-eng": "2020-05-19 20:17:24",
                "_STATISTICS_TAGS-eng": "BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES"
            }
        },
        {
            "index": 1,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "HE-AAC",
            "codec_type": "audio",
            "codec_time_base": "1/48000",
            "codec_tag_string": "[0][0][0][0]",
            "codec_tag": "0x0000",
            "sample_fmt": "fltp",
            "sample_rate": "48000",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/1000",
            "start_pts": 0,
            "start_time": "0.000000",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {
                "BPS-eng": "126795",
                "DURATION-eng": "00:00:12.331000000",
                "NUMBER_OF_FRAMES-eng": "289",
                "NUMBER_OF_BYTES-eng": "195440",
                "_STATISTICS_WRITING_APP-eng": "mkvmerge v46.0.0 ('No Deeper Escape') 64-bit",
                "_STATISTICS_WRITING_DATE_UTC-eng": "2020-05-19 20:17:24",
                "_STATISTICS_TAGS-eng": "BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES"
            }
        }
    ],
    "format": {
        "filename": "D:\\VID\\test.mkv",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "matroska,webm",
        "format_long_name": "Matroska / WebM",
        "start_time": "0.000000",
        "duration": "12.331000",
        "size": "4178054",
        "bit_rate": "2710601",
        "probe_score": 100,
        "tags": {
            "encoder": "libebml v1.3.10 + libmatroska v1.5.2",
            "creation_time": "2020-05-19T20:17:24.000000Z"
        }
    }
}

You would think the avg_frame_rate should be 24 but it's indicating 30. So I think _Al_'s post suggests the file really is 24fps as mediainfo indicates.

[jagabo]

The file is exactly what I said: VFR that alternates between 1/30 and 1/20 second per frame. I believe this is because it was made from a source with 3:2 pulldown (30i) or 3:2 duplicates (60p) and the duplicate fields/frames were removed. Of course, the original production was constant 24 fps, like virtually all film sources. So you should treat it as constant 24p.

Here's the ffprobe command I used to generate stats for each frame:

ffprobe.exe -v 32 -stats -y -hide_banner -i input.mkv -select_streams v -print_format csv -of csv -show_entries frame >> "output.csv"

The time codes are included in the mkv file. They are used by players to determine how long to display each frame.

[_Al_]

Code: .......
ffprobe.exe -v 32 -stats -y -hide_banner -i input.mkv -select_streams v -print_format csv -of csv -show_entries frame

Thanks,
that pattern 0,33,50,33,50 ... is just mkvmerge algorithm solution to come up with 24 frames in 1 second? Could it be some other appropriate sequence that fits into that one second?

frame,video,0,1,0,0.000000,0,0.000000,0,0.000000,33,0.033000,11017,1001,1280,720,yuv420p,1:1,I,0,0,0,0,0,tv,bt709,bt709,bt709,topleft
frame,video,0,0,33,0.033000,33,0.033000,33,0.033000,50,0.050000,12026,40,1280,720,yuv420p,1:1,P,1,0,0,0,0,tv,bt709,bt709,bt709,topleft
frame,video,0,0,83,0.083000,83,0.083000,83,0.083000,33,0.033000,12078,40,1280,720,yuv420p,1:1,P,2,0,0,0,0,tv,bt709,bt709,bt709,topleft
frame,video,0,0,117,0.117000,117,0.117000,117,0.117000,50,0.050000,12126,40,1280,720,yuv420p,1:1,P,3,0,0,0,0,tv,bt709,bt709,bt709,topleft
frame,video,0,0,167,0.167000,167,0.167000,167,0.167000,33,0.033000,12178,40,1280,720,yuv420p,1:1,P,4,0,0,0,0,tv,bt709,bt709,bt709,topleft
.
.

[jagabo]

When a 24p source is broadcast as 720p60 video 24 film frames has to somehow become 60 video frames. This is usually accomplished by duplicating film frames in a 2:3 pattern. One film frame is displayed for the duration of 2 video frames, the next is displayed for the duration of 3 video frames, alternating between the two. So the film frames are displayed for 1/30 (2/60) second and 1/20 (3/60) second. After one second 24 film frames have become 60 video frames. Someone started with such a 720p60 video and encoded it as variable frame rate. The encoder saw that two frames were identical and encoded them as one frame with a duration of 1/30 second, then three identical frames were encoded as one frame with a duration of 1/20 second, etc.

[syrist]

Thanks for all the info. I guess the bottom line is I need a way to accurately report the average framerate of a source video so I can pass that info on the encoder. From what I learned in this thread, ffprobe probably only looks at the first frame and thus gives an incorrect representation of the overall framerate on certain variable framerate videos (like the sample I attached).

So I'll just use mediainfo to create my json going forwards.

[jagabo]

Though it works better in this case, I don't know if MediaInfo is any more reliable than ffprobe in general. I believe MediaInfo only reports the frame rate in the header. That may not always be accurate either. Some programs indicate the max frame rate in the header when encoding VFR. And even the average frame rate doesn't mean you can encode an entire video at that frame rate without conversion. For example, someone may have joined a 24p video and a 30p video into a single VFR video. The average might be 27p but if you encode 27p CFR the first part of the video will play too fast, the second part too slow. The audio will be in sync at the very start and the very end, but will be out of sync throughout the rest of the video.