Lower FPS, same or larger file size, why?

[radellaf]

I was playing around with handbrake trying to re-encode a short video podcast. Talking head, so little motion. The original file was 60fps and I was using CRF of 23 to 27 trying x265 and x264. FWIW, x265 with an opus codec for audio can do watchable QVGA in 32-40kbps, less than AAC-LC can do good voice audio. However, I don't think anything but VLC will play it.

So, I'd expect that going to 30, 24, or even 15fps at the same resolution (VGA or QVGA) would make for a smaller file. It didn't. Sometimes the file was only maybe 10% larger, other times maybe 30%.

Any idea why that might be?

[poisondeathray]

Maybe your commands didn't actually get passed correctly? or maybe some other settings changed between the encodes? Check the log file and/or mediainfo

Strictly speaking, the FPS doesn't matter. The frame count matters. If you just change the FPS, without changing the frame count (ie. speed up or slow down the video), the filesize should be the same, if everything else is the same

[radellaf]

I checked the FPS of the result in various players (QT, mplayer, VLC) and they did report the same frame rate I specified. The result had fewer frames than the original and the same runtime. The handbrake activity window showed that the parameters were being passed and other settings the same. if ffmpeg made internal changes not reflected by the command line, who knows.

My only guess is that the smaller changes between frames at higher rates increase compression efficiency. i.e., 100 small changes takes less space than 50 larger changes.

[Cornucopia]

Frame rate does matter in the sense that inter-frame codecs will see more difference between successive frames on lower FPS and less on higher FPS, and with a CRF encoding those similarities can be more efficiently compressed (with the higher FPS), so for similar overall quality/frame you'd need less additional bits for the additional frames (or vice-versa if looking at it the other direction). It's not a linear change, nor is it logarithmic but somewhere in between.

Scott

...you got back before my response was finished editing!...

[radellaf]

OK, so it just IS a mathematical reality, even though it is counter-intuitive.

In that case I just need to judge whether a slightly larger file is worth it for the video to be more compatible with simple-minded players that can't handle 60fps. For x264 I guess it is. Any player capable of x265 should be able to handle 60fps even if the display (or display driver) it uses as output needs a lower rate.

Thanks!

[poisondeathray]

Yes, smaller changes between frames means fewer differences, thus less residuals and smaller filesizes - but that doesn't overcome having more frames

Are you comparing the same generation ?

ie

source => crf 23

vs

source => crf 23 with 1/2 frames and 1/2 frame rate


It's very easy to demonstrate what you are observing doesn't happen with other sources that are interview type or "talking heads" (or ANY type of source for that matter). This suggests something is peculiar with your source, or you aren't doing a proper comparison (like comparing different generations). I'm guessing probably the latter . If you can upload your source, or at least cut a decent sample from it, I'd be curious to examine it

[radellaf]

It is the same exact 60fps source. Peculiar perhaps in that it is only about 10 seconds long and has very little motion. Also, assume I'm using beta handbrake with x265. I can't be absolutely sure I did a x264 test.

I will try it tonight with a different 60fps clip of something more common, in 264, with exact numbers.

Seems it _could_ be possible that compression is enough more efficient to more than compensate for twice as many frames in some cases. That 15 fps is still larger than 60 is even harder to believe.

FWIW the source was a Sanyo F1A in webcam mode recorded by QuickTime 7 on OS X 10.9.4. That is not the most common of setups to say the least, could even have been mjpeg.

[poisondeathray]

It's never possible to compensate. Not with the methods x264 or x265 use. A frame always costs "something" to encode,even a tiny residual. Something is more than nothing. You should always get a reduction in filesize, even if it's a tiny one. Because of temporal compression, and VBR, you never get a 1:1 reduction (ie. it's non linear. Conversely, if you had CBR, and IDR frames only, you would get a 1:1 reduction). "More efficient" just means the reduction you get from decimating frames is less.

It's more likely something was wrong with your comparison. Check that the settings between the tests were the same

For example, the well known test clip akiyo (which is a "talking head" news caster scenario)

crf 23 => 136kb
crf 23, 1/2 number frames => 111kb

[radellaf]

Here are two encoded results. I'll edit in a dropbox link for the 100MB original later.

[poisondeathray]

Ok thanks

The settings were different between them. The 60FPS is VFR with a --keyint of 530 --min-keyint 53, the 15FPS is CFRwith a --keyint of 150 --min-keyint of 15

If everything else is equal (and they aren't because VFR decimation is not quite comparible) , then a longer GOP interval will offer higher compression . A very crude way of thinking of it is the keyframe interval of 530 allows you to "stuff" more lower cost b and p frames into the encode, but a shorter max keyframe interval forces you to expend more on higher cost I frames

[sneaker]

Also, x264's CRF is framerate-aware, not sure about x265. (Unless HandBrake messes with it - I didn't test that)

[poisondeathray]

And that explanation should only work, if you have a long enough video, with enough GOP's. In practice, you do see this effect. However - you only have 1 GOP, so that sort of throws that explanation out of the window

Anyways you should repeat it with the same settings, and if something is still amiss , then I'd love to see the source

[poisondeathray]

Also, x264's CRF is framerate-aware

Yes, you are correct , lower quality at higher framerates - but I've never seen it (over)compensate for encoded frame number

x264 | branch: master | Jason Garrett-Glaser <darkshikari at gmail.com> | Thu Dec 23 19:33:01 2010 -0500| [22bfd310abced4df0b11c9620b152e5cfe746d43] | committer: Jason Garrett-Glaser

VFR/framerate-aware ratecontrol, part 2
MB-tree and qcomp complexity estimation now consider the duration of a frame in their calculations.
This is very important for visual optimizations, as frames that last longer are inherently more important quality-wise.
Improves VFR-aware PSNR as much as 1-2db on extreme test cases, ~0.5db on more ordinary VFR clips (e.g. deduped anime episodes).

WARNING: This change redefines x264's internal quality measurement.
x264 will now scale its quality based on the framerate of the video due to the aforementioned frame duration logic.
That is, --crf X will give lower quality per frame for a 60fps video than for a 30fps one.
This will make --crf closer to constant perceptual quality than previously.
The "center" for this change is 25fps: that is, videos lower than 25fps will go up in quality at the same CRF and videos above will go down.
This choice is completely arbitrary.

Note that to take full advantage of this, x264 must encode your video at the correct framerate, with the correct timestamps.

[radellaf]

Well here's a 59.97 and 15 constant FPS version just to remove that variable. I put "keyint=530:min-keyint=53" into handbrake and it looks like it accepted it. Ignore "Movie59vga.mp4"

If ffmpeg is making behind the scenes changes to its own "slow" preset then I'll assume theres some good reason for it, but who knows.

dropbox failed on the first attempt and 100MB takes 15 minutes. sigh.

[poisondeathray]

As I said earlier - that explanation shouldn't make a difference with 1 GOP. (There is only 1 I-frame) .It will only make a difference on a typical scenario with longer video

add --force-cfr, to make them both CFR

[radellaf]

Here's the source as a dropbox link. https://www.dropbox.com/s/kd8jeb8w2qus5yr/Movie.mov?dl=0

Those last two are CFR"
"[17:27:47] + bitrate 258969 kbps
[17:27:47] + filters
[17:27:47] + Framerate Shaper (1:27000000:450450)
[17:27:47] + frame rate: 52.688 fps -> constant 59.940 fps
[17:27:47] + Crop and Scale (640:480:0:0:0:0)
[17:27:47] + source: 640 * 480, crop (0/0/0/0): 640 * 480, "

"[17:26:11] + filters
[17:26:11] + Framerate Shaper (1:27000000:1800000)
[17:26:11] + frame rate: 52.688 fps -> constant 15.000 fps
[17:26:11] + Crop and Scale (640:480:0:0:0:0)
[17:26:11] + source: 640 * 480, crop (0/0/0/0): 640 * 480, "

[sneaker]

I can reproduce it here as well. It is x264's framerate awareness.

[radellaf]

OK, so essentially CRF 23 doesn't mean exactly "CRF 23" and varies a bit depending on framerate, and this is necessary for the same perceived quality.

i.e., increasing the CRF manually isn't a good idea.


Thanks for all the help. I know it's not an important issue, just a technical curiosity, so I appreciate the help all the more.

[poisondeathray]

I can reproduce it here as well. It is x264's framerate awareness.

I don't think that's the entire explanation - There is more to it

If you keep the same source colorspace 422, the relationship switches, the 15FPS/lower framecount clip is smaller

The source clip was VFR to begin with, it might have something to do with it

If you try it on any other clip e.g. speed up akiyo to 59.94 and repeat a similar test, the 15FPS/lower framecount clip is smaller


Anyways , it's interesting, thanks for the clip

[radellaf]

Here's a test with a less bizarre original.

https://www.dropbox.com/s/o23ycw0gorchwor/SANY0061.MP4?dl=0

1080 60fps from the same camera recorded to SD card. CFR 59.97 and 15, both "keyint=530:min-keyint=53".

The 15fps is smaller but surely not by much. I'd prefer 30fps to 15, so if 15 doesn't save me any space, I won't use it. Over 30fps I don't want for other reasons.

[poisondeathray]

That last one is a more typical/expected result . I just tested 4 different clips, different types of sources, some with more motion, some with less even down to 10FPS for one of them - and got the expected lower filesize on all of them. Is it worth it ? On some sources maybe, others probably not. Eitherway you should be getting a smaller filesize

Your MOV is the first time I've ever seen it "overcompensate", at least for the 4:2:0 version - I think there is something else going on.

[radellaf]

The original MOV is I think a really odd file. I know there are videos that _switch_ framerates and that's what VFR is for, but how many of them are actual _varying_ unstable frame rates?

Maybe all webcams or captured video chat streams (some hit 0fps from network issues for several seconds but I assume the video isn't encoded that way).
The FH1A is pretty bad as webcam and might be unique. Sometimes it drops off the USB, and why isn't it at least 720 mpeg4? At least it does something. All digicams should have high quality webcam modes. Just lazy software development that they don't.

Lowering the pixel dimensions is the easiest way for me to save bandwidth for a private video podcast. The reduction in file size seems proportional or better compared to number of pixels. I was expecting the same for reduced frame rates.

[radellaf]

Here's the same vfr webcam source and type of scene but a longer clip that I didn't use because the original is over 300MB.

All very close. I would think the original VFR could be more efficient than forcing in some extra frames to maintain 59.97.

VFR:
frame rate: same as source (around 53.319 fps)
x264 [info]: frame I:2 Avg QP:23.23 size: 17170
x264 [info]: frame P:145 Avg QP:24.09 size: 1268
x264 [info]: frame B:430 Avg QP:25.38 size: 162

59.97 CFR:
x264 [info]: frame I:2 Avg QP:23.38 size: 16957
x264 [info]: frame P:201 Avg QP:24.13 size: 975
x264 [info]: frame B:445 Avg QP:25.50 size: 154

15 CFR:
x264 [info]: frame I:1 Avg QP:20.61 size: 23034
x264 [info]: frame P:42 Avg QP:20.53 size: 4393
x264 [info]: frame B:121 Avg QP:22.72 size: 766

If I raise the CRF to 26 at 15fps I get Avg QP closer to the 59 or VFR results and a much smaller file.

15 CFR CRF 26 (no sample attached, 263.3 KB):
x264 [info]: frame I:1 Avg QP:23.61 size: 17012
x264 [info]: frame P:42 Avg QP:23.34 size: 2118
x264 [info]: frame B:121 Avg QP:25.23 size: 373

And just for fun, if I compare at lossless CRF=0 I get 56MB at VFR and 17.4MB at 15 FPS CFR. THAT'S the kind of difference I was expecting. QuickTime X won't play it at CRF=0, QuickTime 7 will. Apple's new video functions are such a disappointment.

[poisondeathray]

The original MOV is I think a really odd file. I know there are videos that _switch_ framerates and that's what VFR is for, but how many of them are actual _varying_ unstable frame rates?

It's very common . 99% of all camera phones for example .

It's content dependent, like webcams . If you have long periods of inactivity, the framerate can drop right down for extended periods of time - so very variable. On the other hand, it can almost approach CFR at the base rate if there is constant activity or minimally variable

A word on handbrake - the CRF 0 won't pass through unless you change the profile (check with mediainfo and it will show CRF=1) . At least it was that way with older versions, not sure what it's like now

[Cornucopia]

Actually, most phone cams' VFR is due to maintaining adequate exposure in the face of fluctuating (and predominantly LOW) light levels, regardless of content motion/activity/complexity. This can be verified experimentally. (Outdoor sunny daylight sports activities being recorded hardly ever exhibit VFR sections)

Webcams are another matter.

Scott

[radellaf]

The current nightly build version of handbrake (with x265) will pass RF 0 _if_ the profile is set to auto (or maybe high as well). Set to main there's an error in the log. 0.5 seems to behave the same, and both won't play in QT-X.
I tried to get RF 1 and 4:4:4 but it doesn't seem to support pix-format on the additional arguments line.
I'm not clear how it's "lossless" vs the mjpeg source but it makes some intuitive sense that lower frame rates make less difference when compression techniques get more sophisticated. Uncompressed video being the limit.

I thought older phones would just convert low shutter speed video to 30fps or whatever (15, 8). Didn't think AVI or 3GPP supported variable rate on a frame by frame basis. I guess VFR is older than I thought. Not sure what iOS does with low shutter speed. I think it comes out 60fps no matter what but I'll have to look.

Edit: Google tells me iPhone video is VFR and it does, indeed, give people fits trying to keep AV sync while editing it in many apps. I'm inclined to use CFR when doing my usual resolution reduction conversions on those videos. I hate AV sync problems.

[Cornucopia]

As has been explained before, VFR has many downsides & few upsides. Smartly configured VBR Interframe-compressed CFR coding can provide similar bitrate savings with none of the sync or motion aliasing or exposure problems common to VFR.

Unless I'm mistaken, AVI does NOT support VFR. But phones really don't save in AVI. They primarily use MOV (in the case of iOS devices) or MP4 (for all other devices).
As I explained above, iOS does the same as other phones during low light levels. From what I have read of the OSes and their drivers, there isn't much a programmer can do to override the firmware/hardware limitations and predeliction for VFR, even when they want to create a CFR stream.
3GPP and 3GPP video was more common to phones prior to the Smartphone revolution. As a strict subset of MP4, it might be possible to have VFR, but I'm not familiar with any examples of them using it.

Scott