Antilogical behaviour of Handbrake (in regards to file size)

[Umen Pich]

Hi fellas,
the other day I experimented how would it affect the file size if I deviate from my favourite Handbrake settings.
The version of HB is 0.10.0.6534
At first I played with a small clip of mine, then to confirm the results I encoded one full episode of a TV show with different parameters .

I have no explanation for the results and wanted to share with you.
Below are the file sizes in bytes and names of the files (which are informative for the parameters used)
In all cases I used Constant quality RF - 23.75, and all the same setting for the audio.

I changed ONLY
- Reference frames
- Maximum B-frames
- Motion Est method

My typical line of advance parameters looked like this:

level=4.0:mixed-refs=0:vbv-bufsize=25000:vbv-maxrate=20000:rc-lookahead=10:ref=2:bframes=4:b-adapt=2:direct=auto:me=umh:subme=10:trellis=2

I've got the following sizes of the encoded mkv files (the number 1 and 2 in the file name is the number of reference frames, the number 2 and 4 is the max B frames)

151,809,498 bytes TEST-1-3-hexagon.mkv
151,889,675 bytes TEST-1-3-multihexagon.mkv
154,702,768 bytes TEST-2-4-hexagon.mkv
156,717,740 bytes TEST-2-4-exhaustive.mkv

Paradox 1: multi-hexagon method produces a larger file compared to hexagon (everything else - the same) (first and second file)
Paradox 2: Exhaustive produces larger file compared to hexagon (everything else - the same)
(third and forth file)
Paradox 3: reference frames =1 and Max b frames = 3 produces smaller file compared to reference=2, max b frames = 4 (first and third file)

After all the file that should be smallest -- Test-2-4-exhaustive is largest and the file that should be largest -- Test-1-3-hexagon is in fact smallest.

Any ideas why ???

Thank you !

[TorBru]

The behaviour is in deed very logical since the first file has the worst motion compensation, thus the lowest file size, and the last file has the best motion compensation, thus the largest file size

[LigH.de]

I don't understand your thinking, TorBru.

If an encoder is more successful in finding a good match for a block, it can store a motion vector to a very similar reference area, and the difference will be small and better compressible than a not so similar match, or even an intra coded block. So trivially, more elaborate searching should result in better matches and better compression.

There is just one detail that makes me wonder: CRF is probably not the correct mode to test this tiny detail. One should possibly rather use CQ, to ensure nothing else differs, as CRF won't guarantee comparable results with different parameters, and may take an advantage from better matches elsewhere...

[TorBru]

My thinking is that better motion detection might result in smaller GOP-size and therefore more I-frames which have less compressablility. Increasing the amount of B-frames does not nescessary make a difference since it is a maximum and not a minimum. If the motion detection is less effective some motion might be lost in the encoding leading to larger GOP and smaller file size. The effectiveness of the motion estimation algorithms is exhaustive > multi-hexagonal > hexagonal.

This is how I understand it and this is why it makes sense to me

[LigH.de]

OK, that's an argument I could follow to some degree... May have to be proven.

[Selur]

Isn't that how crf (= constant rate factor) should behave?
Higher motion estimation -> better detail preservation and less bluring -> higher file size for fixed crf.
Remember crf doesn't mean constant quality (just because it's named that way in some user interface), assuming quality is objective characteristic.

[TorBru]

Remember crf doesn't mean constant quality (just because it's named that way in some user interface), assuming quality is objective characteristic.

I disagree with you on this. Constant Rate Factor (CRF) does indeed attempt to achieve a homogenous quality throughout the encoding and therefore it is IMO the same as calling it a constant quality, and this is the way most people (can) understand it in an interface

[Selur]

The point is the 'throughout the encoding' which means if one of the parameters changes the objective quality and file size can easily change quite a bit.
Which is why comparing encoding where the crf is fixed and other settings change is always problematic. (especially if you don't add PSNR/SSIM/... and similar metrics and only compare something as like file size or encoding time )

[Detmek]

I agree. x264 developers used to say that CRF with different settings gives different quality and should not be used to compare effects of various x264 options. Only 2-pass at same bitrate/filesize should be used.

[Umen Pich]

Well, then if you are right, the help tips inside Handbreak are not correct. For example the tip about the reference frames says: "The more you add, the better the compression". Apparently that cannot be true in 2-pass mode, since the compression in two pass mode will be all the same, no matter what parameters you pick. But according to my experiment (and your thoughts) the tip is also not true in constant quality mode.

Same about motion search. They state "a better motion search improves compression"

[LigH.de]

The only mistake here may be the certainty. More references and wider motion search may improve compression under certain circumstances (especially for scenes with heavy and wide motion), but doesn't have to improve in every case (optimal vectors for easily predictable and short motion can already be found with few references and sensible search range).

Still, a brake is written with a-k-e.

[ndjamena]

You can't make judgements like that based on a single encode.

Better Motion Search is more likely to find better matches within the reference frames, but it doesn't end there, it doesn't find PERFECT MATCHES, so if the reference frames aren't enough to reach CR I THINK x264 has to write over them at which point it becomes guesswork as to which of the original reference frames would produce the image with the lowest bitrate. I've always though most of the X264 settings work best with low bitrate encodes.

Ultimately coming to conclusions based on how one single encode turns out produces a flawed thesis. You need to produce many encodes from many different kinds of source material and compare them all, if it turns out on average more reference frame/better ME produces smaller files then the Handbrake tips are correct, regardless of the remaining encodes that were larger. There's a point in encoding where no matter how much CPU power you throw at a job all you end up with is DIFFERENT decisions rather than better ones.

[Detmek]

Well, then if you are right, the help tips inside Handbreak are not correct. For example the tip about the reference frames says: "The more you add, the better the compression". Apparently that cannot be true in 2-pass mode, since the compression in two pass mode will be all the same, no matter what parameters you pick. But according to my experiment (and your thoughts) the tip is also not true in constant quality mode.

Same about motion search. They state "a better motion search improves compression"

Better compression in this case means better quality/filesize ratio. So, in 2-pass mode where file size is constant better compression means higher quality. For constant CRF that can lead to few scenarios. Because CRF with different settings gives you different quality more efficient settings can lead to:
1. Better quality and bigger file
2. Same quality and smaller file
3. Lower quality and smaller file.

[Umen Pich]

I will test this with the same CQ but extremely different parameters