HEVC: 8 vs 10 vs 12bit comparison using Netflix VMAF algorithm

[convert_skeptic]

Recently I downloaded a h265/hevc rip and noticed my device was doing software decoding.
I look into the source and notice it is 12bit ... what the **** ...

So as many of you remember from the x264 era, 10bit encoding has merits even with 8bit source content since it eliminates rounding/quantization errors that cause color banding.
Also, un-intuitively, the larger color space, even though it takes up more data, somehow manages to pull of superior size due to better compression characteristics.

So as history has shown 8bit -> 10 bit was a significant improvement.

Currently, the question remains if the same holds true for 10 bit -> 12 bit.

I am skeptical for two reason.

1) The 10bit color space should have enough internal precision to eliminate banding present from rounding errors, so this point is moot

2) Will the now even larger color space compress better again or will we be left with a larger file? If so, would we be better off using a lower CRF instead of a larger color space.

Luckily Netflix has open sourced its VMAF algorithm. As an Initial test I want to transcode a 20sec clip via the follow settings and get the VMAF score for each transcode. https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652

--preset veryslow
--crf 					[22, 23, 24]
--output-depth				[8, 10, 12]
--ctu 32
-–tu-intra-depth 4 
-–tu-inter-depth 4 
-–limit-tu 1 
-–me 3
--aq-mode 3
--aq-strength				[0.8, 0.9, 1]
-–qcomp					[0.65, 0.7, 0.75]
--sao  					[on, off]	
--strong-intra-smoothing 		[on, off]
--deblock=-2:-2				[on, default]

Which is only ~650 encodes.

Next after I determine how I feel about sao, intra-smoothing, and deblock, I will rerun the test with the following combinations.

--preset veryslow
--crf 					[22, 23, 24]
--output-depth				[8, 10, 12]
--ctu 32
-–tu-intra-depth 4 
-–tu-inter-depth 4 
-–limit-tu 1 
-–me 3
--aq-mode 3
--aq-strength				[0.8, 0.9, 1]
-–qcomp					[0.65, 0.7, 0.75]

-–psy-rd 				[1, 1.5, 2]
-–psy-rdoq				[1, 3]
-–qg-size				[16, 32]
--max-tu-size				[16, 32]
--ref					[3, 4]

Which takes us to an additional ~4000 encodes.


To sum up, I never encode, but seeing is believing if I am about to accept that 12bit hevc should be the new defacto standard, even for 8 bit source content.


~~~~~~~~~~~~~~

Now onto my real question. I will be doing 2-pass encodes via commands similar to the following, as I trimmed the above arguments for it to be readable on this forum.

x265 input.yuv --crf 20 --y4m --input-res 1280x720 --fps 24000/1001 --pass 1 --preset veryfast output_test.mp4 --vbv-bufsize 5000 --vbv-maxrate 5000
x265 input.yuv --crf 20 --y4m --input-res 1280x720 --fps 24000/1001 --pass 2 --preset veryfast output_test.mp4

I arbitrarily chose --vbv-bufsize 5000 --vbv-maxrate 5000 as CRF 22-24 videos will not get close to a ~5000kbps bitrate.

I do not want those two settings to ruin my results.

Can I leave the larger values as is, or is there some relation to CRF I should be aware about?

[pandy]

I arbitrarily chose --vbv-bufsize 5000 --vbv-maxrate 5000 as CRF 22-24 videos will not get close to a ~5000kbps bitrate.

I do not want those two settings to ruin my results.

Can I leave the larger values as is, or is there some relation to CRF I should be aware about?

It depends what is your source and your target... if this is local file with access and transfer speed well better than codec limitation then i would allow to higher vbv-maxrate, lower vbv buffer size may improve seek-ability at a cost of lower quality (assuming bitrate limitation) - main idea behind vbv buffer size and maxrate is to guide video encoder how video should match mostly transmission medium limitations or other system limitation.

[convert_skeptic]

Sure. I’ll bump both up to 10mbps. There is no transfer limit as this is simply a test for 8 vs 10 vs 12 bit encode settings vs resultant quality per file size. It will not be subject to network transfer limits.

Source: blu-ray
Target: desktop

I am not trying to introduce any bottlenecks that can skew my results. This is purely an experiment to see if 12bit encodes on 8bit source content have merit.

Last remaining question I would like to clarify.

On doom9 I read that I can run the —pass 1 encode ONCE to generate the info file, then I can use that same info file for all of the subsequent 650 —pass 2 encodes.

(I read that I don’t have to rerun the —pass 1 encode for each individual—pass 2 encode since keeping the original info file will work as settings of the pass1 encode have no affect on the 2nd pass result besides generating the info file)

This trick should cut down on total encodes from 1300 (650x2) —> 651 encodes ( 650 + initial first pass)

I would clarify on doom9 but they have a stupid 5day waiting limit for new accounts.

[fornit]

Hm,

I would do the pass 1 encodes for the options which have a stronger influence on the bitrate.

That means: For each combination of crf and output-depth. As you want to try 3 crf values with 3 different output-depth values, that would mean to create 9 pass 1 encodes.

[convert_skeptic]

Hm,

I would do the pass 1 encodes for the options which have a stronger influence on the bitrate.

That means: For each combination of crf and output-depth. As you want to try 3 crf values with 3 different output-depth values, that would mean to create 9 pass 1 encodes.

Sure, I have no issues creating 9 first pass encodes.

My doom9 reference is from this thread.
https://forum.doom9.org/showthread.php?p=1830946#post1830946

The member took a CRF19 pass one encode as an input to fixed bitrate pass2 encodes of 4700, 60000, and 470k.

He then compared to a set of three encodes with typical matching pass1/pass2 parameters and found no difference.

[fornit]

In the following post Werner Robitza describes how Netflix is doing it:
http://disq.us/p/1hvzsna

They do a crf 1st pass to capture the bitrate. If it fits then they reuse 1st pass stat data to do a bitrate based 2nd pass.

But if they don't like the bitrate and want to use another one, then they don't reuse that stat data.

Probably you're on the safe side if you do those 9 1st pass encodes.