Understanding Quant and SSIM (HCenc encodes)

[txporter]

I am trying to understand what the Quant value reported by HCenc is telling me about the quality of the encoding that I am doing. I am messing around with some of the various matrix files in HCenc to see what the result is on the final MPEG2 encode. I am taking a 2min video clip that is ~98% FILM. I am simply sending the clip through TITVC and encoding at 4000kbps with different matrices. I then compare that video to the original source to calculate SSIM (avisynth).

Here is what I see:

Matrix      Bitrate    Quant   SSIM
MPEG         4000       6.714  75.51
MPEG STD     4000       6.93   74.54
FOX1         4000      12.528  74.71
MANONO1      4000       8.369  74.98
AVAMAT7      4000       4.615  74.14

Visually, I have a pretty hard time telling them apart. AVAMAT7 looks softer than the others, but that's about what I can see (apparently I am not a very picky viewer visually! )

At any rate, the SSIM values are all fairly close but the reported Quant values from HCenc are vastly different. What exactly are they supposed to be telling me? Are Quant values reported from HCenc something that you can use to compare to some other type of encoder (like XVID)? I guess the values are much higher than I would expect for what I feel looks very comparable to the source, quality-wise.

[hank315]

The final compression is the product of the Quant value and the matrix values, so if you set a fixed bitrate it is to be expected that the Quant values are higher using FOX1 than AVAMAT7.
IMO the MPEG STD is bad because for non-intra (P and B-frames) the values for low and high frequencies are the same (all 16) meaning a lot of noise isn't quantized to zero.
In my tests AVAMAT7 also is always low on SSIM, the MPEG matrix normally is pretty good on SSIM, in general HCLOW produces the highest SSIM metric values, but it doesn't mean it will give the best visual results.

You can't compare MPEG2 Quant values to XVID (MPEG4) Quant values.

[txporter]

So basically, I shouldn't worry about the Quant value that is reported after an encode, but rather find some combination that looks good to me? I thought maybe I was missing something that should have been obvious. FOX1 being ~12 vs ~6-8 for the others seemed to me that I should have been able to see something. I guess that FOX1 must keep more detail in the ranges that are not so visibly obvious to the human eye? Are those details also the ones that are thrown out first if I don't allow it enough bitrate?

I had read this post that you made at the end of last year. I guess I had expected FOX1 to require ~2X the bitrate compared to MPEG matrix based off of that, so I wasn't surprised to see the high Quant. I didn't expect the quality to be basically the same (or the SSIM). That is what has me so confused.

[manono]

So basically, I shouldn't worry about the Quant value that is reported after an encode, but rather find some combination that looks good to me?

You definitely should worry about the quant value. You want to choose a matrix that gives you the quant value for the quality you want. For your particular source and bitrate, the Fox1 matrix is too 'good'. It winds up with too high a Q for the bitrate used. Granted, though, I have seen retail DVDs using that matrix with Q values (as measured by the free version of the Tecoltd Bitrate Viewer) nearly that high, I generally choose a matrix to give me an average quant (as measured by Bitrateviewer) of 7 or so, maybe a bit less when using lower quality matrices.

Granted, the differences are minute, but I think if you saved out pics of the same frame from the different encodes, and then made a slide show out of them, you'd see differences. Me, though, I just rely on the Q values, mostly.

[txporter]

Thanks, Manono. Nearly all of my re-encodes are just to embed subtitles into DVD rips (and maybe sharpen/denoise a bit) so that I can serve them up to my Tivo for playback. It sounds like you might actually do a bit of analysis up front and choose a matrix to hit your Quant target. Is that right?

If I were to decide that I want to always use MPEG or MANONO1 as my default matrix, could I just encode at a constant Quant? Would that result in a similar encode quality as finding out the required bitrate for the average Quant value and doing a 2-pass average bitrate encoding with HCenc?

[manono]

Yes, that's right. It's for DVDs and if I guess wrong (if the Q value is going to come out much higher or lower than I had hoped) I'll stop the encode, switch matrices, and run a few more passes in CCE.

[txporter]

Ok, thanks for the information.

If I were to decide that I want to always use MPEG or MANONO1 as my default matrix, could I just encode at a constant Quant? Would that result in a similar encode quality as finding out the required bitrate for the average Quant value and doing a 2-pass average bitrate encoding with HCenc?

All right, I tried to answer my own question here. I had to use HCenc 0.23 instead of 0.24 since 1pass doesn't work for 0.24. I ran with MPEG matrix because MANONO1 doesn't exist in 0.23.

MPEG, 4k 2-pass - TITVC, Undot.FluxsmoothT(3).LSFMod() - 5.617 Q, 3994 avg bitrate, 74.38 SSIM
MPEG, 5.617 CQ - TITVC, Undot.FluxsmoothT(3).LSFMod() - 3898 avg bitrate, 72.92 SSIM
MPEG, 5.5 CQ - TITVC, Undot.FluxsmoothT(3).LSFMod() - 4014 avg bitrate, 73.26 SSIM

Comparable results, but not the same. Visually, they all look fine to me. I needed to drop Quant down a bit to match average bitrate. Final SSIM is lower with constant Q than with 2-pass bitrate.

[hank315]

I agree with manono, it's better to have a quant value within a reasonable range, lets say 3 - 8, if it isn't in this range, I will choose a different matrix.
A lot of things like bit rate control just work better when the quant value is within this range.

If you want to use the manono1 matrix in HC023, just paste this in the ini file:

*CUSTOMMATRIX
8 8 8 9 11 13 14 17
8 8 9 11 13 13 14 17
8 8 11 12 13 14 17 94
9 11 13 13 14 17 17 94
11 11 13 13 14 17 94 94
13 13 14 16 17 20 94 94
13 13 14 17 94 94 94 94
13 14 17 94 94 94 94 94

12 12 13 14 15 16 22 26
12 13 14 15 16 22 26 32
13 14 15 16 22 26 32 41
14 15 16 22 26 32 41 53
15 16 22 26 32 41 53 94
16 22 26 32 41 53 70 94
22 26 32 41 53 70 94 94
26 32 41 53 94 94 94 94

[txporter]

I agree with manono, it's better to have a quant value within a reasonable range, lets say 3 - 8, if it isn't in this range, I will choose a different matrix.
A lot of things like bit rate control just work better when the quant value is within this range.

If you want to use the manono1 matrix in HC023, just paste this in the ini file:

*CUSTOMMATRIX
8 8 8 9 11 13 14 17
8 8 9 11 13 13 14 17
8 8 11 12 13 14 17 94
9 11 13 13 14 17 17 94
11 11 13 13 14 17 94 94
13 13 14 16 17 20 94 94
13 13 14 17 94 94 94 94
13 14 17 94 94 94 94 94

12 12 13 14 15 16 22 26
12 13 14 15 16 22 26 32
13 14 15 16 22 26 32 41
14 15 16 22 26 32 41 53
15 16 22 26 32 41 53 94
16 22 26 32 41 53 70 94
22 26 32 41 53 70 94 94
26 32 41 53 94 94 94 94

Thanks a lot, Hank! I had just figured out the custom matrix myself and copied over the matrix parameters from 0.24. I am enjoying tinkering around with the settings in HCenc. Excellent program! With a little denoising and sharpening, I am able to get very good video even at 3000 kbps.