x265 crf/cqp problem

[nixiejames]

hi, i am new to encoding to please bear with me.

i'm re-encoding dragon ball kai anime using x265 and i keep having problems right from the intro. i'll have my source and encoded version attached.

if you look carefully where the clouds are moving, it is like bits of it are a bit messed up, delayed, or whatever it's called hahaha.
if i use cqp 23-28, however, the quality is visually lossless in my eyes. why is this? from what i've read, crf and cqp are similar to each other in a sense that one uses variable qp and one uses a constant qp. i find it then hard to believe that my crf 23 one looks a lot worse than my cqp 28 one.

i want to use crf cause it gives a more suitable file size for me. please help.

[LigH.de]

CRF calculates a difference between the original and the reconstructed encoded video, it tries to guarantee that the difference stays below a certain threshold. Detailed scenes need a finer quantization. For scenes with very low detail, like cloudy sky, it will calculate that a coarser quantisation should be sufficient to look "good enough". But if there are very little details, especially no sharp edges, motion detection and estimation is hard to apply. The codec may slightly miscalculate the direction or amount of motions. A too coarse quantization may then sum up artifacts until they reach an annoying amount.

[nixiejames]

CRF calculates a difference between the original and the reconstructed encoded video, it tries to guarantee that the difference stays below a certain threshold. Detailed scenes need a finer quantization. For scenes with very low detail, like cloudy sky, it will calculate that a coarser quantisation should be sufficient to look "good enough". But if there are very little details, especially no sharp edges, motion detection and estimation is hard to apply. The codec may slightly miscalculate the direction or amount of motions. A too coarse quantization may then sum up artifacts until they reach an annoying amount.

i'm not sure if i get it. when i set crf, quantization becomes variable depending on the scene, right? so when i set it to crf 23, qp goes somewhere from qp 23 and up if it has a lot of motion which, in this case, the low detailed moving clouds.

does this mean my encode went higher than qp 28? i also encoded using cqp 28 and the clouds were identical to the source with seemingly no visual problems.

[LigH.de]

There is not only a quantizer per whole frame; there is even a quantizer per slice, different parts of a frame may have different quantizers.

CQP will apply the same quantizer to the whole frame; CRF will vary the quantizer for all slices, therefore you can read average values in statistics.

In your case, you will have to convince x265 not to be too optimistic reducing the quantizer in relation to the detected amount of details. You may need to experiment with Psy-RDO parameters:

   --rd <0..6>                   Level of RDO in mode decision 0:least....6:full RDO. Default 3
   --[no-]psy-rd <0..5.0>        Strength of psycho-visual rate distortion optimization, 0 to disable. Default 2.0
   --[no-]rdoq-level <0|1|2>     Level of RDO in quantization 0:none, 1:levels, 2:levels & coding groups. Default 0
   --[no-]psy-rdoq <0..50.0>     Strength of psycho-visual optimization in RDO quantization, 0 to disable. Default 0.0
   --[no-]rd-refine              Enable QP based RD refinement for rd levels 5 and 6. Default disabled

[nixiejames]

There is not only a quantizer per whole frame; there is even a quantizer per slice, different parts of a frame may have different quantizers.

CQP will apply the same quantizer to the whole frame; CRF will vary the quantizer for all slices, therefore you can read average values in statistics.

In your case, you will have to convince x265 not to be too optimistic reducing the quantizer in relation to the detected amount of details. You may need to experiment with Psy-RDO parameters:

   --rd <0..6>                   Level of RDO in mode decision 0:least....6:full RDO. Default 3
   --[no-]psy-rd <0..5.0>        Strength of psycho-visual rate distortion optimization, 0 to disable. Default 2.0
   --[no-]rdoq-level <0|1|2>     Level of RDO in quantization 0:none, 1:levels, 2:levels & coding groups. Default 0
   --[no-]psy-rdoq <0..50.0>     Strength of psycho-visual optimization in RDO quantization, 0 to disable. Default 0.0
   --[no-]rd-refine              Enable QP based RD refinement for rd levels 5 and 6. Default disabled

oh wow, so different parts of a frame have different quantization. i am only knowing this now. thanks a lot for the informations. i'll get going with those parameters.