x265 HEVC Encoder

[LigH.de]

Running a series of small tests with "Kristen and Sara" 720p (@ varun); not yet done, but already reduced strengths show quite obviously a "bleeding" of energy from areas with fine details (hair) into areas with no details (blue background), causing "wobbling background" artefacts. A later analysis should show if this is caused rather by Psy-RD or Psy-RDOQ, or only in combination of both...

Link will be posted when upload finished...
__

KS_psy.7z in HEVC/samples

[El Heggunte]

Has anybody already tested commit e85b0aa

rc: update vbv for all b frames

HEVC is complex (and slow) enough that we can afford to update/plan the VBV
buffer states for all frames, not just I and P. The leads to smoother rate
control, particularly when there are large stretches of B frames, and less
(over) compensation is necessary for P frames.

https://bitbucket.org/multicoreware/x265/commits/e85b0aaa64e4aaccdfd37de92e14d91a53a789ea

P.S.: binaries here <= clicky

[Gravitator]

Often get problems with access to http://chromashift.org/x265_builds/ - firefox, opera, download master can not go, so now get around the problem through the turbo mode in opera Why is this happening...

[LigH.de]

The "Turbo Mode" is technically a transparent HTTP proxy (with low-quality image recoding to save bandwidth on images in websites). So I wonder if you are region blocked; the webserver will discover the geolocation of Opera's proxy while you enabled Turbo mode.
__

P.S.: Uploading x265 v1.2+413-e85b0aaa64e4 as well, full VBV bitrate control is a serious "achievement"...

[El Heggunte]

More patches 0_o

rc: enable abr reset in the first pass of two pass encode

observe this improves second pass results in ultrafast presets for some videos

https://www.mail-archive.com/x265-devel@videolan.org/msg04064.html

rc: add slow first pass as a cli. set turbo(faster) first pass as default in 2
pass

https://www.mail-archive.com/x265-devel@videolan.org/msg04065.html

[Selur]

+    if (param->rc.bStatWrite && !param->rc.bStatRead)
+    {
+        param->maxNumReferences = 1;
+        param-> bEnableRectInter = 0;
+        param->bEnableAMP = 0;
+        param->searchMethod = X265_DIA_SEARCH;
+        param->subpelRefine = X265_MIN(2, param->subpelRefine);
+        param->bEnableEarlySkip = 1;
+    }

is probably the interesting part

[NOiZE]

Hi,
i have a problem with a 2 pass. I have a x265 v1.2+413-e85b0aaa64e4 version but i have an error with a command line in megui.
My command line is: --pass 2 OR --pass 2 --stats ".stats" with ABR 880 kbps
In both case i have an error. Why?

Thank a lot

[LigH.de]

Did you run it with "--pass 1" first, so that the first pass created a statistics file which the second pass can read?

And do you understand that a statistics file with the filename ".stats" is possibly not the optimum?

And do you understand that MeGUI will (in contrast to x264) not yet support two consecutive passes of x265, so you will have to run two passes (the first with "--pass 1", then the second with "--pass 2") manually after each other?

[NOiZE]

It work Now!!

Thank you so much!!

PS: This is a comparison with 1pass vs 2 pass:http://screenshotcomparison.com/comparison/86185

[LigH.de]

IMHO more detail in the face (wrinkles, beard) for 2-pass. No surprise, as expected.

But static screenshots tell little.

[Selur]

Yes, compared to 1pass abr that is to be expected.

Just to make sure:
Should I get the same quality when I compare a 1pass crf encode and a 2pass encode which aims for the same file size, like it is the case with x264, are the results expected to be noticeable different?

[LigH.de]

Just like x264, as far as I understood, also x265 will run a CRF encode in the 2nd pass. So when you run a 1-pass CRF encoding with "incidently" the exact CRF value you need to match the desired target size, the result should be the same as the result of a 2-pass for this target size, as close as it gets.

[NOiZE]

Hi,
i get another encode with 2 pass x265. But, in this case the first pass is better than second pass (more details):
http://screenshotcomparison.com/comparison/86411
Why? depends on the type of film?

[Selur]

question is: If you compare the file sizes, is the second pass at least nearer to the target file size / average bitrate you aimed for?

[LigH.de]

Two mistakes here:

a) Talking about first pass and second pass is wrong, you should compare one single pass ABR with two consecutive passes VBR.

b) Single screenshots tell little.

A single pass ABR encode will try to keep the bitrate close to the average. That means: In easily compressible scenes it may spend much more bitrate than necessary (a screenshot from here may look better than a two-pass encode with VBR, using CRF in the second pass), but will not spend enough bitrate in another scene which is harder to compress (a screenshot from that scene will look worse).

[NOiZE]

question is: If you compare the file sizes, is the second pass at least nearer to the target file size / average bitrate you aimed for?

In the first pass, the ABR was 801, instead in the second pass the ABR was 842. I set The bitrate in megui in 850 (the encode was in ABR mode in both pass).

Ok, i should compare the one single pass ABR with two consecutive passes VBR, but is very strange that the first pass have a more details than second pass...

[LigH.de]

Not very strange, as explained. I don't know your whole test clip ... if there is another scene with much more action and detail, it will use much more bitrate than the average in a CRF encode (and a second pass in a two-pass encode uses CRF mode too), therefore it can only spend a lot less bitrate for scenes with little detail and little motion.

ABR (and 2-pass, 1st pass):
a) easy scene: average bitrate, waste of quality
b) complex scene: average bitrate, lack of quality

CRF (and 2-pass, 2nd pass):
a) easy scene: average quality, less bitrate
b) complex scene: average quality, more bitrate

[NOiZE]

In fact,
this film is very easy scenes (static, it's a comedy) and not have a movement scenes (like a thriller or war etc).
Therefore, ABR mode in first pass used more bitrate than second pass (CRF) and the quality is better. Is right?

PS: In megui i used in both pass ABR mode and in the second pass megui in automatic used CRF instead abr mode?

[Gravitator]

Hi NOiZE,
in 2-pass skin looks worse, but the hair of women better.

[LigH.de]

Don't refer to MeGUI. MeGUI does not really support 2-pass encoding of x265 yet on its own. Manually preparing two passes by adding the pass in the custom command line is a workaround.

Indeed, you will use the ABR mode of the encoder configuration interface in MeGUI to be able to submit a target bitrate; but internally, the 2nd pass will work like CRF encoding. The 1st pass will have created a statistics file so that the 2nd pass is able to calculate the exact CRF value required to approach the target bitrate.

[NOiZE]

Ok , I have understand now!
Indeed, I adding 2 pass manually in megui.

1 pass: --bitrate 850 --pass 1 --stats "film.stats" and i submit in a batch. After i adding a 2 pass
2 pass: --bitrate 850 --pass 2 --stats "film.stats"
and finally i submit in a two separate batch.

Thanks

[vhelp]

.stats file inconsistancy--

i'm working with the stats file in one of my x265/hevc/h265 projects to parse it, but i like to quickly view them before i start working on ideas to build something for it but the output seems gorky to me. the parameters are fine, that i can understand, but the stats detail following it are not evenly line-feeded.

Q: does anyone know how to configure notepad++ to line-feed off the ";" char ? that will help me look at all the older ones i've created since, until a fix is realized and made. otherwise, i will make a custom viewer and move on. thanks.

messy vs cleaned

in notepad:

#options: 720x480 fps=24000/1001 bitdepth=8 wpp ctu=16 tu-intra-depth=1 tu-inter-depth=1 me=0 subme=0 merange=25 no-rect no-amp max-merge=1 early-skip fast-cbf rdpenalty=0 no-tskip no-tskip-fast strong-intra-smoothing no-lossless no-cu-lossless no-constrained-intra open-gop interlace=0 keyint=250 min-keyint=23 scenecut=0 rc-lookahead=10 bframes=4 bframe-bias=0 b-adapt=0 ref=1 no-weightp no-weightb aq-mode=2 aq-strength=1.50 cbqpoffs=0 crqpoffs=0 rd=2 no-signhide no-lft no-sao sao-lcu-bounds=0 sao-lcu-opt=1 b-pyramid no-cutree rc=crf crf=17.0 qcomp=0.60 qpmin=0 qpmax=51 qpstep=4 ipratio=1.40 pbratio=1.30
in:0 out:0 type:I dur:0.042 q:14.09 q-aq:7.49 tex:61622 mv:10125 misc:905 icu:1350.00 pcu:0.00 scu:0.00 ;in:5 out:1 type:P dur:0.042 q:15.62 q-aq:9.97 tex:161743 mv:14823 misc:1242 icu:758.00 pcu:556.75 scu:35.25 ;in:3 out:2 type:B dur:0.042 q:16.76 q-aq:10.44 tex:86577 mv:7891 misc:921 icu:0.00 pcu:1160.50 scu:189.50 ;in:1 out:3 type:b dur:0.042 q:17.89 q-aq:11.09 tex:15679 mv:3607 misc:415 icu:0.00 pcu:513.25 scu:836.75 ;in:2 out:4 type:b dur:0.042 q:17.89 q-aq:11.20 tex:29430 mv:3890 misc:489 icu:0.00 pcu:704.00 scu:646.00 ;in:4 out:5 type:b dur:0.042 q:17.89 q-aq:11.95 tex:66990 mv:8386 misc:969 icu:0.00 pcu:964.75 scu:385.25 ;in:9 out:6 type:P dur:0.042 q:17.85 q-aq:13.04 tex:144272 mv:23065 misc:1128 icu:1220.00 pcu:98.50 scu:31.50 ;in:7 out:7 type:B dur:0.042 q:17.87 q-aq:12.83 tex:122591 mv:13401 misc:1334 icu:0.00 pcu:1264.50 scu:85.50 ;in:6 out:8 type:b dur:0.042 q:18.45 q-aq:13.12 tex:79782 mv:12284 misc:1175 icu:0.00 pcu:1094.25 scu:255.75 ;in:8 out:9 type:b dur:0.042 q:19.56 q-aq:14.66 tex:53648 mv:10759 misc:1309 icu:0.00 pcu:1059.50 scu:290.50 ;

or this way since code tags here distorts the view, this is how i see it in wordpad:

#options: 720x480 fps=24000/1001 bitdepth=8 wpp ctu=16 tu-intra-depth=1 tu-inter-depth=1 me=0 subme=0 merange=25 no-rect no-amp max-merge=1 early-skip fast-cbf rdpenalty=0 no-tskip no-tskip-fast strong-intra-smoothing no-lossless no-cu-lossless no-constrained-intra open-gop interlace=0 keyint=250 min-keyint=23 scenecut=0 rc-lookahead=10 bframes=4 bframe-bias=0 b-adapt=0 ref=1 no-weightp no-weightb aq-mode=2 aq-strength=1.50 cbqpoffs=0 crqpoffs=0 rd=2 no-signhide no-lft no-sao sao-lcu-bounds=0 sao-lcu-opt=1 b-pyramid no-cutree rc=crf crf=17.0 qcomp=0.60 qpmin=0 qpmax=51 qpstep=4 ipratio=1.40 pbratio=1.30
in:0 out:0 type:I dur:0.042 q:14.09 q-aq:7.49 tex:61622 mv:10125 misc:905 icu:1350.00 pcu:0.00 scu:0.00 ;in:5 out:1 type:P dur:0.042 q:15.62 q-aq:9.97 tex:161743 mv:14823 misc:1242 icu:758.00 pcu:556.75 scu:35.25 ;in:3 out:2 type:B dur:0.042 q:16.76 q-aq:10.44 tex:86577 mv:7891 misc:921 icu:0.00 pcu:1160.50 scu:189.50 ;in:1 out:3 type:b dur:0.042 q:17.89 q-aq:11.09 tex:15679 mv:3607 misc:415 icu:0.00 pcu:513.25 scu:836.75 ;in:2 out:4 type:b dur:0.042 q:17.89 q-aq:11.20 tex:29430 mv:3890 misc:489 icu:0.00 pcu:704.00 scu:646.00 ;in:4 out:5 type:b dur:0.042 q:17.89 q-aq:11.95 tex:66990 mv:8386 misc:969 icu:0.00 pcu:964.75 scu:385.25 ;in:9 out:6 type:P dur:0.042 q:17.85 q-aq:13.04 tex:144272 mv:23065 misc:1128 icu:1220.00 pcu:98.50 scu:31.50 ;in:7 out:7 type:B dur:0.042 q:17.87 q-aq:12.83 tex:122591 mv:13401 misc:1334 icu:0.00 pcu:1264.50 scu:85.50 ;in:6 out:8 type:b dur:0.042 q:18.45 q-aq:13.12 tex:79782 mv:12284 misc:1175 icu:0.00 pcu:1094.25 scu:255.75 ;in:8 out:9 type:b dur:0.042 q:19.56 q-aq:14.66 tex:53648 mv:10759 misc:1309 icu:0.00 pcu:1059.50 scu:290.50 ;

and cleaned up, wordpad / notepad:

in:0 out:0 type:I dur:0.042 q:14.09 q-aq:07.49 tex:061622 mv:10125 misc:0905 icu:1350.00 pcu:0000.00 scu:000.00 ;
in:5 out:1 type:P dur:0.042 q:15.62 q-aq:09.97 tex:161743 mv:14823 misc:1242 icu:0758.00 pcu:0556.75 scu:035.25 ;
in:3 out:2 type:B dur:0.042 q:16.76 q-aq:10.44 tex:086577 mv:07891 misc:0921 icu:0000.00 pcu:1160.50 scu:189.50 ;
in:1 out:3 type:b dur:0.042 q:17.89 q-aq:11.09 tex:015679 mv:03607 misc:0415 icu:0000.00 pcu:0513.25 scu:836.75 ;
in:2 out:4 type:b dur:0.042 q:17.89 q-aq:11.20 tex:029430 mv:03890 misc:0489 icu:0000.00 pcu:0704.00 scu:646.00 ;
in:4 out:5 type:b dur:0.042 q:17.89 q-aq:11.95 tex:066990 mv:08386 misc:0969 icu:0000.00 pcu:0964.75 scu:385.25 ;
in:9 out:6 type:P dur:0.042 q:17.85 q-aq:13.04 tex:144272 mv:23065 misc:1128 icu:1220.00 pcu:0098.50 scu:031.50 ;
in:7 out:7 type:B dur:0.042 q:17.87 q-aq:12.83 tex:122591 mv:13401 misc:1334 icu:0000.00 pcu:1264.50 scu:085.50 ;
in:6 out:8 type:b dur:0.042 q:18.45 q-aq:13.12 tex:079782 mv:12284 misc:1175 icu:0000.00 pcu:1094.25 scu:255.75 ;
in:8 out:9 type:b dur:0.042 q:19.56 q-aq:14.66 tex:053648 mv:10759 misc:1309 icu:0000.00 pcu:1059.50 scu:290.50 ;

[El Heggunte]

In EditPlus, I would CTRL + H and set "replace ; with ;\n",
I hope Notepad++ works in a similar way

(I gave it a try years ago and disliked it, maybe it's time to try it again ).

As for the patch, it's been posted to the mailing list already:

rc: nits; add LF in frame stats of 2 pass

diff -r 6e4eb8542203 -r b29a3f1fbbe2 source/encoder/ratecontrol.cpp
--- a/source/encoder/ratecontrol.cpp	Sat Aug 09 19:43:23 2014 -0500
+++ b/source/encoder/ratecontrol.cpp	Sun Aug 10 22:55:27 2014 +0530
@@ -2084,7 +2084,7 @@
             : rce->sliceType == P_SLICE ? 'P'
             : IS_REFERENCED(slice) ? 'B' : 'b';
         if (fprintf(m_statFileOut,
-  "in:%d out:%d type:%c dur:%.3f q:%.2f q-aq:%.2f tex:%d mv:%d misc:%d icu:%.2f pcu:%.2f scu:%.2f ;",
+  "in:%d out:%d type:%c dur:%.3f q:%.2f q-aq:%.2f tex:%d mv:%d misc:%d icu:%.2f pcu:%.2f scu:%.2f ;\n",
                     rce->poc, rce->encodeOrder,
                     cType, m_frameDuration,
                     pic->m_avgQpRc, pic->m_avgQpAq,

[vhelp]

i thought i would report back on the patch for the .stats file display problem.

the 1.2.510 build correctly opens in wordpad, but not in notepad. i think that most people use notepad because it is the default (and/or used first) for most text files, plus, it has a quick turnaround time, though not that it matters to most users.

i believe the reason has to do with the line-feed code used in the source. it should not be the \n, but instead,
the \r\n (CR/LF) codes. then it will display correctly in any text editor.

[El Heggunte]

Come on --- the fault is of Notepad, period. Why keep using a software that still hasn't learned how to deal with UNIX and MAC text files?

Since you are a Delphi programmer, IMNSHO you have no excuses for not having a commercial text editor
(TextPad, EditPlus, EmEditor, NoteTab, EditPad, UltraEdit, whatever).

[LigH.de]

To be "even more correct" – the linebreak should not be hardcoded either "\n" (Unix), "\r" (classic MacOS), or "\r\n" (DOS/Windows), but instead a constant related to the platform you are compiling the executable for.

If you are compiling x265 for Windows, a Windows linebreak is suitable. For Linux, MacOS X, BSD etc., an Unix style linebreak will be preferred.

Current C language variants are supposed to provide such a constant.

[NOiZE]

Hi,
when i use in megui a HEVC encoder version after 1.2+436-ed49f875ab20 (465,471,496,507 and 510 version), logs are wrong! I set in the first pass --ME umh, but logs are "ME / range / subpel / merge : dia / 16 / 2 / 1". Why it?

[LigH.de]

A complete command line set would be useful. There might be other parameters with some influence here.

[Selur]

may be a preset used?

[NOiZE]

This is a complete command line (preset medium):
--threads 0 --frame-threads 0 --no-wpp --ctu 64 --tu-intra-depth 1 --tu-inter-depth 1 --me umh --subme 7 --merange 16 --max-merge 1 --no-early-skip --no-fast-cbf --rdpenalty 0 --no-tskip --no-strong-intra-smoothing --no-constrained-intra --open-gop --keyint 240 --min-keyint 0 --bframes 3 --weightb --b-intra --b-adapt 2 --b-pyramid --ref 4 --weightp --rc-lookahead 60 --scenecut 40 --bitrate 800 --cbqpoffs 0 --crqpoffs 0 --ipratio 1.4 --pbratio 1.3 --nr 0 --rd 5 --psy-rd 0 --psy-rdoq 0 --signhide --aq-mode 2 --aq-strength 1 --cutree --no-cu-lossless --vbv-maxrate 16000 --vbv-bufsize 16000 --vbv-init 0.9 --no-lft --no-repeat-headers

If i change the versione of HEVC (after 1.2.436) with the same command line, the program use dia instead umh