x265 HEVC Encoder

[LigH.de]

x265 v1.7+328: Zones do work in general. But probably not as strictly as expected. Per-slice quantizers may still adapt to requirements, away from a given base quantizer. But you can spot in the CSV report that there is indeed a range with higher or lower average quantizers if you define a zone with fixed quantizer or bitrate factor.

[-Habanero-]

Still no dice. **** it, I'm just gonna encode two separate videos with different bitrates and combine them after.

[LigH.de]

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2

Don't tell me zones don't work.

[-Habanero-]

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2

Don't tell me zones don't work.

Zones don't work.

But they do appear to work for --crf, just not 2pass.

[poisondeathray]

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2

Don't tell me zones don't work.

Zones don't work.

But they do appear to work for --crf, just not 2pass.

Zones works for 2pass

If you look at the attached logfiles in notepad++ or a decent text editor (ie in an organized column view), it's easy to see the differences. But the log is only written on the 1st pass (--pass 2 doesn't overwrite it, so it's not necessarily "proof" of what's going on in the 2nd pass)


But it works in the end result, confirming what the 1st pass log file says






You can see in the analyzer the "height" of the bars is reduced in the affected zone, as expected

ffmpeg -i 1.avi -c:v rawvideo -pix_fmt yuv420p -an -f rawvideo - | x265 --input - --fps 30000/1001 --input-res 848x480 --bitrate 1000 --pass 1 --sar 1:1 --stats "2pass.log" -o 2pass1000kbps.265
ffmpeg -i 1.avi -c:v rawvideo -pix_fmt yuv420p -an -f rawvideo - | x265 --input - --fps 30000/1001 --input-res 848x480 --bitrate 1000 --pass 2 --sar 1:1 --stats "2pass.log" -o 2pass1000kbps.265

ffmpeg -i 1.avi -c:v rawvideo -pix_fmt yuv420p -an -f rawvideo - | x265 --input - --fps 30000/1001 --input-res 848x480 --bitrate 1000 --pass 1 --sar 1:1 --zones 0,50,b=0.1 --stats "2passzones.log" -o 2pass1000kbpszones.265
ffmpeg -i 1.avi -c:v rawvideo -pix_fmt yuv420p -an -f rawvideo - | x265 --input - --fps 30000/1001 --input-res 848x480 --bitrate 1000 --pass 2 --sar 1:1 --zones 0,50,b=0.1 --stats "2passzones.log" -o 2pass1000kbpszones.265

I'll upload the utvideo source file used if you want, but it's probably a better idea if you upload your video that you can't get zones to work on. There might be some other condition or something peculiar about that specific zone

[LigH.de]

-Habanero- is correct; I just didn't notice that he works with 2-pass encoding. I compared 3 passes:

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --bitrate 1000 --pass 1 --csv-log-level 2 --csv foreman_cif_pal.pass1.csv
x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --bitrate 1000 --pass 3 --csv-log-level 2 --csv foreman_cif_pal.pass3.csv
x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --bitrate 1000 --pass 2 --csv-log-level 2 --csv foreman_cif_pal.pass2.csv

The bitrate distribution graphs reveal that zones are only respected during pass 1.

[poisondeathray]

2pass works in the example I posted above. 2pass works with foreman when I just tested it. It might not work exactly as you specify with the multiplier (e.g. I don't get 4x bitrate with b=4 in that section, or reduction to 0.1 in the b=0.1 section, but it's definitely increased and decreased respectively in those sections)

Maybe something wrong with --pass 3 ? I'll test it later when I get a chance

I'm not looking at csv output. I'm looking at the actual final video bitstream, both with an analyzer and with my eyes. Looking with eyes confirms it's adjusting in the correct direction

x265 --y4m "foreman_cif.y4m" --bitrate 500 --pass 1 --stats "2pass.log" -o foreman2pass500kbps.265
x265 --y4m "foreman_cif.y4m" --bitrate 500 --pass 2 --stats "2pass.log" -o foreman2pass500kbps.265

x265 --y4m "foreman_cif.y4m" --zones 0,99,b=4/100,199,b=0.1 --bitrate 500 --pass 1 --stats "2passzones.log" -o foreman2pass500kbpszones.265
x265 --y4m "foreman_cif.y4m" --zones 0,99,b=4/100,199,b=0.1 --bitrate 500 --pass 2 --stats "2passzones.log" -o foreman2pass500kbpszones.265

[LigH.de]

Uhm, no; the result of 2-pass VBR appears to have no recognizable drop in quality, whereas 1-pass ABR and 1-pass CRF clearly drop quality after frame 100.

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --crf 20 -o foreman_cif.1pass-crf.hevc

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --bitrate 1000 -o foreman_cif.1pass-abr.hevc

x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --bitrate 1000 --pass 1 -o foreman_cif.2pass-vbr.hevc
x265 --y4m foreman_cif_pal.y4m --zones 0,99,b=5.0/100,199,b=0.2 --bitrate 1000 --pass 2 -o foreman_cif.2pass-vbr.hevc

[Selur]

did you check if adding '--allow-non-conformance' otherwise there might be vbv&co restrictions at work

[poisondeathray]

1000kbps for a CIF clip might make it more difficult to see differences with the eye

Clearly there are measureable and visible differences when I did it with 500kbps for foreman, at least in the first zone. Clearly there are differences in the tree clip - just look at the screenshots. The tree test only had 1 zone.

To be clear, I'm testing 2pass (normal) vs. 2pass with zones . I'm looking at, and measured, the actual output files

Looking more closely - the first zone works well as expected in my 500kbps foreman test; but - the 2nd zone doesn't really work as expected. It might be because of the 2pass algorithm, it has to readjust and redistribute

[poisondeathray]

That HEVC analyzer is on sourceforge BTW , but hasn't been updated in a year
http://sourceforge.net/projects/gtilhevcanalyzer/

Another method to confirm is manually using ffprobe to get frame information, and graphing software. It's not as elegant as using plotbitrate.py - but I can't get that latter method working on windows. I started a thread for help but nobody answered

Here is the ffprobe syntax example . It will generate csv output wihch you can use in graphing software, excel, openoffice etc... I included frame type and size in this example (but only bitrate is plotted on the graph)

ffprobe -i "INPUT.ext" -v quiet -pretty -print_format csv -show_entries "frame=coded_picture_number,pict_type,pkt_size" -select_streams v:0 > OUTPUT.csv

Excuse the crappy looking graph but here is the tree test above , but it's pretty clear the 1 zone works to some degree (it might not be approx 1/10 the bitrate but clearly it's reduced that section)

[LigH.de]

Is there any reason not to trust the CSV logs' "used bits per frame" column?

And I care less about achieving the desired ratio as close as possible, more about having a certain difference between bitrate distributions with and without altering zones. According to the CSV logs, there is a certain difference between a b=5.0 zone and a b=0.2 zone only for CRF and ABR, but not for a 2nd pass in 2-pass VBR, using my GCC build of x265 v1.7+338.

1-pass CRF:


1-pass ABR:


2-pass VBR, pass 1:


2-pass VBR, pass 2:


Obvious enough now?

[poisondeathray]

I tend use multiple tools to do multiple checks, in case bias or error creeps in. If it's accurate, all tools should produce similar results. (And they do here, the graph below used the internal csv log-level, but I check with other tools and they are similar)

But to say that --zones doesn't work at all for 2pass - is wrong. Clearly there is a difference in the correct direction - it's just that the magnitude is less than 1passABR - and not working as well as you would want it to



In this longer test, the fluctuations are clearly larger with 1pass ABR (what you would expect with --zones) , but the filesize was almost 1.4x larger. What I think is going on , is the 2nd pass attempts to converge to the set bitrate, attenuating the effect of the zones . You can clearly see in the 2pass runs, that the red graph is lower than the blue at the 1st zone, but higher at the 2nd zone. If there was no effect at all, it should have produced identical results

(I cut the graph off at the top, to emphasize that there are differences at the lower end - but the 1pass ABR peak is actually larger)

[LigH.de]

Well, okay, so then the impact of zones on 2-pass encodes appears to be so low that further investigations from the developers should be necessary. The impact in the 2nd pass may even just be a leftover from the loss of precision.

[-Habanero-]

Thanks PDR for the graphs. So zones in 2pass do work but not like they're instructed. I'll encode separately in the meantime.

[-Habanero-]

So the encode is done, FINALLY. It took 9 days straight for this one 480p movie. I thought encoding credits separately would make things easier but instead I get a wrong bitrate despite using 2pass.
I encoded using 42 kb/s and I got 17 instead. The credits are unrecognizable. I'm gonna re-encode credits now with 101 kb/s if 42 gave me 17. Let's see what we get...

[-Habanero-]

Okay... now I got 82 kb/s and it took twice as longer to encode. This is annoying. The build I used in march didn't have this problem.

[Selur]

The build I used in march didn't have this problem.

my guess is that build which do not include 8bit and 10+bit encoding inside one binary might still work,....

[LigH.de]

So do compare, my last releases are "wholesome" (all single EXE's, single DLL's, and the multilib EXE)...

[-Habanero-]

I didn't think using the all-in-one EXE would matter. I did use the -D 10 switch which I thought invokes the 10-bit x265 inside the integrated app. Here is the command line though, it did say something about 2pass curve failing to converge or something. What does this mean?

M:\>avs4x26x.exe --x26x-binary x265 -D 10 "KK3c1.avs" --pass 1 --bitrate 62 --preset veryslow --ref
4 --bframes 16 --qcomp 0.7 --allow-non-conformance -o "KK3c.hevc"
avs [info]: AviSynth 2.60, build:Mar 31 2015 [16:38:54]
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 768x480
avs [info]: Video framerate: 24000/1001
avs [info]: Video framecount: 5081
avs4x26x [info]: "x265" - -D 10 --pass 1 --bitrate 62 --preset veryslow --ref 4 --bframes 16 --qcomp
 0.7 --allow-non-conformance -o KK3c.hevc --frames 5081 --fps 24000/1001 --input-res 768x480 --input
-csp i420
yuv  [info]: 768x480 fps 24000/1001 i420p8 unknown frame count
raw  [info]: output file: KK3c.hevc
x265 [info]: HEVC encoder version 1.7+338-8023786c5247
x265 [info]: build info [Windows][GCC 4.9.2][32 bit][noasm] 10bit
x265 [info]: using cpu capabilities: none!
x265 [info]: Main 10 profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(8 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : dia / 57 / 2 / 1
x265 [info]: Keyframe min / max / scenecut       : 23 / 250 / 40
x265 [info]: Lookahead / bframes / badapt        : 40 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 1 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 64 / 1
x265 [info]: Rate Control / qCompress            : ABR-62 kbps / 0.70
x265 [info]: tools: rd=2 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 early-skip signhide
x265 [info]: tools: tmvp b-intra fast-intra strong-intra-smoothing deblock sao
x265 [info]: tools: stats-write
x265 [info]: frame I:     21, Avg QP:38.28  kb/s: 1504.22
x265 [info]: frame P:    778, Avg QP:38.63  kb/s: 215.65
x265 [info]: frame B:   4282, Avg QP:47.91  kb/s: 25.38
x265 [info]: Weighted P-Frames: Y:16.3% UV:4.0%
x265 [info]: Weighted B-Frames: Y:37.7% UV:11.4%
x265 [info]: consecutive B-frames: 1.0% 0.6% 17.4% 8.8% 11.1% 12.0% 15.4% 16.9% 6.0% 3.6% 3.8% 1.3%
1.1% 0.8% 0.1% 0.1% 0.0%

encoded 5081 frames in 856.57s (5.93 fps), 60.63 kb/s, Avg QP:46.45

M:\>
M:\>avs4x26x.exe --x26x-binary x265 -D 10 "KK3c1.avs" --pass 2 --bitrate 62 --preset veryslow --ref
4 --bframes 16 --qcomp 0.7 --allow-non-conformance -o "KK3c.hevc"
avs [info]: AviSynth 2.60, build:Mar 31 2015 [16:38:54]
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 768x480
avs [info]: Video framerate: 24000/1001
avs [info]: Video framecount: 5081
avs4x26x [info]: "x265" - -D 10 --pass 2 --bitrate 62 --preset veryslow --ref 4 --bframes 16 --qcomp
 0.7 --allow-non-conformance -o KK3c.hevc --frames 5081 --fps 24000/1001 --input-res 768x480 --input
-csp i420
yuv  [info]: 768x480 fps 24000/1001 i420p8 unknown frame count
raw  [info]: output file: KK3c.hevc
x265 [info]: HEVC encoder version 1.7+338-8023786c5247
x265 [info]: build info [Windows][GCC 4.9.2][32 bit][noasm] 10bit
x265 [info]: using cpu capabilities: none!
x265 [info]: Main 10 profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features       : 3 / wpp(8 rows)
x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 62.00 kbit/s, expected: 182.82 kbit/s, avg QP: nan
x265 [warning]: internal error
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 57 / 4 / 4
x265 [info]: Keyframe min / max / scenecut       : 23 / 250 / 40
x265 [info]: Lookahead / bframes / badapt        : 40 / 16 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 4 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 64 / 1
x265 [info]: Rate Control / qCompress            : ABR-62 kbps / 0.70
x265 [info]: tools: rect amp rd=6 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 signhide
x265 [info]: tools: tmvp b-intra strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I:     21, Avg QP:45.62  kb/s: 1788.93
x265 [info]: frame P:    778, Avg QP:50.76  kb/s: 17.54
x265 [info]: frame B:   4282, Avg QP:50.94  kb/s: 8.48
x265 [info]: Weighted P-Frames: Y:9.6% UV:0.8%
x265 [info]: Weighted B-Frames: Y:20.3% UV:1.8%
x265 [info]: consecutive B-frames: 1.0% 0.6% 17.4% 8.8% 11.1% 12.0% 15.4% 16.9% 6.0% 3.6% 3.8% 1.3%
1.1% 0.8% 0.1% 0.1% 0.0%

encoded 5081 frames in 3420.68s (1.49 fps), 17.23 kb/s, Avg QP:50.89

[Selur]

min quant, vbv or something similar might be hindering the bit rate control,...

[LigH.de]

In 2-pass mode, the encoder tries to calculate a CRF value which would return the desired target bitrate as close as possible. You must have chosen a target bitrate so insanely low that the calculation of the optimal rate factor failed. If you want to use a scene with lowest possible quantization (e.g. for credits you don't read anyway), you should rather use a fixed quantizer 1-pass encode.

By the way: You are using the 32 bit EXE with a high bit depth. Such compiles don't use assembler optimized routines, no MMX/SSE instructions. No surprise it is slow as hell.

x265 [info]: build info [Windows][GCC 4.9.2][32 bit][noasm] 10bit
x265 [info]: using cpu capabilities: none!

The developers decided not to waste development time on optimizing software with too little RAM capacity to handle even FullHD, not to mention UDH, and does not even have the advantages of the double with and double number of CPU registers the 64 bit architecture offers, compared to the 32 bit instruction set.

[-Habanero-]

The problem is that I'm combining them. I don't trust this to be possible if I use 2pass for the movie but CRF for the credits. I keep settings the same except bitrate to minimize problems.

Yeah I'm using the 32-bit version because I'm on a 32-bit system. I do not even have more than 4GB of RAM so it was pointless to upgrade to a 64-bit system when I had the chance. In 2012 when RAM prices started to skyrocket (and STILL haven't come down to pre-2012 levels) it was even more pointless to upgrade.
Out of curiosity, how much is the 64-bit version faster?

[poisondeathray]

Out of curiosity, how much is the 64-bit version faster?

It seems about 2x faster on x64 here on a quick 1pass ABR test, but the version (from LigH.de) I'm using has CPU capabilities for the x86 version (it doesn't say "none" like yours) ??

x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZ CNT BMI2

If that build isn't using anything, not even older instruction sets like MMX, SSE2 etc... it's going to be abnormally slow

The one I tested was "x265_1.7+338-8023786c5247.GCC492"


Edit - Ahh, you tested 10bit - maybe there aren't any optimizations in the x86 branch

Edit#2 - Yes thats why. The 10bit x86 has none. It's about 8x slower than the 10bit x64 version

Edit#3 - doh! Ligh.de already said that. Not awake yet

By the way: You are using the 32 bit EXE with a high bit depth

But it's interesting that the x64 version of 8bit was still about 2x faster (it was on SD test, so no huge memory requirements). x264 speed difference wasn't as large

[-Habanero-]

Interesting, I will try x265 64-bit on gentoo. How do you compile x265 for linux? I'm using CMake but I don't know which generator to use. I'm guessing Unix?

[LigH.de]

The directory x265/build/linux contains a file "make-Makefiles.bash" and a file "multilib.sh"; this should be useful material to adapt your preferred batch from.

[-Habanero-]

CMake only asks me for the source code and build directories as well as which generator to use.

[LigH.de]

Did you check out the sources with Mercurial (hg) as documented in the x265 Wiki? Do not use git, it is not completely compatible to Mercurial.

http://x265.org/ => https://bitbucket.org/multicoreware/x265 => https://bitbucket.org/multicoreware/x265/wiki/Home

[-Habanero-]

Did you check out the sources with Mercurial (hg) as documented in the x265 Wiki? Do not use git, it is not completely compatible to Mercurial.

http://x265.org/ => https://bitbucket.org/multicoreware/x265 => https://bitbucket.org/multicoreware/x265/wiki/Home

No I looked at this http://packman.links2linux.org/package/x265/633396
Downloaded the files and did the 1 click install and it said everything went fine. Do I look for the installed program or do I use the RPM as I'd use an EXE on Windows?

[LigH.de]

You downloaded a package with a precompiled executable? Oh, and I assumed you wanted to compile it yourself from the source repository?!

I believe you should at first get to know for yourself what you want. And then explain it in detail without leaving any doubts.