x265 HEVC Encoder

[Selur]

MSVC or some (assembler) code which only fails on some systems.

[Selur]

and now I could reproduce the system with a MinGW build,...
(artifacts are not that visible as with the linked sample above, but they are there)

---

btw.:

--[no-]b-intra Enable intra in B frames in veryslow presets. Default enabled

Why is it by default enabled and only does stuff when using veryslow presets?
Shouldn't it be disabled then by default and only get enabled by those presets?
-> seems a bit confusing

--[no-]interlace <bff|tff> Indicate input pictures are interlace fields in temporal order. Default progressive

shouldn't the default be '--no-interlace' instead of 'progressive'

[LigH.de]

What do these Psy-RD artefacts depend on? Preset? Resolution? CPU instructions? I ran a sequence of all presets with a 640x272 video on a Phenom-II (SSE2) and did not spot issues. At least not as remarkable as LoRd_MuldeR described them.

[Selur]

@LigH: I think the presets do not use ' --psy-rd ' atm. so running the presets probably will not help.
To reproduce the issue I took a sample which:
a. doesn't have much motion; with a lot of motion/noise, these artifacts are not always easy to spot
b. is a few minutes long
I too do not see them as pronounced as the sample from LoRd_MuldeR shows them, but they are there.

[LigH.de]

Not explicitly repeating that I added this parameter does not mean I forgot it.

-o tos_60s_360_%preset%.hevc --preset %preset% --psnr --ssim --log 3 --csv tos_60s_360_%preset%.csv --psy-rd 1.0

[Selur]

Might be releated to crf (or psy-rd is disabled when using cq).

Calls used:

mencoder -lavdopts threads=8 -really-quiet -of rawvideo -o - -ovc raw -noskip -vf scale,format=444p,scale,format=i420 -forcedsubsonly -nosub -nosound -mc 0 "C:\Users\Selur\Desktop\version.avs" | x265 --preset veryslow --input - --input-res 440x80 --fps 24 --frames 240 --ctu 16 --crf 28 --psy-rd 2 --colormatrix bt470bg --output "H:\Temp\crf_20_psy-rd_2.265"

mencoder -lavdopts threads=8 -really-quiet -of rawvideo -o - -ovc raw -noskip -vf scale,format=444p,scale,format=i420 -forcedsubsonly -nosub -nosound -mc 0 "C:\Users\Selur\Desktop\version.avs" | x265 --preset veryslow --input - --input-res 440x80 --fps 24 --frames 240 --ctu 16 --qp 20 --psy-rd 2 --colormatrix bt470bg --output "H:\Temp\qp_20_psy-rd_2.265"

content of the Avisynth script simply is:

version()

-> try using crf instead of cq

[LigH.de]

I spotted some in the Sintel trailer now. It may be more obvious with fewer details and slower motion.

I never used CQ. The default is CRF 28, isn't it? ... Hmm, there are defaults for both CQ and CRF in the help output. And the log reports varying QP.
__

Confirmed by developers: "--crf 28.0" is the default, the other is outdated and confusing.

The Psy-RD artefacts in very slow motions look a bit like they may not reference the correct frame, the motion appears to be choppy in some regions.

Will upload a couple later: 720p50 parkrun veryslow, with and without --psy-rd 1.0; the umbrella will catch your attention ...
__

Done.

[LigH.de]

Some test results of the "noise reduction" (encoded with x265 v1.0+103) are being uploaded: tos_60s_fast_nr[0|100|1000].mp4

My first impression is that it can give a smaller file with a cleaner image and less motion-shaky artefacts.

[Selur]

@x265: regarding noise reduction:

An integer value in range of 100 to 1000,..

Why not simply use a range from 0-100 and internally map:
0 to disabled
x to reduction level = x * 100 with x being a float from 0 to 10.
Alternatively: Why not use a range from 0 to 1000?

Seems like a strange decision to use a range from 100 to 1000.

[x265]

We've pushed a number of improvements to x265, including faster performance at all presets from the default setting (medium) to Ultrafast. Your mileage may vary, but you may see up to 80% faster performance with certain clips at default settings, with very little impact on encoding efficiency.

The psycho-visual rate distortion optimization feature (psy-rd) is disabled by default, but you will probably notice improved visual quality with psy-rd strength somewhere between 0.2 and 0.5 (based on my very limited early testing).

There is also a new lossless mode. Of course, lossless encoding will generate high bit rates, but the decoded content should be identical to the uncompressed source.
--lossless forces full lossless coding of every frame and every CU.
--cu-lossless forces the encoder to perform RD cost analysis between lossy and lossless modes and choose the least cost mode for each CU.

Feedback is welcomed.

[Selur]

psy-rd still producing artifacts on slower/still scenes?
Any plans to change the 'nr-range' like I suggested?
Regarding lossless: are there special decoders needed, or should the normal decoder all be able to decode it? Does lossless coding do color space conversions (I hope not).
Any news regarding 2pass encoding?

Is the preset table 'http://x265.readthedocs.org/en/default/presets.html?highlight=preset' up-to-date? (with no psy-rd in it)

--cu-lossless forces the encoder to perform RD cost analysis between lossy and lossless modes and choose the least cost mode for each CU.

Shouldn't the lossy always 'win' here? If it doesn't shouldn't this be a foxed part of RD processing?

[El Heggunte]

Now it's official,

x265 version 1.1 0_o

Hello,

Release 1.1 has been tagged. This is an incremental update with
several important rate control improvements and a few new features.

= New Features =

1. Psycho-visual rate distortion optimizations. These RD optimizations
are only effective on presets which use RDO (rd levels 5 and 6).
Psy-rd is still considered experimental in this release and is not
enabled by default. We recommend evaluating with a low psy weight
factor, for instance: --rd 5 --psy-rd 0.4

2. Lossless coding. This release of x265 can create a bit-accurate
output bitstream by using --lossless. This feature disables rate
control and distortion metrics, and instead just reports the
compression ratio at the end of the encode. Lossless coding is
considered experimental in this release, we believe there is room for
improvement in both compression efficiency and performance.

3. Support for Y4M streams with more than 8bit depth (ffmpeg -i
vid.avi -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | ./x265 -
--y4m o.hevc)

= API Changes =

* new x265_picture.forceQp for qpfile functionality
* new param.levelIdc to force a decoder requirement level
* new param.psyRd for (experimental) psycho-visual rate distortion
optimizations
* new param.bIntraInBFrames to disable intra predictions in B slices
regardless of preset
* new param.noiseReduction, very similar to x264 noise reduction
* new param.bLossless to enable lossless coding (experimental)
* new param.bCULossless to include trans-quant bypass modes in CU RD
analysis
* new param.rc.rfConstantMin to limit rate factors in rate control
* param.rc.aqMode now defaults to 2 (to match CLI behavior)

new x265_encoder_parameters() function which retrieves a copy of the
active parameters from the encoder. x265_encoder_open() was modified
to ensure it never modified the param structure passed to the
function; it makes a private copy of the param prior to making any
modifications to it.

The default setting (the medium preset) was be adjusted to include the
--no-rect and --no-amp options, becoming faster (on average, about
70%, but as much as 90%), with a very slight (~ 1 - 4%) impact on
encoding efficiency.

We have sped up the ultrafast preset by about 10 to 30% (bigger
benefit at higher bit rates). There is a very small impact on encoding
efficiency, but you can always increase efficiency by using a slower
(higher quality) preset. We've also sped up the superfast, veryfast
and faster presets in a similar way.

= CLI Changes =

New options:
--level
--repeat-headers (older feature, newly exposed to CLI)
--nr
--lossless
--psy-rd
--crf-min
--no-b-intra
--cu-lossless
--qpfile

--tune fast-decode now also disabled intra in B frames

As always, the most detailed documentation for the command line
arguments can be found in our online documentation:
http://x265.readthedocs.org/en/1.1/

= Rate Control =

Single pass ABR received a lot of attention in this release, in
particular the tendency for ABR to undershoot and overshoot wildly in
the first two seconds of the video. We added two new features to ABR
to limit this tendency. First, we now amortize a portion of the cost
of I frames across many frames. Second, we limit frame parallelism
until we have about a half-second of P frames encoded. Together these
two changes have greatly improved the ability of single pass ABR to
arrive at the good QP for the first GOP without any large swings.

Further improvements were made to ABR to allow it to reach very high
bitrates.

We also did some re-balancing of CRF between Main and Main10 so they
achieve closer quality, and several fixes were made to VBV.

Recovery Point SEI are now generated at each keyframe

In the near future we will be focusing on two-pass encoding and making
mode decision more efficient.

--
Steve Borho

[Selur]

extending the list of questions from #311

--level

Will this enforce the vbv restrictions which belong to the level, or will this only indicate something which might not be sufficient for the stream?

[vhelp]

first, let me mention: i wish the official repository could provide the builds as they post their updates, so that performing encoding tests can be consistant from build to build. however, they or an entity mainly post the build version but not the actual encoder. i wait patiently for the latest build so that i can run tests on them, but days or weeks go by while several more build versions are posted. this creates gaps in the tests that people like myself perform, which can be a problem for us since we can not report any issues or pros and cons or something contributable to the project. this is all sad.

today, i ran tests against the following builds { 1.0.106 / 1.0.156 / 1.1.003 } (using ultrafast preset) although there are gaps (for reasons explained above) i can only report that, quality'wise, build 1.0.156 produced the best product, or the least amount of artifacts (h265 term for encoding artifact or pixelation) than the others. the finished encoded source was near perfect. a personal notation: as for tweeking encoding speed, i'm in favor of the slower parameters since quality is ultimately my end goal. thus, with respect to the above tests, and tipacle noisy h264 compression artifacts (from tv broadcast) video sources, these tweeks were not an improvement.

i can't quite figure out what has caused the differences or more exactly, what parameters and/or presets had changed across those tested versions. if i knew that info i could re-run those tests with the missing parameter changes and that would solve the difference between those tests. to note: just one value can change the scale of quality. accuracy of the results from these tests can provide a welth of info to the project. to do that successfully, the team should provide the preset parameters and values for a new build that incorporated those changes to the presets.

the other thing i wanted to bring up is, (i've asked this once before, a long time ago, maybe) is there an order (perhaps in terms of priority) that the parameters should be entered ?

also, does a parameter "cancell out" or overide or take priority of a parameter and/or value, whether in a preset or not in a preset ?

or it doesn't matter what order the parameters are entered ?

as always, thank you for the encoder updates, the team and project!

[DarrellS]

the other thing i wanted to bring up is, (i've asked this once before, a long time ago, maybe) is there an order (perhaps in terms of priority) that the parameters should be entered ?

also, does a parameter "cancell out" or overide or take priority of a parameter and/or value, whether in a preset or not in a preset ?

or it doesn't matter what order the parameters are entered ?

Good question. I tried my usual x265 presets that worked fine for the prior versions with two different downloads of 1.1+3 and it gets to the last frame and crashes but looking at the log file, there is no visible error in the command. I kept tinkering with my command but couldn't get anything to work. I changed the encoder to an earlier one and it worked fine with the same command although slower than 1.1.3 before crashing. Almost seems like another one of those permission issues that 64 bit operating systems suffer from.

[x265]

first, let me mention: i wish the official repository could provide the builds as they post their updates, so that performing encoding tests can be consistant from build to build. however, they or an entity mainly post the build version but not the actual encoder. i wait patiently for the latest build so that i can run tests on them, but days or weeks go by while several more build versions are posted. this creates gaps in the tests that people like myself perform, which can be a problem for us since we can not report any issues or pros and cons or something contributable to the project. this is all sad.

I wish we could provide you with builds also. As you may know, HEVC implementations require patent licenses from MPEG-LA. We fully support this, and all of our licensees pay these royalties. We can't give something away for free that we have to then pay for ourselves. Even if we charged money, we would have to track each copy (so that you could get updates for free), and this takes a fair bit of infrastructure. We've thought about this problem, but we're a core technology provider (B2B), not a consumer software company (B2C), and so we don't have this infrastructure. There are, however, a number of open-source projects downstream that utilize x265, including FFMPEG and Handbrake. If you're anxious to always get access to the latest build, you'll want to follow our easy-to-follow assembly instructions, to create your own builds.

today, i ran tests against the following builds { 1.0.106 / 1.0.156 / 1.1.003 } (using ultrafast preset) although there are gaps (for reasons explained above) i can only report that, quality'wise, build 1.0.156 produced the best product, or the least amount of artifacts (h265 term for encoding artifact or pixelation) than the others. the finished encoded source was near perfect. a personal notation: as for tweeking encoding speed, i'm in favor of the slower parameters since quality is ultimately my end goal. thus, with respect to the above tests, and tipacle noisy h264 compression artifacts (from tv broadcast) video sources, these tweeks were not an improvement.

i can't quite figure out what has caused the differences or more exactly, what parameters and/or presets had changed across those tested versions. if i knew that info i could re-run those tests with the missing parameter changes and that would solve the difference between those tests. to note: just one value can change the scale of quality. accuracy of the results from these tests can provide a welth of info to the project. to do that successfully, the team should provide the preset parameters and values for a new build that incorporated those changes to the presets.

We did explain the changes, as we made them, and then in the 1.1 update message. Ultrafast was changed from --rd 3 to --rd 2, and from --lft to --no-lft (we turned off the loop filter). If you want the old Ultrafast back, just add --rd 3 and --lft. I would suggest that you try --lft only, as this probably has more impact to quality than the change from --rd 3 to --rd 2. Or you could just try the superfast preset.

the other thing i wanted to bring up is, (i've asked this once before, a long time ago, maybe) is there an order (perhaps in terms of priority) that the parameters should be entered ?

also, does a parameter "cancell out" or overide or take priority of a parameter and/or value, whether in a preset or not in a preset ?

or it doesn't matter what order the parameters are entered ?

Order doesn't matter. Options that are set by performance presets will be overridden if you specify any of those options in your command line.

as always, thank you for the encoder updates, the team and project!

You're welcome.

[Selur]

Since my questions got lost/ignored,..

@x265: Got a few questions:

1. Does '--level' enforce the vbv restrictions which belong to the level, or will this only indicate something which might not be correct/sufficient for the stream?
2. Does psy-rd still producing artifacts on slower/still scenes?
3. Any plans to change the 'nr-range' like I suggested? (from 100-1000 to 0-100)
4. Does lossless require special decoders or should the normal HEVC decoder already support it?
5. Does lossless coding do color space conversions (I hope not).
6. Reading that '--cu-lossless forces the encoder to perform RD cost analysis between lossy and lossless modes and choose the least cost mode for each CU. ' Shouldn't the lossy always 'win' here? If it doesn't shouldn't this be a fixed part of RD processing and not optional?
7. "Order doesn't matter. " -> What happen if I call '--crf 16 --crf 18' ?

[LigH.de]

2. Probably yes, if the strength exceeds 0.5; it is not yet rescaled or "completely fixed", AFAIR.

[El Heggunte]

^
^

4. Lossless is decoded normally by both OpenHEVC and Lentoid.
5. Apparently it doesn't (N.B., I have run only one test on a YV12 sample).

[LigH.de]

More to 2.: No, even with only strength 0.3, artefacts are still visible in "foreman". Somehow I believe that these areas may refer to the wrong base frame number.

[El Heggunte]

Hmmm, I have just tried the lossless thing upon 4:2:2 and 4:4:4 sources, but neither LAV Video nor the Lentoid Decoder like the resulting streams

Anyway, here goes a one-frame wonder (for our friend poisondeathray )

[Selur]

@El Heggunte: lav decodes the sample you uploaded without a problem here,.. (using latest MPC-HC nightly build)

[Stears555]

@El Heggunte: lav decodes the sample you uploaded without a problem here,.. (using latest MPC-HC nightly build)

MPC-HC is a backward player. It has fewer options and it is less customizable than potplayer !

[El Heggunte]

Selur, I didn't say that the uploaded sample is of the "unplayable" type

[Selur]

Yes, Orchid_Cosplay.mp4 plays fine in latest nightly MPC-HC (which uses LAV-Filter, avcodec for decoding)
---

@x2665:

still open questions:

• Does '--level' enforce the vbv restrictions which belong to the level, or will this only indicate something which might not be correct/sufficient for the stream?
• when crf-min with crf+vbv is used which setting is more important crf-min or vbv (I hope vbv). If crf-min would be more important, vbv-violations could occur!
• Any plans to change the 'nr-range' like I suggested? (from 100-1000 to 0-100)
• Reading that '--cu-lossless forces the encoder to perform RD cost analysis between lossy and lossless modes and choose the least cost mode for each CU. ' Shouldn't the lossy always 'win' here? If it doesn't shouldn't this be a fixed part of RD processing and not optional?
• "Order doesn't matter. " -> What happen if I call '--crf 16 --crf 18' ? (in x264 the later would overwrite the earlier)

answered through videohelp community, so far:

• Does psy-rd still producing artifacts on slower/still scenes?
  Yes, even with 0.3 psy-rdo stoll propduices artifacts on sources like foreman.
• Does lossless require special decoders or should the normal HEVC decoder already support it?
  Lossless is decoded normally by both OpenHEVC, Lentoid and LAVFilter. (Lentoid only supporting 4:2:0 decoding)
  So seems like no special support is needed if the used color space is supported.
• Does lossless coding do color space conversions (I hope not).
  Good, doesn't seem like any color conversion takes place.

Also:

• http://x265.readthedocs.org/en/default/cli.html?highlight=crf#cmdoption--crf
  defines --max-crf and --min-crf, but the command line uses --crf-max and --crf-min

Cu Selur

[LigH.de]

The latest patch is supposed to reduce the RAM utilization. May be interesting to check v1.1+193 against an older build regarding encodability of 4K video even with a 32-bit build.
__

Stable at 1.6 GB with preset slow, looks promising.

[LigH.de]

Impressive results with the command line

--crf 30 --preset slower --aq-mode 2 --aq-strength 1.5 --psy-rd 0.3

Even the lawn in the background, which used to lose a lot of detail, is now satisfyingly persistent.
__

P.S.:

Another really impressive efficiency result for the 4K clip of in_to_tree; the sky is almost satisfying even at such a low bitrate.

[LigH.de]

And there is version 1.2; just waiting for the verbose summary.

[El Heggunte]

Yes$S......

From: Steve Borho <steve@borho.org>

Reply-To: Development for x265 <x265-devel@videolan.org>

To: Development for x265 <x265-devel@videolan.org>

Subject: [x265] [ANN] x265 1.2 released

Date: Thu 2014-07-10 05:46 PM


Hello,

x265 release 1.2 was tagged today. This was a regularly scheduled
release with improvements in performance, major improvements in memory
usage, and improved psy-rd behavior.

There were a few of new options introduced:

--cu-stats, x265_param.bLogCuStats - enabling logging of CU stats

--hrd, x265.bEmitHRDSEI - enable HRD SEI signaling

--ipratio/--pbratio were exposed to the CLI

--lambda-file - allows experimentation with lambda tables

Plus a number of options added for multi-pass encoding (incomplete). I
will document those in the next release after the feature is complete.

Full documentation for the features supported in the release can be
found at http://x265.readthedocs.org/en/1.2/.

-- 
Steve Borho

[Selur]

Since the x265 folks seem to ignore my questions at all here are my thoughts about the options:

--cu-stats, --no-cu-stats

Records statistics on how each CU was coded (split depths and other mode decisions) and reports those statistics at the end of the encode. Default disabled

Is there any gain in using this for a non-developer?

--hrd, --no-hrd

Enable the signalling of HRD parameters to the decoder. The HRD parameters are carried by the Buffering Period SEI messages and Picture Timing SEI messages providing timing information to the decoder. Default disabled

okay, is there any use for this at the current time?

--ipratio <float>

QP ratio factor between I and P slices. This ratio is used in all of the rate control modes. Some --tune options may change the default value. It is not typically manually specified. Default 1.4

QP ratio factor between P and B slices. This ratio is used in all of the rate control modes. Some --tune options may change the default value. It is not typically manually specified. Default 1.3

okay (hopefully there will be a documentation about the --tune option in the future which shows what they do)

--lambda-file <filename>

Specify a text file containing values for x265_lambda_tab and x265_lambda2_tab. Each table requires MAX_MAX_QP+1 (70) float values.

The text file syntax is simple. Comma is considered to be white-space. All white-space is ignored. Lines must be less than 2k bytes in length. Content following hash (#) characters are ignored. The values read from the file are logged at --log-level debug.

Note that the lambda tables are process-global and so the new values affect all encoders running in the same process.

Lambda values affect encoder mode decisions, the lower the lambda the more bits it will try to spend on signaling information (motion vectors and splits) and less on residual. This feature is intended for experimentation.

okay, so no way a normal user will/can use this.