[HEVC] x265.EXE: mingw builds

[Jacobr]

The second code uses 4 threads and 17 streams but still only encodes at .70 fps using my Q6600 CPU.

Oh, and MP4Box won't accept .hm10

I know, they weren't meant to be very 'efficient', but rather a test, plus i discovered my gui won't mkv mux the b frames .hm10 . Mp4 still muxes.
Mp4box accepts any compliant hevc raw stream if you only set "fmt", at least on my end.

Jacobr, I've tried everything and could not audio. I switched mp4box.exe, graph2.exe, x265.exe and no audio.

Originally it was encoded by ffdshow and muxed with graph->mkvmerge or mp4box.

[LigH.de]

Is there a slightly more verbose explanation of the parameters?

Simple sample: "ConstQP" — is it a hard constant quantizer, or a "Constant Quality Parameter(?)" with varying quantization, similar to CRF in x264?
__

P.S.:

x265_0.3+562-ea85b67907ca appears to be not leaky anymore, has a reasonable "Private Bytes" value mainly below 1 GB while processing the Sintel 1080p trailer.

[vhelp]

encoding specs: 720x480 24fps using 122 frames test file 
--frames 122 --q 20 --keyint 24 --rect --max-merge 1 --hash 1    --wpp --no-rdo --no-rdoq --tu-intra-depth 1 --tu-inter-depth 1 --no-tskip --input "_video.y4m" -o "video.hm10" 
-------------------------------------------------------  
computer #1, desktop pc, amd dual core x2 3600+, 2g ram winxp home sp2, results: random crashes, incomplete encodes 
x265 [info]: detected SIMD architectures SSE2 SSE3  
x265 [info]: thread pool with 2 threads, WPP enabled (8 streams) 
-------------------------------------------------------

i tried again many times on my desktop pc but it crashes, or the x265 (taskmgr) stalls (to put it bluntly) at zero cpu pecent. strange thing happend when i got home last night: i turned on my desktop pc, loaded my gui for x265 and it completed successfully. i started doing other work, and then fell asleep. this morning, i went straight to the gui and proceeded to encode, but it crashed, or stalled at zero. i re-started the gui and the same thing, i even ran straight from dos console window, and the same thing, it would crash or stall in taskmgr.

[DarrellS]

Make sure if you are copying command arguments from this thread that there are no double or triple spaces between commands. Also if you paste part of your command into another command make sure that you don't get something like this... ""video.hm10". I've gone through commands that didn't work and found both of these issues. Also, your input file must be multiples of 64. Whether it's 64x64, 320x320, 320x240, 512x384, 640x480, 720x480, 960x540, 1280x720 or 1920x1080. I assume anything on this list will work but you'll have to test to make sure...

http://en.wikipedia.org/wiki/List_of_common_resolutions

I changed up my argument a little and got these numbers using a 1280x720 file. None of my high resolution files will play in MPC-HC but they all play in the Osmos4 player. Seems the other players have a lot of work to do if they're going to play these files with wpp.

Here is my bat file for creating the y4m file, encoding to .hvc and muxing to mp4...

avs2yuv.exe "C:\Tools\x265\input.avs" -o input.y4m & x265 --frames all --q 24 --keyint 100 --max-merge 1 --hash 1 --wpp --no-rdo --no-rdoq --tu-intra-depth 1 --tu-inter-depth 2 --no-tskip --input "input.y4m" --output "video.hvc" & C:\Tools\GPAC\mp4box.exe -add "C:\Tools\x265\video.hvc" -new outfile.mp4

...and my last encode using Q6600 processor and Windows XP Pro...

[DarrellS]

The uncompressed AVI that I used for the input encoded at 15 fps so the x265 encode was 1/3 as fast and x264 at medium preset was 10.58 fps so the x265 encoder was almost 1/2 as fast. Wait a minute, I just encoded x264 at slow and it was 59.51. I need to double check my x264 arguments. They're both crf 18 and tune film so I don't know what happened there. At that rate, x264 is is over ten times as fast.

Uncompressed - 2,073,627 KB - 15 fps
X264 Medium CRF 18 - 412 KB - 10.58 fps
X264 Slow CRF 18 - 444 KB - 59.51 fps (whacko)
X264 Placebo CRF 18 - 444 KB - 14.26 fps
X265 CQ 24 - 252 KB - 4.81 fps

[LigH.de]

New test finished overnight.

Encoded with x265 ver. 0.3+562-ea85b67907ca: "x265.exe sintel_trailer_1080.y4m -o sintel_trailer_1080.hevc -q 12 -b 3 --weightp --weightbp --rdpenalty 1"

Decoded with HM 12.0 r3541 release: TAppDecoder.exe -b sintel_trailer_1080.hevc -o sintel_trailer_1080_hevc.yuv

Checked with AviSynth MT 2.6a4 and RawSource26-20130826.

Artifacts every now and then; mostly green, also pink; sometimes even red, cyan, and black. Usually smeared forward and even backward by motion vectors. Example: Frame 464

[El Heggunte]

New test finished overnight.

Encoded with x265 ver. 0.3+562-ea85b67907ca:

"x265.exe sintel_trailer_1080.y4m -o sintel_trailer_1080.hevc -q 12 -b 3 --weightp --weightbp --rdpenalty 1"

Then x265 still has a problem with B-frames, WPP, or when both are used at the same time. ( )

Semi-OT:

And if the well-known PDF is to be trusted..... , well, I confess that I thought that
"there can't ever exist such thing as an IBBBBBBBBBBBIBBBBBBBBBBBIBBBBBBBBBBB..... GOP structure"

[El Heggunte]

build 0.3+594-06c30405d308 gets out of the oven:

[LigH.de]

Compared with r562, r594 has a different memory consumption behaviour (sometimes using more than 1 GB). The decoding may reveal interesting differences...

[DarrellS]

Compared with r562, r594 has a different memory consumption behaviour (sometimes using more than 1 GB). The decoding may reveal interesting differences...

That might be a good thing for encoding speed? I thought on my last encodes that it could probably use a little more memory since it was only using 900 MB to a little over 1 GB. I can use a little over 2GB before it crashes in XP.

I'll run another test later and see if it improves the speed any.

[LigH.de]

No, it may simply mean that it creates a different GOP structure, e.g. keeping a few P frames in memory which didn't exist before (see the mentioned IBBB... structure EH wonders about).

Unfortunately, I had to cancel the encode due to moving between work and weekend flat, need to run another one.

[vhelp]

it may appear to be that the crashes i've been experiencing are due to low system memory during encoding. i've been trying to figure out how through trial and error. and then it hit me when i was shutting down the last app..the gui. i closed it down, nothing else was opened and re-ran it. all the sudden, the crashing stopped. i ran it 5 times before i decided to stop. the encoder may be checking for available memory and ok'ing it, but when you have other apps open, like firefox and many tabs open, the x265.exe crashes randomly. my guess, blaim it on poor memory managment. i have 2gigs on my desktop pc, plenty for my needs. and the earlier x265.exe apps never crased. they required 70mb to encode, per taskmgr. the version after that required 126mb, but the latest version, 06c3040, currently requires 146mb. i'm afraid that this memory requirement will keep going up. i will test further to be sure.

[DarrellS]

Yeah, you probably need 4 GB of memory on a 32bit OS. I've only got Firefox opened right now and Task Manager says it's using 464kb of memory. I use Firemin for Firefox since it's a memory hog and it's using 528 KB of memory and Task Manager says I'm using 645 MB of memory. I'm not sure where I was crashing, 2.5 GB I believe so that would explain your crashes if you've only got 2 GB of memory.

I haven't had a chance to do another encode with the new encoder. I fell asleep after my last post and just woke up. I must've been worn out.

[vhelp]

..but 06c3040 doesn't crash on my 500mb ram (atom) netbook. intel vs amd. all my crashes are happening on my amd cpu. the netbook i am using has the atom chip, 1st generation, i believe. not sure, someone better at cpu/chipset might have a better answer. this is the sylvania meso netbook 500mb 800x600 w/ xp home.

i started working on my own browser (with tabs) but got interuped with other pressing projects. i did not check to see how much ram it consumes. i should check that, the next time i'm in that delphi project.

someone savy in c/c++ should have a look at the c code for the yuv and y4m input source. this should be easily upgraded to accept avisynth scripts, or at very least, a genaral avi reader. according to the c code, there are two options for accepting/reading in the video source for yuv and y4m. add a third option to read in .avi and it should accept it. just add in the routine a routine to convert the avi (or avs) to yuv or y4m, whichever is easiest, and you're done. well, it looks simple enough to me (if it were pascal) but it is in c/c++ can someone have a look at the source code ? i don't have the source in front of me at the place i saw this at. i'll look later since i am blocked from that website at work.

[vhelp]

ok, i found the source. it is on the bitbucket website, here

https://bitbucket.org/multicoreware/x265/src/0e0a822fd344bf0e86f4ec8989acb6432fd10ef4/...rce?at=default

select the Input link, (this whole section (yuv parts) would need to be updated) and view the yuv.cpp since it is the least amount of code to understand.

see this:

#include "input.h"
#include "yuv.h"
#include "y4m.h"
#include <string.h>

using namespace x265;

Input* Input::open(const char *filename)
{
    const char * s = strrchr(filename, '.');

    if (s && !strcmp(s, ".y4m"))
        return new Y4MInput(filename);
    else
        return new YUVInput(filename);
}

and also this:

#include "yuv.h"
#include "PPA/ppa.h"
#include <stdio.h>
#include <string.h>

using namespace x265;
using namespace std;

YUVInput::YUVInput(const char *filename)
{
    ifs.open(filename, ios::binary | ios::in);
    width = height = 0;
    depth = 8;
    buf = NULL;
}

YUVInput::~YUVInput()
{
    ifs.close();
    if (buf) delete[] buf;
}

int YUVInput::guessFrameCount()
{
    ifstream::pos_type cur = ifs.tellg();

    ifs.seekg(0, ios::end);
    ifstream::pos_type size = ifs.tellg();
    ifs.seekg(cur, ios::beg);
    int pixelbytes = depth > 8 ? 2 : 1;

    return (int)((size - cur) / (width * height * pixelbytes * 3 / 2));
}

void YUVInput::skipFrames(int numFrames)
{
    int pixelbytes = depth > 8 ? 2 : 1;

    int framesize = (width * height * 3 / 2) * pixelbytes;

    ifs.seekg(framesize * numFrames, ios::cur);
}

// TODO: only supports 4:2:0 chroma sampling
bool YUVInput::readPicture(x265_picture_t& pic)
{
    PPAStartCpuEventFunc(read_yuv);

    int pixelbytes = depth > 8 ? 2 : 1;

    int bufsize = (width * height * 3 / 2) * pixelbytes;

    if (!buf)
    {
        buf = new char[bufsize];
    }

    pic.planes[0] = buf;

    pic.planes[1] = (char*)(pic.planes[0]) + width * height * pixelbytes;

    pic.planes[2] = (char*)(pic.planes[1]) + ((width * height * pixelbytes) >> 2);

    pic.bitDepth = depth;

    pic.stride[0] = width * pixelbytes;

    pic.stride[1] = pic.stride[2] = pic.stride[0] >> 1;

    ifs.read(buf, bufsize);
    PPAStopCpuEventFunc(read_yuv);

    return ifs.good();
}

obviously, the other parts (files) would have to be udpated/modified somewhat as well, but this seems to be the correct place to look at in adding the avi/avs support. just convert to the yuv raw format that the 'buf' needs to work with, and might work.

[Koti]

x265 [info]: HEVC encoder version 0.3+614-0e0a822fd344
x265 [info]: build info [Windows][GCC 4.8.1][32 bit] 8bpp

[El Heggunte]

Thanks a lot ,

now included in post #1.

[vhelp]

real-time piping of line by line progress report not working in my gui. but only works when inside a separate dos console window.

[vhelp]

resolved..i was copying the --input sourcefile to another directory, but it was moved instead. and i was in the new folder when i was in a dos console window while the encoder was working fine there--all this time to fiture that out

[DarrellS]

OK, something weird in the 594 build. I got 4.80 fps in 562 but in 594, I only got 3.89 fps using the same input file and the same settings. It looked good about half way through at 5.38 fps then suddenly dropped. My CPU also dropped to under 50% and my memory dropped to 812MB.

Will try 614 and see what happens.

EDIT: 614 much better! Still not using very much memory though.

[LigH.de]

r594: I believe someone still mixes up things. And I fear: What if it is not the reference software?!

Decode log, GOPs with up to 3 consecutive P-slices between B-slices

Testing r614 then...
__

Maybe not yet. Any reason why weighted prediction parameters got removed?

[Koti]

Maybe not yet. Any reason why weighted prediction parameters got removed?

326e375 - x265: disable weighted unipred CLI option
dd2e849 - x265: disable weighted bipred CLI option

The recent changes to TComPrediction:PredInterLumaBlk() to generate subpel
on demand almost certainly busted weighted prediction.

[vhelp]

is the *newest* x265.pdf in richtext or ascii text format somewhere ?
my browser doesn't open it and my pdf reader doesn't offer option to copy/paste it.
i want it in the same format so i can insert its details correctly in my gui. i statred typing it in but man i have soo many typos. if its avail somewhere to d/l, thanks in advanced.

edit: i'm retyping it. i will post here in separate post when complete. you'll just have to spell check it.

[El Heggunte]

Are you talking about the Evaluator's Guide released on last August 13?
If your answer is "yes", well, ... Its PDF-level = 1.4 ( = requires Acrobat Reader 5 or higher ).
And is fully "functional" in SumatraPDF or STDU Viewer, for example.

[vhelp]

good, ty..i have the Sumatra PDF v1.1. didn't know it has a selector/copy funtion, but i figured out how to use:

1. press and hold Ctrl key and drag mouse around area to select
2. press Ctrl-C
3. and paste into your editor, ie notepad

[vhelp]

what is the purpose of the following parameter ?

-r, --recon : reconstructed image YUV or Y4M output file name

i don't understand why we need a duplicate of the source (yuv/y4m) file.

[DarrellS]

what is the purpose of the following parameter ?

-r, --recon : reconstructed image YUV or Y4M output file name

i don't understand why we need a duplicate of the source (yuv/y4m) file.

I don't know. I haven't used it since the beginning because it was part of the other encoders. I didn't see anything in the PDF that said you had to use it so I haven't.

The Lentoid encoder allows you to use any input file (well almost) where you have to use AVI with this encoder (uncompressed I presume).

[LigH.de]

Looks like it is to support debugging by writing a decoded output of the just even encoded bitstream. This may save me an additional call of the HM decoder; but I would have to test how that works at all, and possibly combine it with the CSV debug output.

[vhelp]

positive report:
looks like this latest version 614 - 0e0a822 is stable. no crashes so far from my desktop pc.

@ DarrellS, i can't believe your system is using that much ram for the hevc encodes w/ x265.exe app. but from your last posted pic (post # 410) of your taskmgr usage looks like you are showing the total system used memory from the Performance tab. i've been giving mem usage from the processes tab, for x265 only. currently it is at 146mb when using this command on a 720x480 24fps 122 frames as source:

--frames 122 --q 20 --keyint 24 --rect --max-merge 1 --hash 1    --wpp --no-rdo --no-rdoq --tu-intra-depth 1 --tu-inter-depth 1 --no-tskip --input "video.y4m" -o "video.hm10"

someone commented on the extra "spaces" i use between command params (used for formatting purposes when reporting multiple params) and it does not interfear with x265 processing. plus, i also have a custom coded TrimEX() function i write since delphi doesn't have one, since when i trim spaces, i want to remove all spaces except at the param string, ie [ --rdo ].

here is the code, in case it can help others without this delphi function.

function TrimEX(s: string; ch: char): string;
var index: integer;
begin
  index:=0;
  while (pos(' ',s)>0) and (index<=length(s)) do begin
    inc(index);
    if (s[index]=' ') and (s[index+1]=' ') then begin
      delete(s,index,1); dec(index);
    end;
  end;
  result := s;
end;

usage: 

CleanParamStr := TrimEX(ParamStr,' ')

[LigH.de]

Without weighted prediction parameters, x265 r614 went through; but it still either uses B-frames very sparsely, or mixes up maximum consecutiveness for P and B frames.

Or HM 12.0 mixes up P and B slice names?! No, x265 reports a lot less B than P frames, too. Is there a reason for this decision (besides a lack of implementation of features for many consecutive B-frames)? I would still expect several B-frames being efficient (bitrate sparing), as long as they are effective (correctly implemented).

At least I could not spot any macroblock errors as they used to appear in earlier versions with the accidental "B-frames only GOPs"...