x264 Thread limit?

[Jumper1k]

Hey.

Just one simple question. Is there any way to increase number of threads in x264? Am using MeGUI to encode and the maximum number of Threads i can set is 16. I have Ryzen 2700 and i read somewhere that i should be able to set it to more threads (8 in my case prob ... correct me if am wrong). Anyway I tried to stress my CPU as much as i can and it only work about 55% of max power. I remember on Ryzen 1700x i don't remember how but my mate overdrived this limit and from 70% it jumped to 100% (but he doesn't remember how either ). Any ideas?

Thanks,
Jumper1k.

[jagabo]

I don't know about MeGui, but the command line option in x264 is "--threads #" where # is the number of threads you want to use, eg "--threads 24". Typically, you want about 1.5x the number of threads your processor supports. But if you have a bottleneck elsewhere in your processing adjusting x264 threads won't help.

[hello_hello]

Try adding something like this to the x264 custom command line section.
--thread-input --threads 24

Here's what x264 does by default. http://www.chaneru.com/Roku/HLS/X264_Settings.htm#thread-input

If you're using slow Avisynth filtering that'll probably be the bottleneck and increasing the number of x264 threads won't make any difference.
That can be an opportunity to use a slower x264 preset though. If the encoder is currently waiting around, a slower speed preset might increase quality a little, and also increase CPU usage a little, without slowing the encoding speed.

If CPU usage is down around 50% or slightly higher, and you are using slow filtering, the simplest solution is to run two encodes at the same time, or split the encode into halves and encode them together. Be sure to add --stitchable to the x264 command line to prevent problems when appending the encoded video.

[hello_hello]

Damn, jagabo beat me to it!

[KarMa]

You don't need to set the thread count, it should just be able to know on it's own. The actual thread limit is much higher than 16 but Megui knows most people don't need that much. I'm going to venture to say that all of your threads are working on x264 but there is a bottleneck. Either you are using a fast x264 setting which won't fill your CPU usage to 100%. You have a decoding bottleneck where you can encode faster than you are able to decode. Or you are applying additional filters (deinterlacing, etc etc) which are single core and can't work fast enough.

What x264 settings are you using exactly. Are you getting even usage across all cores? What filters are you using? What type of video is your source?

[jagabo]

You can check x264's metadata in the encoded video to see how many threads it used. MediaInfo shows that info.

[Jumper1k]

--preset veryslow --crf 20 --threads 16 --qpmin 10 --qpmax 51

that's my profile and same profile i used on my mates desktop, but we had to set up more threads to make it work as it should. When i put threads more than 16 manually in profile (coz MeGUI or x264 settings exacly doesn't allow me to do it) it works the same, but when i come back to config in megui UI freaks out and i can't even enter into it. I have to set threads lower to have access to the profile from MeGUI level. Thread input is set by default, but not sure if it works with more than 1 thread (at-least it appear when i lower no of threads).

In avisynth script i don't have much which is i just load up file previously edited in other program and saved as lossless (and ConvertToYV12 ... sometimes point resize when i work with low res videos).

About bottleneck ... x264 needs just CPU power. Can't imagine what could cause such thing...
Cores usage ... on every same which is ~55%.

[KarMa]

Sounds like a decoding of source bottleneck, where x264 is encoding more frames than can be decoded by the given decoder.

Edit: You can use something like AVSMeter, to figure out how fast your avisynth script can provide frames to x264. Just drag a drop your .avs script on AVSMeter.exe (probably don't use the 64 version unless you are using 64 Avisynth, I'm going to guess).

[Jumper1k]

AvsMeter doesn't work somehow ... cannot load avisynth.dll and when i point it out:

Cannot load avisynth.dll

Dependencies that could not be loaded:
MSVCP140.dll
VCRUNTIME140.dll

Note:
Visual Studio 2015/2017 Runtime doesn't seem to be installed

I installed Visual Studio 2017 Runtime and still it doesn't work for me.

[KarMa]

Oh, you probably don't have avisynth even installed as you probably use MEGUI only. Megui has an included avisynth but you can also have a installed version which AVSMEter needs. You can download the official 2.6 32-bit version here, https://www.videohelp.com/software/Avisynth

[jagabo]

I installed Visual Studio 2017 Runtime and still it doesn't work for me.

Make sure you install the right "bitness" -- 32 bit runtimes for 32 bit AviSynth, or 64 bit runtimes for 64 bit AviSynth. Note that 32 bit editors/encoders require 32 bit AviSynth and AviSynth filters (even under 64 bit Windows), 64 bit editors/encoders require 64 bit AviSynth and 64 bit filters. Neither of the two environments can see the other or its components.

I believe the actual thread limit for x264 is 128.

Another possible bottleneck with lossless sources is disc speed.

[Jumper1k]

[Attachment 47454 - Click to enlarge]

[Attachment 47455 - Click to enlarge]

Still don't see where might be bottleneck... program itself most likely due settings. Threads in AVSMeter i see more than i can only set in x264.

Just in case i couldn't launch AVSMeter before coz of Admin permission.

[KarMa]

Set the GUI Threads to 0 and put --threads 24 or whatever number in the custom command line at the top of your picture.

Assuming you are not encoding above 400fps, then decoding is probably not the bottleneck based on your picture. Also probably not a Hard Drive read speed problem.

[Jumper1k]

Still the same ...
I have SSD then yeah ... as i wrote before i don't imagine bottleneck in my case, but just option which blocks performance. On the same exacly copy of MeGUI i remember we could set it up to 20 threads in options even, but i don't remember what else he had to change to remove this cap.

I downloaded new copy from soundforge ... the same situation.

[Groucho]

Oh, you probably don't have avisynth even installed as you probably use MEGUI only. Megui has an included avisynth but you can also have a installed version which AVSMEter needs.

In AVSMeter one can select a specific avisynth.dll using the "-avsdll" switch:

Command line usage 1 (analyse a script):  AVSMeter script.avs [switches]

Switches:
  -avsdll             Specify avisynth.dll to be used
  -info   [-i]        Display clip info
  -log    [-l]        Create log file
  -csv                Create csv file
  -gpu                Display GPU/VPU usage (requires GPU-Z)
  -range=first,last   Set frame range
  -timelimit=n        Set time limit (seconds)
  -priority=n         Set process priority (1:low, 2:normal, 3:high)
  -o                  Omit script pre-scanning

[Jumper1k]

As you see (or not) I posted screenshot from AVSMeter already i just had no admin permission forgot to give one to command prompt. (and i used that option to spot the avisynth attached to MeGUI)

[KarMa]

On the same exacly copy of MeGUI i remember we could set it up to 20 threads in options even, but i don't remember what else he had to change to remove this cap.

I downloaded new copy from soundforge ... the same situation.

Shouldn't really matter if the GUI supports it because you can manually entering in the --threads # in the top box.

It might be one of the assembly options that is slowing down x264 but that's a guess and the documentation is not great on those and it's not easy trouble shoot your problem online. The easiest fix for you would to just encode two videos at the same time using Megui's Workers. So when two videos are in the queue, they both will get processes at the same time and hopefully using 100% of your CPU.

[hello_hello]

Jumper1k,
I have no idea why you're having problems with MeGUI's custom command line. It works fine for me (I'm running a quad-core and XP). MeGUI possibly limits the thread count to 16 in the GUI, as apparently more than 16 threads can start to impact quality.

If you're decoding with ffms2, check to see the decoding thread count isn't set to 1 in Options. Normally it's a good idea (so it's the default), but for decoding a lossless source, increasing it or setting it to zero shouldn't cause problems. Or index with LSmash instead. You're not using AviSource or DirectShowSource, are you?

Does the command line at the bottom of the encoder configuration show what you've entered in the custom command line section? ie

--preset slow --tune film --crf 18.0 --threads 20 --sar 1:1

http://www.chaneru.com/Roku/HLS/X264_Settings.htm#threads
Enables parallel encoding by using more than 1 thread to increase speed on multi-core systems. The quality loss from multiple threads is mostly negligible unless using very high numbers of threads (say, above 16). The speed gain should be slightly less than linear until you start using more than 1 thread per 40px of vertical video, at which point the gain from additional threads sharply decreases.

I tried two custom command lines, specifying a different number of threads each time.

--[Information] [13/12/18 3:33:44 AM] custom command line: --threads 24
--[Information] [13/12/18 3:33:44 AM] Job command line: "C:\Program Files\MeGUI\tools\x264\x264.exe" --preset slow --tune film --crf 18.0 --threads 24 --sar 1:1 --frames 88954 --output "D:\test.mkv" "D:\test.avs"

--[Information] [13/12/18 3:35:48 AM] custom command line: --threads 20
--[Information] [13/12/18 3:35:48 AM] Job command line: "C:\Program Files\MeGUI\tools\x264\x264.exe" --preset slow --tune film --crf 18.0 --threads 20 --sar 1:1 --frames 88954 --output "D:\test.mkv" "D:\test.avs"

If for some reason that won't work for you, I think you'll find the simplest solution is to run two encodes simultaneously, even if it requires splitting an encode into halves and appending the encoded sections. You don't even have to mess with the worker rules unless you want to. You can run one encoding job by right clicking and running it in a temporary worker, and a second job by starting the job queue normally. Just remember to enable --stitchable if you want to append the encoded video.

I found this in the changelog. The 16 thread limit in the GUI appears to have been implemented a log time ago.
(I assume it refers to the GUI)

0.2.4.1010
Commit by Sharktooth:
- Max number of threads for x264 is now '16'

[jagabo]

Since you're already using AviSynth scripts why don't you get rid of the middleman and use the x264 command line encoder? You can control everything that way. If you use a batch file it's as easy as dragging and dropping the AVS script onto the batch file.