Testing effects of CPU multi-threading on x264 encoding

[ziggy1971]

I compared the sample clip to the original and there is a K frame (opened directly in VirtualDub2 (x64), no script) at the location where I started the cut, however, for some reason, the older version of mkvmerge 6.1.0 I used originally didn't cut it clean and ended up starting with a B frame instead.

I cut the same clip with a much newer version and the resulting clip does start with a K frame (assuming K is otherwise known as an I frame or at least close to it?)

I'll reupload the original clips and update the others so I don't use up too much of VideoHelp's storage space.

[Selur]

however, for some reason, the older version of mkvmerge 6.1.0

Reason probably being, that mkvmerge 6.1.0 is from 2013-03-02 and there have been tons of fixes being made in mkvtoolnix to better handle H.264 content.
see: https://www.bunkus.org/videotools/mkvtoolnix/doc/ChangeLog
Current version is 30.1.0 from 2019-01-05,....

[ziggy1971]

Has anyone had a chance to compare the new clips?

I changed the source clip to the beginning of the first episode so the cut/split happens at the end of the clip instead of the beginning.

I'd like to know if the results are closer to what I should be looking for. Honestly, to me it still looks like snow... what should I look for?

[Selur]

what should I look for?

This is why quality metrics like PSNR, VMAF, SSIM,... were invented.
If you can't detect a difference with the methodology you use, there are seem to be at least two possible conclusions:
a. the chosen methodology is flawed (in principal or execution)
b. there is nothing to detect

Has anyone had a chance to compare the new clips?

using:

# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
#load source
org = core.ffms2.Source(source="c:/Users/Selur/Desktop/S02-E01 - The North Remembers (Source).mkv", cache=False, format=vs.YUV420P8,alpha=False)
org = core.resize.Point(org, matrix_in_s="unspec",range_s="limited")
org = core.std.AssumeFPS(org, fpsnum=24000, fpsden=1001)
org = core.std.SetFrameProp(clip=org, prop="_ColorRange", intval=1)
org = core.Trim(org, 0)
org = core.sub.Subtitle(org, "ORIGINAL")
# load T1
t1 = core.ffms2.Source("C:/Users/Selur/Desktop/compare/S02-E01 - The North Remembers T1.mkv", cache=False, format=vs.YUV420P8,alpha=False)
t1 = core.resize.Point(t1, matrix_in_s="unspec",range_s="limited")
t1 = core.std.AssumeFPS(t1, fpsnum=24000, fpsden=1001)
t1 = core.std.SetFrameProp(clip=t1, prop="_ColorRange", intval=1)
t1 = core.sub.Subtitle(t1, "T1")
# load T16
t16 = core.ffms2.Source("C:/Users/Selur/Desktop/compare/S02-E01 - The North Remembers T16.mkv", cache=False, format=vs.YUV420P8,alpha=False)
t16 = core.resize.Point(t16, matrix_in_s="unspec",range_s="limited")
t16 = core.std.AssumeFPS(t16, fpsnum=24000, fpsden=1001)
t16 = core.std.SetFrameProp(clip=t16, prop="_ColorRange", intval=1)
t16 = core.sub.Subtitle(t16, "T16")
# Interleave
clip = core.std.Interleave([org, t1, org, t16])

# Output
clip.set_output()

clips are not alinged. -> can't compare.

[ziggy1971]

what should I look for?

This is why quality metrics like PSNR, VMAF, SSIM,... were invented.
If you can't detect a difference with the methodology you use, there are seem to be at least two possible conclusions:
a. the chosen methodology is flawed (in principal or execution)
b. there is nothing to detect

Has anyone had a chance to compare the new clips?

using:

# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
#load source
org = core.ffms2.Source(source="c:/Users/Selur/Desktop/S02-E01 - The North Remembers (Source).mkv", cache=False, format=vs.YUV420P8,alpha=False)
org = core.resize.Point(org, matrix_in_s="unspec",range_s="limited")
org = core.std.AssumeFPS(org, fpsnum=24000, fpsden=1001)
org = core.std.SetFrameProp(clip=org, prop="_ColorRange", intval=1)
org = core.Trim(org, 0)
org = core.sub.Subtitle(org, "ORIGINAL")
# load T1
t1 = core.ffms2.Source("C:/Users/Selur/Desktop/compare/S02-E01 - The North Remembers T1.mkv", cache=False, format=vs.YUV420P8,alpha=False)
t1 = core.resize.Point(t1, matrix_in_s="unspec",range_s="limited")
t1 = core.std.AssumeFPS(t1, fpsnum=24000, fpsden=1001)
t1 = core.std.SetFrameProp(clip=t1, prop="_ColorRange", intval=1)
t1 = core.sub.Subtitle(t1, "T1")
# load T16
t16 = core.ffms2.Source("C:/Users/Selur/Desktop/compare/S02-E01 - The North Remembers T16.mkv", cache=False, format=vs.YUV420P8,alpha=False)
t16 = core.resize.Point(t16, matrix_in_s="unspec",range_s="limited")
t16 = core.std.AssumeFPS(t16, fpsnum=24000, fpsden=1001)
t16 = core.std.SetFrameProp(clip=t16, prop="_ColorRange", intval=1)
t16 = core.sub.Subtitle(t16, "T16")
# Interleave
clip = core.std.Interleave([org, t1, org, t16])

# Output
clip.set_output()

clips are not alinged. -> can't compare.

I hadn't updated the clips with T1 or T16 yet.

Does that script work with AviSynth? I haven't used Vaporsynth.

So, to eliminate confusion as to which clips to compare, I uploaded a different source clip above.

[Selur]

I'll look at it once all the clips have been updated.

Does that script work with AviSynth? I haven't used Vaporsynth.

Nope, thats Vapoursynth.

[ziggy1971]

New clips... and since the clip is part of the beginning of an episode, I hope it eliminates the offset issue (or other issues in previous clips)

Oh, and thanx to all for taking the time to help me with this.

[Aludin]

In one test I've done comparing 1 thread with 12 threads, the 12 thread needed 3-4% bitrate increase to match the quality of the 1 thread or 0.3 extra CRF, but was 3.2 times slower to encode. So the quality never "significantly drops at some point". The loss is always minimal.

[Selur]

Alignment for me is still off.
Same when using Vapoursynth:

# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
#load source
org = core.ffms2.Source(source="c:/Users/Selur/Desktop/compare/S01-E01 - Winter Is Coming (Source).mkv", cache=False, format=vs.YUV420P8,alpha=False)
org = core.resize.Point(org, matrix_in_s="unspec",range_s="limited")
org = core.std.AssumeFPS(org, fpsnum=24000, fpsden=1001)
org = core.std.SetFrameProp(clip=org, prop="_ColorRange", intval=1)
# load T2
t2 = core.ffms2.Source("C:/Users/Selur/Desktop/compare/S01-E01 - Winter Is Coming T02.mkv", cache=False, format=vs.YUV420P8,alpha=False)
t2 = core.resize.Point(t2, matrix_in_s="unspec",range_s="limited")
t2 = core.std.AssumeFPS(t2, fpsnum=24000, fpsden=1001)
t2 = core.std.SetFrameProp(clip=t2, prop="_ColorRange", intval=1)

# load T16
t16 = core.ffms2.Source("C:/Users/Selur/Desktop/compare/S01-E01 - Winter Is Coming T16.mkv", cache=False, format=vs.YUV420P8,alpha=False)
t16 = core.resize.Point(t16, matrix_in_s="unspec",range_s="limited")
t16 = core.std.AssumeFPS(t16, fpsnum=24000, fpsden=1001)
t16 = core.std.SetFrameProp(clip=t16, prop="_ColorRange", intval=1)
# create diff
diff2 = core.std.MakeDiff(org,t2)
diff16 = core.std.MakeDiff(org,t16)
# add subtitle
org = core.sub.Subtitle(org, "ORIGINAL")
org = core.text.FrameNum(org, 9)
t2 = core.sub.Subtitle(t2, "T2")
t2 = core.text.FrameNum(t2, 9)
t16 = core.sub.Subtitle(t16, "T16")
t16 = core.text.FrameNum(t16, 9)
# Interleave
clip1 = core.std.Interleave([org, t2, org, t16])
clip2 = core.std.Interleave([diff2, diff2, diff16, diff16])

# StackVertical
clip = core.std.StackVertical([clip1, clip2])

# Output
clip.set_output()

and when using Avisynth:

LoadCPlugin("I:\Hybrid\32bit\AVISYN~1\ffms2.dll")

org = FFVideoSource("C:\Users\Selur\Desktop\compare\S01-E01 - Winter Is Coming (Source).mkv",cache=false)
t2 = FFVideoSource("C:\Users\Selur\Desktop\compare\S01-E01 - Winter Is Coming T02.mkv",cache=false)
t16 = FFVideoSource("C:\Users\Selur\Desktop\compare\S01-E01 - Winter Is Coming T16.mkv",cache=false)
diff2 = Subtract(org, t2)
diff16 = Subtract(org, t16)

org = org.Subtitle("Source").ShowFrameNumber(false)
t2 = t2.Subtitle("T2").ShowFrameNumber(false)
t16 = t16.Subtitle("T16").ShowFrameNumber(false)

stack = Interleave(org, t2, org, t16)
diff  = Interleave(diff2, diff2, diff16, diff16)

StackVertical(stack, diff)
return last

In one test I've done comparing 1 thread with 12 threads, the 12 thread needed 3-4% bitrate increase to match the quality of the 1 thread or 0.3 extra CRF, but was 3.2 times slower to encode. So the quality never "significantly drops at some point". The loss is always minimal.

personally I would count 3-4% bitrate increase as rather much.

[poisondeathray]

Same here as selur; T2,T16 don't match the source - framecount mismatch . Double checked with different source filters (and threads=1 for ffms2)

If you can "trust" mediainfo , it also reports 2882 vs. 2877

How are you decoding the source and encoding ? (what are your steps?)

[ziggy1971]

OK, my bad. Actually ALL the clips are wrong...

Source didn't get cut where I expected it to be, thus a few frames longer.

Found the following, but it was posted in 2013 so I don't know if things have changed since then.
https://forum.videohelp.com/threads/358762-Cut-MKV-frame-accurate#post2266371


Encoding script I used was a copy of the original, therefore still had the Trim(0, 2876), forgot to comment it out.

Thanx again for helping. It could've take me days/weeks to figure this out on my own.

Removed bad files above, try the new one before I transcode the rest.

In one test I've done comparing 1 thread with 12 threads, the 12 thread needed 3-4% bitrate increase to match the quality of the 1 thread or 0.3 extra CRF, but was 3.2 times slower to encode. So the quality never "significantly drops at some point". The loss is always minimal.

My testing doesn't show such a drastic change. (based on 1 source clip)
1 thread: 787 seconds to encode, 206.193.554 bytes
12 thread: 91 seconds to encode, 206.597.650 bytes

787 / 91 = 12 threads is about ~8.648x faster
1 thread -> 206.193.554 bytes
12 threads -> 206.597.650 bytes - only about 0.195% gain in bitrate
32 threads -> 207.452.929 bytes - only about 0.61% gain in bitrate

Again, based on 1 short clip thus far. Will update when I gather more data.

[poisondeathray]

In one test I've done comparing 1 thread with 12 threads, the 12 thread needed 3-4% bitrate increase to match the quality of the 1 thread or 0.3 extra CRF, but was 3.2 times slower to encode. So the quality never "significantly drops at some point". The loss is always minimal.

My testing doesn't show such a drastic change. (based on 1 source clip)
1 thread: 787 seconds to encode, 206.193.554 bytes
12 thread: 91 seconds to encode, 206.597.650 bytes

787 / 91 = 12 threads is about ~8.648x faster
1 thread -> 206.193.554 bytes
12 threads -> 206.597.650 bytes - only about 0.195% gain in bitrate
32 threads -> 207.452.929 bytes - only about 0.61% gain in bitrate

Again, based on 1 short clip thus far. Will update when I gather more data.

But you haven't measured "quality" . Strictly speaking, CRF isn't really a measure of quality. It's a rate control method

Aludin probably used ssimplus , maybe added in ssim / psnr for fun

[ziggy1971]

In one test I've done comparing 1 thread with 12 threads, the 12 thread needed 3-4% bitrate increase to match the quality of the 1 thread or 0.3 extra CRF, but was 3.2 times slower to encode. So the quality never "significantly drops at some point". The loss is always minimal.

My testing doesn't show such a drastic change. (based on 1 source clip)
1 thread: 787 seconds to encode, 206.193.554 bytes
12 thread: 91 seconds to encode, 206.597.650 bytes

787 / 91 = 12 threads is about ~8.648x faster
1 thread -> 206.193.554 bytes
12 threads -> 206.597.650 bytes - only about 0.195% gain in bitrate
32 threads -> 207.452.929 bytes - only about 0.61% gain in bitrate

Again, based on 1 short clip thus far. Will update when I gather more data.

But you haven't measured "quality" . Strictly speaking, CRF isn't really a measure of quality. It's a rate control method

Aludin probably used ssimplus , maybe added in ssim / psnr for fun

No, Aludin didn't specify.

No, CRF isn't a measure of quality, but, in this thread we're comparing encoded clips using CRF to the original clip. Isn't that considered a measure of quality?

I removed the old clips above and updated it with a new one.

[ziggy1971]

I'm looking at the encodes against the original clip using the various scripts in this thread, but unfortunately I do not know what I should be looking for.

[poisondeathray]

No, Aludin didn't specify.

When someone says " needed x% bitrate increase to match the quality ...". It means they did serial testing, and had a quality measurement endpoint. e.g. if the endpoint was , say 40db , you do encodes at 18.1, 18.3, 18.5 etc.. measure the quality for each. When you have equivalent quality , you look at the bitrate required to reach that "quality" level

No, CRF isn't a measure of quality, but, in this thread we're comparing encoded clips using CRF to the original clip. Isn't that considered a measure of quality?

Not really , but it can be used as a rough estimate

An objective quality metric's value indicates the same thing. Always. Under any condition. So 3cm is 3cm . 1km is 1km. 40db is 40db. etc...

Whereas "CRF 18" can yield wildly different filesizes (e.g. try changing some of the other settings, presets) , and considerable differences in quality. Some scenes can look bad at CRF 18 , while other's ok. CRF18@superfast is usually going to look worse than CRF18@veryslow . And the filesize does not necessarily say anything about quality (you can infer or make educated guesses, but it's not really measuring something) . So CRF is only a rough estimate of "quality" . Technically it's not even a measure. It's just a rate control method

Bitrates need to be similar. Usually +/- 0.5% is acceptable, but some tests might have stricter variances . So if you were using CRF rate control, you would use incremental values, CRF 17.9, 18.1, 18.3 etc... until you got bitrates at the desired bitrate goal . Or if you were using 2pass, you'd entire the target directly. Sometimes with 1pass, you under or over shoot (a lot) , so you might have to entire a different value (e.g. If your goal was 2Mb/s , you might get 1.5Mb/s actual. You might have to enter 2.5Mb/s to "get" 2.0Mb/s . For those reasons, 2pass testing is usually conducted). But 1pass ABR VBV is still used in many streaming, low latency, server scenarios - so it's still valid to test and those results will apply directly to those scenarios

You need to define "quality" , and measure it in some way. It doesn't have to be an objective measure. Subjective is valid too. Each has various pros/cons . Some have low rates of correlation, some have higher weighting towards subjective perception, etc... But the more measures, the more they "say" the same thing, the stronger strength of the analysis. If they all say quality sucks, it's more likely the quality sucks . (or there might be other explanations , like frame misaligment)

I'm looking at the encodes against the original clip using the various scripts in this thread, but unfortunately I do not know what I should be looking for.

As mentioned earlier, at CRF 18, you're not going to be able "see" any major differences between T1, T36. Even on a threads sensitive clip (one with lots of motion, especially vertical motion vectors) , you won't "see" much difference. The bitrate range used will hide any obvious defects. But on a threads sensitive clip, you will see a larger deviation in the filesize at T1 vs. say T36. e.g. if you only got 0.5% size difference here, you might get 5% instead (so you'd have repeat the test to get closer filesizes with fractional CRF's) . In order to determine smaller differences that you cannot see, you typically use metrics. So that was how Aludin determined that he needed" x%" to match quality "y". But metrics can also help guide you as to which frames have the largest deviations . eg. If you see a large deviation in the pattern on frame 2354 (e.g. the dB score drops massively), that's probably where you should look. So per frame metrics are useful in other ways

You won't see differences as large as the GPU encodes in the other posts . Some of those needed >2-3x the filesize to retain textures and details in subjective assessment. x264/x265 perform poorly with dark scenes in general , but it's just that the GPU encoders tested there perform a few magnitudes worse . Aludin is reporting a 3-4% from 1T to 12T on a certain test (which suggests to me it's either it was threads sensitive content, or he was using a lower bitrate range) . Typically, from 1T to 12T across all types of content, the difference is <0.5% .

[ziggy1971]

I should NOT have said anything about Aludin at all; my apologies. I don't care what Aludin intended to say or had done. My response was based on information provided by him/her, nothing more. Don't try to fill in the blanks as to what they may have done. Like really, why try to fill in the blanks for someone else when you know nothing more about it than I do? Pure speculation and assumptions; GUESSING.



I was thinking about changing the title several days ago as it doesn't actually imply my intentions.

For most people, at this time, they don't have to worry about the number of threads the x264 encoder uses as it's not an issue. In the future more people might, I don't know what people intend to purchase or what becomes available.


What I was trying to accomplish was establish at what point it was no longer worth it, either speed or quality wise, to use more threads and I was hoping that more experienced video editors would help me to establish that using the hardware I have. By default, MeGUI was using 34 threads. Yes, I know that's not 1.5x the number of threads available, but that's what MediaInfo shows.

If using 12 threads nets me 99.985% quality and using 30+ threads only achieves 99.986%, really what's the point in putting those extra resources/threads towards getting an extra 0.001%? My question was at what point (number of threads) does it not make any more sense? Is it 12 threads? 16? 18?

As I stated in an earlier post that my old i7-4930K OC'd to 4.6GHz would achieve similar results to a new i7-8700K at stock speed (3.7GHz) and using the same settings, same threads, same everything. Now, assuming all threads being equal, I figured I'd just dial down the number of threads my current system uses and achieve similar results. It doesn't work that way. I've dialed down my system to 3.6GHz and reduced the number of threads in MeGUI to --threads 8 and it's still much quicker than any 6 core/12 thread CPU using the same encoder settings minus the --threads 8 value.

It's the same thing as adjusting the CRF up or down to determine a point at which the speed vs. quality trade-off is acceptable to an individual.

I think the whole point of this thread was lost which is highly unfortunate. What more can I say? My apologies for wasting everyone's time...

What I've learned here is much more about the people than video/audio.

[Aludin]

Well damn, I thought my blunt answer would be the silver bullet to end the endless speculation but if a 3-4% difference is considered huge, then that opens up a new dimension of subjectivity and I am reminded that what you seek IS subjective.
You wanna know at what point you get diminishing returns but if you're willing to wait 3 times longer to get 3% better quality then your preference is very unconventional and I question if you should be using multithreads at all.

For the record, my test was done with 2 different clips and yes ssim+ was used as the control. The 1-thread was encoded with CRF29.13 and the 12-thread with CRF29.41 when the bitrate was used as the control. As you can tell by the high CRFs, it was low bit rate.

I can't answer whether 0.5% is more typical than 3% on higher bitrates but if the former is acceptable and the latter isn't then all you can do right now is perform a series of tests and see the percentages rise as you increase the threads and decide when enough is enough.
And use an objective metric. CRFs aren't a measure of quality.

[poisondeathray]

I should NOT have said anything about Aludin at all; my apologies. I don't care what Aludin intended to say or had done. My response was based on information provided by him/her, nothing more. Don't try to fill in the blanks as to what they may have done. Like really, why try to fill in the blanks for someone else when you know nothing more about it than I do? Pure speculation and assumptions; GUESSING.

Don't assume that I knew nothing.

You didn't have have information that I had. Don't fill in the blanks because you're actually the one guessing, that I was "guessing"

It turns out I was correct on all accounts (ssimplus, low bitrate, crf) . I was 99% reasonably sure because it was an educated reasoning based on previous interactions and information. I know about his testing methodology from before

For most people, at this time, they don't have to worry about the number of threads the x264 encoder uses as it's not an issue.

Yes, exactly . The effect of threads is mostly negligible under most user cases.

For most people it's not worth 3-10x slower settings (it's like choosing "placebo" over "medium") , for a a few per % difference that you can't see at that bitrate range

But, at lower bitrates, those small differences add up. It's easier to see differences. That's why you'd consider using a few more references, a few more b-frames. It becomes more important to squeeze every little bit of efficiency out here and there. Then you have those people that placebo everything threads 1 (and I guess disable SIMD...)

Well damn, I thought my blunt answer would be the silver bullet to end the endless speculation but if a 3-4% difference is considered huge,

I read that what selur meant was that delta is large for T1 vs T12 . It's larger than expected. Now that is a real "guess" fro me , but based on the phrasing

Personally, ~3x slower for ~3% gain is not a speed/quality trade off I'd make, unless I could "see" the difference

[ziggy1971]

My apologies, again. I shouldn't have said anything.



In my defense I'll just say one thing about my attitude towards the people on here.

A very long time ago I asked some questions regarding video encoding/transcoding and the people who answered proceeded to mock the sources I was providing. They weren't my sources to begin with and I was only trying to do some work for someone else who didn't have the resources at that time and I was trying to learn about the processes of working with video at the same time.

Initially, I didn't even realize they were mocking me. After I realized what was going one I developed a very bad attitude towards the people here. Especially towards some specific people who respond frequently (I won't name names). So yes, I do have a lot of reservations about asking questions. This is a VideoHelp Forum, a place to ask for help. I guess I was wrong about that also. It seems more of a place where people with egos hang out simply because they know more about video than most and mocking others makes them feel as though they're better than anyone else. Very childish in my opinion, but hey, what do I know???



I could ask, again, about intermediate codecs, as that's how this thread was started, but it'll only provide more fuel to a burning thread already.

From my testing, Ut Video Codec Suite 20.2.0 and Lagarith encode at around 50-70fps @ 1080p, while MagicYUV encodes well over 100fps @ 1080p. I'm still leery about MagicYUV because until recently, I hadn't heard of it before.

I've read many threads and Googled quite a lot to find more information. Frequently, Ut Video comes up as being the fastest encoder to an intermediate, but I don't get those results. Perhaps the settings I'm using? I don't know.


No need to respond. Who knows what kind of response it would be anyway, so save yourselves the trouble.

NOTE: Mod may close this thread.... Thank you!

[redwudz]

Thread closed at OP's request.

Moderator redwudz