x264 builds 2006-2013 quality compared

[Mephesto]

I'll just copy and paste my journal entry, I've spent enough time writing as it is:

Test video is Ex Girlfriends - We are the Party.mp4 at 1080p, resized to 720x320 and trimmed to the first minute (1440 frames.)

Testing old x264 builds to see how far efficiency has progressed. Every build is January of each year starting 2006, or close to january.

x264 2006, no floating point CRF, would've picked 17.5 if it was possible because the SSIM was under the target goal of 0.99000.
x264 2007, floating point CRF, subme7 and --direct is here.
x264 2008, not much change here.
x264 2009, lots of changes. Subme9, psyRDO (not used) and b-adapt. Had to lower CRF to match SSIM.
x264 2010, shitload of new settings, most notably mb-tree. Others are improved b-pyramid, autovariance AQ, weighted P frames, subme10, and finally x264 actually tells me what I did wrong if there was an error in my script. Before it only gives misleading feedback like "you have to input resolution for RAWYUV video" if I forgot to type two hyphens for one of the encoding parameters.
x264 2011, despite no new settings appearing this time, the increase in efficiency was just as dramatic as when mb-tree was introduced. Whatever mainline optimizations were done were awesome.
x264 2012, subme is upgraded to 11.
x264 2013, nothing new. Bitrate slightly increased, quality very slightly decreased. Speed increased. Nothing but tweaky speed optimizations. Good sign that H.264's days are numbered. H.265 is barely out.

MPG1: 33,105KB (4410 kbps) 24.00 fps 0.98980 SSIM
XVID: 25,262KB (3365 kbps) 30.00 fps 0.99000 SSIM
2006: 15,794KB (2104 kbps) 10.16 fps 0.98957 SSIM
2007: 15,718KB (2094 kbps) 19.83 fps 0.98999 SSIM
2008: 15,661KB (2086 kbps) 18.69 fps 0.99001 SSIM
2009: 14,872KB (1981 kbps) 12.71 fps 0.99003 SSIM
2010: 14,115KB (1880 kbps) 12.27 fps 0.98998 SSIM
2011: 13,460KB (1793 kbps) 11.06 fps 0.98996 SSIM
2012: 13,208KB (1759 kbps) 10.24 fps 0.98994 SSIM
2013: 13,214KB (1760 kbps) 11.42 fps 0.98993 SSIM

Bitrate decrease
2007 - 0.5%
2008 - 0.4%
2009 - 5.3%
2010 - 5.4%
2011 - 4.9%
2012 - 1.9%
2013 - 0%

Now for a different test video, a snowy level of an old platformer game. Command lines the same except different CRFs and 600 keyint.
MPG1: 28,819KB (5937 kbps) 333 fps 0.99507 SSIM
XVID: 9,708KB (2000 kbps) 150.0 fps 0.99507 SSIM
2006: 3,050KB (628 kbps) 47.63 fps 0.99553 SSIM
2007: 2,621KB (540 kbps) 141.41 fps 0.99510 SSIM
2008: 2,611KB (538 kbps) 142.40 fps 0.99511 SSIM
2009: 2,341KB (482 kbps) 86.09 fps 0.99506 SSIM
2010: 1,791KB (369 kbps) 95.51 fps 0.99524 SSIM
2011: 1,676KB (345 kbps) 96.81 fps 0.99511 SSIM
2012: 1,511KB (311 kbps) 71.17 fps 0.99517 SSIM
2013: 1,523KB (314 kbps) 75.83 fps 0.99506 SSIM

Bitrate decrease
2007 - 16.4%
2008 - 0.4%
2009 - 11.5%
2010 - 30.7%
2011 - 6.9%
2012 - 10.9%
2013 - 0%

Command-line scripts:

x264_2006 --crf 18 --filter -1:-1 --threads 12 --bframes 16 --b-pyramid --ref 16 --qcomp 0.7 --analyse all --me umh --weightb --subme 6 --b-rdo --mixed-refs --bime --8x8dct --trellis 2 --no-fast-pskip --progress --output "exgf2006.mkv" "exgf.avs"

x264_2007 --crf 18 --deblock -1:-1 --threads 12 --bframes 16 --b-pyramid --ref 16 --qcomp 0.7 --analyse all --direct auto --me umh --weightb --subme 7 --b-rdo --mixed-refs --bime --8x8dct --trellis 2 --no-fast-pskip --progress --output "exgf2007.mkv" "exgf.avs"

x264_2008 --crf 18 --deblock -1:-1 --threads 12 --bframes 16 --b-pyramid --ref 16 --qcomp 0.7 --analyse all --direct auto --me umh --weightb --subme 7 --b-rdo --mixed-refs --bime --8x8dct --trellis 2 --no-fast-pskip --progress --output "exgf2008.mkv" "exgf.avs"

x264_2009 --crf 16.7 --deblock -1:-1 --threads 12 --bframes 16 --b-adapt 2 --b-pyramid --ref 16 --qcomp 0.7 --analyse all --direct auto --me umh --weightb --subme 9 --psy-rd 0.0:0.0 --mixed-refs --8x8dct --trellis 2 --no-fast-pskip --progress -o "exgf2009.mkv" "exgf.avs"

x264_2010 --crf 15.9 --deblock -1:-1 --threads 12 --bframes 16 --b-adapt 2 --b-pyramid normal --ref 16 --rc-lookahead 250 --aq-mode 2  --qcomp 0.7 --analyse all --direct auto --me umh --weightb --weightp 2 --subme 10 --no-psy --mixed-refs --8x8dct --trellis 2 --no-fast-pskip -o "exgf2010.mkv" "exgf.avs"

x264_2011 --crf 16.9 --deblock -1:-1 --threads 12 --bframes 16 --b-adapt 2 --b-pyramid normal --ref 16 --rc-lookahead 250 --aq-mode 2  --qcomp 0.7 --analyse all --direct auto --me umh --weightb --weightp 2 --subme 10 --no-psy --mixed-refs --8x8dct --trellis 2 --no-fast-pskip -o "exgf2011.mkv" "exgf.avs"

x264_2012 --crf 17.1 --deblock -1:-1 --threads 12 --bframes 16 --b-adapt 2 --b-pyramid normal --ref 16 --rc-lookahead 250 --aq-mode 2  --qcomp 0.7 --analyse all --direct auto --me umh --subme 11 --no-psy --mixed-refs --8x8dct --trellis 2 --no-fast-pskip --qpmin 10 --qpmax 51 -o "exgf2012.mkv" "exgf.avs"

x264_2013 --crf 17.1 --deblock -1:-1 --threads 12 --bframes 16 --b-adapt 2 --b-pyramid normal --ref 16 --rc-lookahead 250 --aq-mode 2  --qcomp 0.7 --analyse all --direct auto --me umh --subme 11 --no-psy --mixed-refs --8x8dct --trellis 2 --no-fast-pskip --qpmin 10 --qpmax 51 -o "exgf2013.mkv" "exgf.avs"

TL;DR: for photographic videos with lots of action, efficiency has increased by 20% in 2013 compared to the 2006 x264 codec, and 100% for linear, flash-like graphics.

I wanted to test an animated sample as well but I didn't have any cartoon/anime handy.
XVID and MPEG-1 added there just for fun. I didn't test MPEG-2 because it isn't much better than its predecessor and my trial for TMPENC expired.

[jagabo]

Interesting look at x264 over the years.

[deadrats]

did your tests happen to isolate the exact moment when x264 managed to violate the laws of physics by sucking and blowing at the same time?

[Mephesto]

did your tests happen to isolate the exact moment when x264 managed to violate the laws of physics by sucking and blowing at the same time?

Look man, I know you wanna delude yourself about x264 because you hate that Dark_Shikari guy. I don't like him either, but evidence is evidence. x264 is unmatched in quality and Mainconcept can't compete in any circumstance. I recall you even saying x264 is only superior because of psy-rdo which I didn't even use in this test.

I've known people way smarter and resourceful than Dark_Shikari who were twice as deplorable. I still use their material if it's provably superior to alternatives.

[deadrats]

actually what i have said is that it's not the job of an encoder to psychovisually enhance the output, the job of the encoder is to reproduce the source file as close as possible within a given bit rate.

to say x264 is unmatched in quality is a lie and it's very easy to prove: if i say youtube video everyone here will say that it sucks, that it's of poor quality relatively speaking, well i hate to rain on anyone's parade but youtube uses x264 as their encoder as do many streaming and VOD services and invariable the video quality of all of them is substandard.

x264 enjoys it's place in video enthusiasts arsenal of tools because it's legally free, if they charge a per seat licensing fee of $1000 no one would use it, likewise if one of the commercial competitors that costs thousands of dollars, like ateme's encoder, cce or elemental's were either legally free or easily found on a torrent site and pirated then no one would even consider using x264.

i'm just glad that hvec is almost here and that suck ass software and it's lame-o developers can just go crawl back under the rock from which they emerged nearly a decade ago.

[jagabo]

to say x264 is unmatched in quality is a lie and it's very easy to prove: if i say youtube video everyone here will say that it sucks, that it's of poor quality relatively speaking, well i hate to rain on anyone's parade but youtube uses x264 as their encoder as do many streaming and VOD services and invariable the video quality of all of them is substandard.

That doesn't mean anything. All h.264 encoders will suck at the bitrates Youtube uses. If/when Youtube decides to use an HVEC encoder they will use even lower bitrates and the video will suck just as badly. Their intention isn't to provide high quality video. It's to provide barely watchable video as cheaply and with as little bitrate as possible.

[_Al_]

So are you saving for some good HVEC encoder now? You are not expecting somebody to develop a free HVEC encoder for you I hope.

I love what is happening now, free utility developed by somebody and resulting GUI fronts , an orchestra, where everything plays well, for free.
Avisynth, x264, VirtualDub, AC3to, ffmpegsource, dmfs, mp4box, muxers-tsmuxer, mkvmerge, etc., and all GUI fronts for those, humanity at its best.

[Mephesto]

actually what i have said is that it's not the job of an encoder to psychovisually enhance the output, the job of the encoder is to reproduce the source file as close as possible within a given bit rate.

Agreed for the most part, but enhancement algorithms can be used for compression too. Debanding for example. It undeniably improves the quality of the output by reversing the banding created by the DCT compression.

to say x264 is unmatched in quality is a lie and it's very easy to prove: if i say youtube video everyone here will say that it sucks, that it's of poor quality relatively speaking, well i hate to rain on anyone's parade but youtube uses x264 as their encoder as do many streaming and VOD services and invariable the video quality of all of them is substandard.

Tests have been done, by MSU, doom9 and others including myself. Mainconcept always outputs worse quality at the same bitrate on every kind of video in every kind of circumstance.

Btw, I wouldn't use YouTube as a credible example for anything. It is operated by morons and it took them until 2009 to start using H.264 despite the standard being out for 5 years. Not to mention crappy mono copyrighted MP3 when they could've used the free, superior quality Vorbis.

They have a billion videos and operate at a loss of half a billion a year to maintain that piece of shit website so what do you expect but very low bitrates and substandard quality? If they used Mainconcept with the same bitrates, the videos would look worse.

x264 enjoys it's place in video enthusiasts arsenal of tools because it's legally free, if they charge a per seat licensing fee of $1000 no one would use it, likewise if one of the commercial competitors that costs thousands of dollars, like ateme's encoder, cce or elemental's were either legally free or easily found on a torrent site and pirated then no one would even consider using x264.

So if MP3 was legally free it would be the most popular audio compression format despite its inferior quality? Oh wait...

[deadrats]

So are you saving for some good HVEC encoder now? You are not expecting somebody to develop a free HVEC encoder for you I hope..

rovi/divx is doing just that, they will have a free hvec decoder available within 3 weeks and a free hvec encoder shortly thereafter.

rovi, which now owns divx which had bought main concept, also licenses their sdk to various ISV's such as sony, pegasus, cyberlink and so on, and their hvec beta sdk has been available under NDA for about 6 months already, so i expect to see tons of popular software from numerous vendors that be able to encode hvec.

last but not least the developer of media coder has already said he is looking at both the reference hvec encoder plus a few other hvec encoders to include this functionality into his app as soon as possible.

here's the funny thing, avc was released at least a decade ago yet mpeg-4 asp still hasn't been displaced, there are still people that stick with divx and/or xvid instead of using x264. i don't think the same will be true once hvec comes out, i seriously think that within a year or two of hvec encoders being publicly released you will be hard pressed to find anyone that will use x264, unless they specifically need avc for blu-ray production.

x264 can't die soon enough, though i do think we'll be in for about a year or so of some really creative FUD from the x264 developers and the x264 faithful as they take to every forum trying to convince users that x264 is superior to hvec and that all the tests that show otherwise are somehow flawed or purposely skewed.

[Mephesto]

x264 can't die soon enough, though i do think we'll be in for about a year or so of some really creative FUD from the x264 developers and the x264 faithful as they take to every forum trying to convince users that x264 is superior to hvec and that all the tests that show otherwise are somehow flawed or purposely skewed.

What irony.
You spread FUD about x264 because you hate the developer, say every codec test of x264 is flawed and use youtube's low bitrates as proof of your assertion. Then you type this shit out.

[deadrats]

What irony.
You spread FUD about x264 because you hate the developer, say every codec test of x264 is flawed and use youtube's low bitrates as proof of your assertion. Then you type this shit out.

i don't hate Jason, i hate what he and his loyal band of FUD'sters have spent the past 10 years doing. there have been a number of tests that had x264 losing to other codecs, including main concept's, and one of those tests was done by MSU. how did these clowns respond? DS and his followers spent post after post ripping the tests for being "biased" and DS went on his "diary of an x264 developer" blog to bitch about "unfair" and "incorrect" testing methodologies.

but the thing that really galls me most of all is that Jason has single-handedly shit all over gpu assisted video encoders. say the words "GPU powered encoders" and it immediately conjures up associations with "poor quality" and the explanations for this are straight from DS, some of the most retarded bullshit you have ever heard, like:

"video encoding is fundamentally a linear process, not well suited for heavily threaded workloads that gpu's are designed for"

"gpu's are designed for very simple calculations, they can't handle the complex algorithms that video encoding requires"

"the latency involved from moving the data from main memory to gpu memory and back negate any speed up that you get from gpu acceleration"

the fact of the matter is that Jason did try to port x264, at least portions of it, to CUDA as far back as 2008 (i followed his posts on the NVIDIA developer's forum) but didn't know what he was doing. he had trouble getting the simplest of algorithms to run on the gpu and when help was given he still didn't understand how to optimize it so that it ran fast, then he took to the doom9 forums to rant about gpu SIMD's.

to this day he and Lord Mulder and a few others will continue to spout the same crap that was disproved years ago by all the good gpu developers that ported SETI, BLAST, chemical and physics simulation software, chess engines, all sorts of software to gpu's. even the people behind badaboom, elemental, showed what a putz Jason is by developing an enterprise grade gpu powered vc-1, h264 and mpeg-2 encoder that is faster than x264 running on dual hexa-core xeon's and offering better quality, as testified to by one of the biggest x264 proponents over on the doom9 forums, it's one of the mods, i don't remember if it's Blue Misfit or Ben Wagoneer.

having said that, i didn't mean to shit all over your test, it was a good test, my comment was meant mostly in jest, tongue-in-cheek, with a healthy helping of truth thrown in.

[Mephesto]

Many, many operations simply can't be parallelized. Entropy coders for example. Multithreading x264 on the CPU even degrades quality marginally for the benefit of much faster speed, so I don't have a lot of faith in GPU processing either. Multithreading is just a temporary solution for the complex problem of transistor leakage that's halting progress in computing power. It's just a trend that's gonna pass when we figure out a way to make the transistor obsolete and move onto quantum or clockless CPUs.

This is besides the point. So Jace refuses opportunities to make x264 encode 10 times faster, the codec remains the best quality of all. Whatever test MSU did that put Mainconcept ahead of x264 was probably a really old one. The advancement since 2010 with mb-tree and all the mainline optimizations left it unable to compete in the latest test.
If you have credible proof of Mainconcept outclassing x264, i'd like to see it.

having said that, i didn't mean to shit all over your test, it was a good test, my comment was meant mostly in jest, tongue-in-cheek, with a healthy helping of truth thrown in.

You haven't really hit a nerve with me, I'm just responding to the untruths you derive from emotion than evidence. Dark_Shikari IS a moron who said way stupider shit before. Here are two that I remember off the top of my head:

"JPEG2000 is so badly developed and mismanaged that it failed as a quality standard"

"H264 in AVI produces corrupted frames and bitstream, I know because I encoded a clip (using shitty broken Avidemux but he doesn't reveal this until much later) and my explorer crashed because it tried to render a corrupt thumbnail"

What does it prove? That people with MENSA memberships are imperfect and just as capable of being illiterate as any more common dullard?

[_Al_]

Deadrats
It is all irrelevant, x264 is part of that well oiled free cog machine that is out there together with other free stuff, it just works all together. It was picked up by everybody, lying there in good condition for free and they built something with it. If anybody needs help in this forum with encoding first answer is usually Handbrake, not MainConcept etc... You take x264 encoder and build encoder yourself in a form of BAT for example just that what works for you.

[deadrats]

Many, many operations simply can't be parallelized.

this is simply not true from a video encoding standpoint and is one of the pieces of FUD that originates from His Darkness; with closed GOP video streams frames don't make references to other frames outside the GOP boundary, you can simply launch encode each GOP in it's own thread and there is no reason why video quality would be affected, in fact i believe this is how media coders SVE (segmented video encoding) feature works.

This is besides the point. So Jace refuses opportunities to make x264 encode 10 times faster, the codec remains the best quality of all. Whatever test MSU did that put Mainconcept ahead of x264 was probably a really old one. The advancement since 2010 with mb-tree and all the mainline optimizations left it unable to compete in the latest test.
If you have credible proof of Mainconcept outclassing x264, i'd like to see it.

first of all, mb-tree came about, as i mentioned before, because Jason discovered that main concept was producing much higher quality I frames than x264, so DS went to work trying to figure out why this was and how he could level the playing field. second, mb-tree is a mess and a half, Jason himself has admitted that there are times when it kills image quality and even worse it creates a massive stats file several gigs in size on your hard drive.

now i will admit, using 10bit x264 does improve things a bit; with regards to "credible tests" i have yet to see any credible video encoding comparison if you go by what Jason says; in numerous posts he has said that both SSIM and PSNR are poor indicates of actual perceived image quality yet in every test that x264 has won the metric used to determine quality was either PSNR or SSIM. thus by Jason's own admission x264 has never won a single test by any metric that has any validity.

What does it prove? That people with MENSA memberships are imperfect and just as capable of being illiterate as any more common dullard?

i doubt Jason is anywhere near MENSA material, but he is a total douche and i have told him as much.

i think the thing that really ticks me off the most about him is what he did back when they first formed x264 LLC. supposedly there were numerous companies that wanted to use x264 but where apprehensive because of the GPL, so the developers form x264 LLC, basically releasing an identical fork of x264 under a commercially friendly software license.

the first ISV to license x264 was Pegasys, the tmpg people. Pegasys put on a big production about the deal, press releases and conferences, put Jasons picture up on their website shaking hands with Hiroyuki Hori, all smiles, basically legitimizing x264 as a commercial product. what does Jason do?

less than 2 weeks after the deal was signed and tmpg had released their video mastering works 5 a user of vmw5 went to the doom9 forum asking for help in configuring the integrated x264 encoder. DS actually posts telling the guy that he shouldn't be using such a crappy product as vmw5, that he should be using an encoder that isn't shitty.

i took his ass to task, asking him why he would shit on a company that was putting money in his pocket and legitimizing his software, the first to do so mind you, and if he felt that it was shitty software why did he sign the deal in the first place.

that *******, trying to either build or maintain street cred, at least in his juvenile minds, tells me that he has to call it as he sees it and that he didn't know it was going to be that shitty when he signed the deal. of course i asked him why he doesn't just try to get what he sees as messed up behind the scenes rather than publicly rip on Pegasys and if he feels that strongly about it then why doesn't he return the money he got for the deal.

needless to say he doesn't answer me, because he knows he's a dick, that's why **** him and the horse he rode in on.

[Mephesto]

Many, many operations simply can't be parallelized.

this is simply not true from a video encoding standpoint and is one of the pieces of FUD that originates from His Darkness; with closed GOP video streams frames don't make references to other frames outside the GOP boundary, you can simply launch encode each GOP in it's own thread and there is no reason why video quality would be affected, in fact i believe this is how media coders SVE (segmented video encoding) feature works.

That's the idea but there's more to it than that. I'm not qualified to discuss this in detail but parallelization gets less and less possible at lower resolutions, at 256x224 only 4 out of 8 threads being utilized. I'm curious why this would be if the GOP principle stands. Other things like entropy coding is not possible to multithread at all.

We talked about mb-tree before. Inspiration isn't plagiarism and it doesn't create huge statfiles if you use CRF. Only annoying thing is the memory requirement for 720p and above which is easily fixed by keeping the lookahead at default.

now i will admit, using 10bit x264 does improve things a bit; with regards to "credible tests" i have yet to see any credible video encoding comparison if you go by what Jason says; in numerous posts he has said that both SSIM and PSNR are poor indicates of actual perceived image quality yet in every test that x264 has won the metric used to determine quality was either PSNR or SSIM. thus by Jason's own admission x264 has never won a single test by any metric that has any validity.

SSIM is not reliable for precise quality control but more than qualified to quantify more obvious differences. A score of 0.98400 and 0.99000 between two outputs almost always correlates exactly with perceived quality, but smaller differences are to be skeptical. On the other hand, sui generis cases exist when you have videos with lots of high-frequency details that mean a lot more to SSIM than our eyes.

In the tests MSU has done, Mainconcept was visibly weaker quality and needed 20% more bitrate to match x264.

i think the thing that really ticks me off the most about him is what he did back when they first formed x264 LLC. supposedly there were numerous companies that wanted to use x264 but where apprehensive because of the GPL, so the developers form x264 LLC, basically releasing an identical fork of x264 under a commercially friendly software license.

the first ISV to license x264 was Pegasys, the tmpg people. Pegasys put on a big production about the deal, press releases and conferences, put Jasons picture up on their website shaking hands with Hiroyuki Hori, all smiles, basically legitimizing x264 as a commercial product. what does Jason do?

less than 2 weeks after the deal was signed and tmpg had released their video mastering works 5 a user of vmw5 went to the doom9 forum asking for help in configuring the integrated x264 encoder. DS actually posts telling the guy that he shouldn't be using such a crappy product as vmw5, that he should be using an encoder that isn't shitty.

i took his ass to task, asking him why he would shit on a company that was putting money in his pocket and legitimizing his software, the first to do so mind you, and if he felt that it was shitty software why did he sign the deal in the first place.

that *******, trying to either build or maintain street cred, at least in his juvenile minds, tells me that he has to call it as he sees it and that he didn't know it was going to be that shitty when he signed the deal. of course i asked him why he doesn't just try to get what he sees as messed up behind the scenes rather than publicly rip on Pegasys and if he feels that strongly about it then why doesn't he return the money he got for the deal.

needless to say he doesn't answer me, because he knows he's a dick, that's why **** him and the horse he rode in on.

Haha, I do believe Jace would pull off something like that.
Here's another tidbit from him I found in my logs:
"I'm a contractor"
...
"I contract gonorrhea"

[jagabo]

The problem with splitting up the encoding by GOPs is that you lose locality of reference. Once you start cache thrashing performance falls through the floor.

[deadrats]

That's the idea but there's more to it than that. I'm not qualified to discuss this in detail but parallelization gets less and less possible at lower resolutions, at 256x224 only 4 out of 8 threads being utilized. I'm curious why this would be if the GOP principle stands. Other things like entropy coding is not possible to multithread at all.

2 things:

1) entropy coding works on a per frame basis in general, it's a lossless compression technique that depending on the method used can be very difficult to parallelize and vectorize at the frame level but some algorithms like CALVC and Huffman coding are easier than CABAC. CALVC works on 4x4 or 2x2 blocks, it's relatively easy to assign each block coding to it's own thread, it's just not efficient from a programming point of view. there is no reason why you can't launch multiple threads per GOP and assign 1 thread to only doing entropy coding per GOP; in fact the CUDA encoder is capable of doing CABAC and CALVC on the gpu with multiple threads. when programmers say that something is difficult to multithread they mean within the context of a given task, but that doesn't mean you can't launch multiple different non-dependent tasks simultaneously.

even if video encoding was a 100% completely serial task, there would be no reason why an encoder couldn't cut a video stream at the GOP level, only at I frames (just like video editing apps do), encode each segment on it's own thread and then concatenate the results; in fact this would be the best way to multithread an encoder as it would clean up the code quite a bit and it would scale nearly linearly with cores, about the only problem would be with open GOP's, thinking about it i wonder what would happen if a frame within one thread made reference to a frame within another thread, i suppose you could handle it in code so that it didn't crash the app but there would definitely be a performance impact.

2) at ridiculously small resolutions like 256x224 the workload is finished so fast that there isn't enough time to launch large numbers of threads, modern cpu's are so fast they fly through the work needed to encode such a small resolution, by the time the program can issue the commands to spawn a new thread, issue a lock, assign ram, do the work, release the ram, release the lock and destroy the thread the processor has already finished with the previous thread and has been sitting idle for a number of cycles.

[deadrats]

The problem with splitting up the encoding by GOPs is that you lose locality of reference. Once you start cache thrashing performance falls through the floor.

i'm confused, how does the principle of locality come into play when multithreading at closed GOP boundaries?

this principle only comes into play when you have to access the same value or storage location frequently, with closed GOP's frames don't reference other frames outside the GOP boundary, thus they aren't accessing the same values nor the exact same storage location, i don't see how the cache would be thrashed.

i suppose with slow hard drives you would get disk thrashing, but with enough ram and proper memory management, buffering the file to ram before disk writes, i think that could be minimized effectively.

[DarrellS]

I read a post by Avery Lee where he was excited about the possibilities of GPU encoding when it first came out but after a lot of testing, realized that the all the time that it took the GPU to keep referencing the memory cache defeated any increase in speed that it could have offered. Some of his decoders and filters are GPU optimized but his hopes of using it in an encoder in Virtualdub went out the window. I believe most video software developers feel the same way. It may be advantageous in games, graphics or some other software but very disappointing in the video encoding business.

Every thread you read on every video forum says the same thing about GPU encoding. x264 is just as fast if not faster on a decent CPU with the superfast preset and the quality of the superfast preset is better than the Cuda encoder. If Nvidia added more options to the encoder then maybe the quality could go up and the file size could go down but there are only so many options as it is now.

I was excited to finally be able to use the Cuda encoder with Virtualdub's external encoder feature but my tests produced the same results as everyone else. Not only did my Q6600 (an old CPU) encode x264 just as fast and produce better quality than Cuda, the Cuda encoded files were much larger than the original MPEG2 which were much larger than the x264 encoded file.

I don't see anyone dethroning x264 for a while. I tried experimenting with x265 but after downloading the special encoder, decoder and special player (since no other player would play the format) I gave up on it. Maybe in a few years if the industry accepts it over H264 and it stays free the video encoding crowd will embrace it.

I'm always looking for more command line encoders that can be used in Virtualdub just to see what is possible but when it comes down to it, the only encoders I need are x264, ffmpeg (to passthrough the original AC3 audio) and MKVmerge.

[hello_hello]

Mephesto,
Did you by any chance make note of differences in encoding speed over the years? I'm just curious as I recall trying x264 back in the early days and thinking it was just too slow so I went back to Xvid, then a few years later I tried again (I'm pretty sure using the same PC) and I've been using it ever since.

I'll confess I find the negative comments regarding x264 somewhat puzzling and deadrats arguments don't seem logical to me. I don't see how a claim some people still use Xvid in preference to x264 offers any proof directly relating to x264's quality..... if it comes to that I know people who still burn DVDs in preference to using Xvid. I think you'd need to be living in a cave not to have noticed AVC has displaced ASP, even if it only managed to do so in the last year or two, and in my opinion the length of time it took to do so is probably more hardware than quality related.

I know nothing about HVEC but it wouldn't surprise me if it takes just as long to replace AVC as AVC did to replace ASP, if not longer. Quality aside, it might be several years before the average PC has a fast enough CPU to make HVEC encoding practical for most people, and I'd wonder how long it'll be before devices capable of playing HVEC become the norm. It's almost hard to buy a device today which can't decode AVC but even that's a fairly recent thing. The smartphone I bought a year ago will happily decode 1080p, Level 4.1, my TV and Bluray player can both do it, yet it wasn't all that long ago I was thrilled the smartphone I had at the time could manage to decode the standard definition AVIs I was encoding for viewing with my DVD player.

It's been a long time since I've read an encoder comparison anywhere which hasn't rated x264 as the best encoder, in fact I recall reading a couple of comparisons where the x264 encoder was effectively the benchmark by which the rest were judged. deadrats doesn't seem to have offered any real evidence to show everybody else is getting it wrong.

[Mephesto]

Mephesto,
Did you by any chance make note of differences in encoding speed over the years? I'm just curious as I recall trying x264 back in the early days and thinking it was just too slow so I went back to Xvid, then a few years later I tried again (I'm pretty sure using the same PC) and I've been using it ever since.

Yes, look at where it says "FPS". Note that I'm using maximum settings except for TESA and ME-range which I leave at default. If I used default settings, quality would be a little bit lower for the music video with much higher encoding speed, so Xvid does not even compete in that department any longer.

deadrats wasn't comparing x264 to Xvid but to MainConcept's H264 codec which is second best.

It's really stupid waiting until the end to move on. Back in january 2007 when I was a complete newb to video and used Vdub for the very first time, I started using x264 and only that codec when I noticed the better quality. Never touched Xvid since, and this is before I had a god damn clue what a video codec was.

When I started DVD ripping, I always used x264 and everybody praised the high quality at such tiny bitrates asking me how I did that. This is when I noticed that 95% of people use that outdated Divx/Xvid shit. I spread the word and insisted for everyone to move on H264. I was met with nothing but whining and change-resistance. Everybody kept telling me that not everyone has a computer capable of playing back AVC, that Xvid looks so much "warmer" and prettier, that it's hard to find codecs to play back AVC etc. All hilarious considering I knew FUCKALL about video yet I had the codecs, had a SLOW Pentium 4 that could play back 576p perfectly and had hundreds of fans praising the quality of the "weird codec" I use.

In 2010 when only a few people used VFR for animated videos, I switched to it immediately.
In 2011 when 10-bit x264 came out and there was just barely ffdshow support for decoding 10-bit, I switched to it immediately.

And when H265 is out, I'll be switching to it immediately. Everybody should. There's no good reason not to.

[hello_hello]

Yes, look at where it says "FPS". Note that I'm using maximum settings except for TESA and ME-range which I leave at default. If I used default settings, quality would be a little bit lower for the music video with much higher encoding speed, so Xvid does not even compete in that department any longer.

Ahh.... I must have missed the fps info. Cheers.

deadrats wasn't comparing x264 to Xvid but to MainConcept's H264 codec which is second best.

Neither did I.
If he mentioned the length of time it took x264 to replace Xvid for a reason other than to imply something negative about x264's quality, I guess I missed it.

It's really stupid waiting until the end to move on. Back in january 2007 when I was a complete newb to video and used Vdub for the very first time, I started using x264 and only that codec when I noticed the better quality. Never touched Xvid since, and this is before I had a god damn clue what a video codec was.

If I was the only one ever watching my encodes I'd probably agree, but there's several TVs in this house and several hardware players etc. I use my PC for playing video but I'm the only one who does.

During my Xvid to x264 transition I encoded most video twice. Initially using Xvid to watch now and x264 for future viewing. Eventually that became x264 for me now and Xvid for everyone else now.....
Unfortunately the novelty of encoding video has passed, and while I still don't mind doing it as a hobby, waiting for indexing to finish gets harder, creating scripts and comparing filtering before encoding no longer seems to be the fun it once was, and extracting/converting subtitles just shits me.
However it happens and whenever it happens, for me the next transition will at least be much faster.

[pandy]

rovi/divx is doing just that, they will have a free hvec decoder available within 3 weeks and a free hvec encoder shortly thereafter.

rovi, which now owns divx which had bought main concept, also licenses their sdk to various ISV's such as sony, pegasus, cyberlink and so on, and their hvec beta sdk has been available under NDA for about 6 months already, so i expect to see tons of popular software from numerous vendors that be able to encode hvec.

Haha, thx dude, you make my day, good joke - rovi (macrovision) and free stuff... well done!

[Mephesto]

Neither did I.
If he mentioned the length of time it took x264 to replace Xvid for a reason other than to imply something negative about x264's quality, I guess I missed it.

He says Mainconcept outclasses x264.

If I was the only one ever watching my encodes I'd probably agree, but there's several TVs in this house and several hardware players etc. I use my PC for playing video but I'm the only one who does.

Screw television and all societal devils who watch it.

During my Xvid to x264 transition I encoded most video twice. Initially using Xvid to watch now and x264 for future viewing. Eventually that became x264 for me now and Xvid for everyone else now.....
Unfortunately the novelty of encoding video has passed, and while I still don't mind doing it as a hobby, waiting for indexing to finish gets harder, creating scripts and comparing filtering before encoding no longer seems to be the fun it once was, and extracting/converting subtitles just shits me.
However it happens and whenever it happens, for me the next transition will at least be much faster.

That'll happen to you if you encode twice all by yourself. I can barely even stand encoding short clips for Xvid vs. x264 tests multiple times.
I work with a team when doing DVD and Blu-ray rips which makes things a lot easier.
Individual short pieces or ambiguous vintage films/episodes/anime shorts are a whole different world unto itself and it didn't get boring yet for me because the workflow is never the same. All require a different approach and filtering to restore it.

[DarrellS]

I just remembered why I lost interest in h265 so fast. Not only did I have to download all the encoders, decoders and special player and create a configuration file and edit it to match a yuv file that I had to create from an avi file that I had to create but then wait forever for the hevc file to encode so that I can mux it into a container to finally watch.

I've been waiting over an hour for my 300 frame 640x360 file to encode and I'm at frame 214. At this rate, it will never overtake x264. No matter how much better it might look. It will probably need the fastest I7 computer (or whatever the top CPU by then) to decode a HD movie and a $10,000 TV to watch it on.

BluRay hasn't caught on with any of my friends. I know two people with a Bluray player. One bought it because it had NAS to stream movies from his computer (he ended up getting a laptop and doesn't use the bluray) and the other uses hers to play HD MKV on DVDs.

It will be another niche market for the rich kids like Betamax and laserdisc.

Almost an hour and a half and I'm up to frame 274. I could've encoded this file in a few seconds with x264 either in Virtualdub or from an ffmpeg script.

[pandy]

Almost an hour and a half and I'm up to frame 274. I could've encoded this file in a few seconds with x264 either in Virtualdub or from an ffmpeg script.

OK, but this very special case for unoptimized C code which is used to research and develop codec not to encode videos - H.265 shouldn't be slower more than 4 times from H.264 in worst case scenario, issue is that there is no HW capable to decode H.265 with reasonable power consumption - this will change for sure with time however H.264 have still many years before it will be fully superseded by H.265. MPEG-2 is still used even if H.264 can offer nowadays similar speed (encoding/decoding) especially with comparable quality/bitrate settings.

Everything need time - H.264 is good and relatively modern codec, H.265 is more like H.264 improvement than completely new idea how video should be encoded.