x264 vs cuda vs media sdk

[deadrats]

hello everyone, i conducted a 3 way test of x264, cuda h264 and intel media sdk software encoder for the guys over at the doom9 forums and thought you guys may find it interesting as well.

for source i went here:

http://www.demo-world.eu/trailers/hi...n-trailers.php

and downloaded "The Gorgeous Ocean World" demo under the "HDclub" heading. i think this source is a good test because it's a nice 1920x1080p29.97 blu-ray source, it only uses a nominal bit rate of 8.34 mb/s, has lots of good motion, lots of vibrant colors, so it makes a good test for any encoder. it also uses 256 kb/s ac3 audio.

the application used was the japanese version of the soon to be released english version of tmpg express 5, which supports mkv output, uses x264 for software encoding, cuda for gpu accelerated encoding and supports sandy bridge's quick sync encoding, as well as the software version of intel media sdk encoder.

for a target, i chose 1280x720p29.97 mkv, with 128 kb/s audio.

for bit rate i did the following: i calculated a per pixel bit rate for the 1080p source, works out to .134 bits/pixel and derived a nominal bit rate for the target encode of 3.7 mb/s for the video portion, all encodes were done using cbr.

i did test encodes with the default settings for x264 and cuda (remember, i don't speak japanese) and i threw in a special treat. tmpg express 5 allows you to use the media sdk encoder in software mode if you don't have a QS enabled processor installed. i did a test encode with this as well, it was slow as hell but it should be indicative of the quality of the encodes QS will offer.

note: it appears that quick sync is little more than a hardware accelerated version of the IPP encoder, the only commercial app that uses IPP, as far as i know, is gom encoder (in addition to the default x264 encoder) and in numerous tests i found there to be no quality difference between the IPP encoder and x264, they pretty much traded punches, under some tests x264 showed slightly better quality under other tests IPP showed slightly better quality.

the big benefit of IPP was that it ran like stink on a monkey with an intel cpu: using a quad core phenom 2 x264 was easily twice as fast as IPP, using a dual core penryn (E7400) IPP smoked x264. this really shouldn't be surprising considering IPP stands for "intel performance primitive".

the source runs 4:05 long, x264 took 13:35 and cuda took 8:17, intel media sdk software encoder brought up the rear with a super slow 44:39.

you can download the finished encodes here:

cuda
http://www.mediafire.com/?0dy2c1n8mjy6rj1

media sdk
http://www.mediafire.com/?6xp933gxj2xtt9e

x264
http://www.mediafire.com/?rcc9yb3elv4ahag

from a quality standpoint, i would put the x264 encode and the cuda encode as overall equal, in certain parts x264 seemed to hold a minor edge in visual quality, in other parts it appeared to me that the cuda encoder produced the better results, overall i would call it a tie.

the sdk encoder was an interesting case: the over all impression i got was that it produced a slightly lower quality than either x264 or cuda but that it produced more vibrant colors.

my guess is if someone wanted to nitpick, one could find frames from each encode to support the claim that any of the encoders was better than the other two.

the reality is that 3.7 mb/s for 720 is ridiculously low, people usually capture hdtv at 12-18 mb/s and let's be honest with ourselves and point out that 8.3 mb/s is atrocious for 1080p, if we had the original source, straight from the camera (not a previously compressed source), we could get much higher quality encodes with all 3 encoders, even at 3.7 mb/s for 720p.

quite frankly, most people do not have access to uncompressed or losslessly compressed film transfers, what most people have is blu-rays that they "backup" (i.e. steal, pirate, "fair use" copy) or they have hdtv captures or they have footage shot on consumer grade camcorders.

these people won't be transcoding down to 3.7 mb/s for 720p (unless they are complete imbeciles that have a "bit rate starve" fetish), they'll be using saner bit rates, at 5 mb/s for 720p any differences between the 3 encoders start to vanish and at what i consider the minimum for 720p, 8 mb/s, there is no difference at all.

as a test, i tried the cuda encoder at 720p 8 mb/s, just to see what kind of slow down could be expected, and the test encode finished in an almost identical 8 and a half minutes, while x264 slowed down to 14 and a half minutes.

this is definitely not a conclusive test, nor do i expect anyone to accept these results, i'm sure the criticism will fly from the x264 faithful, but it does seem like software based encoding is destined to go the way of the dodo.

[intracube]

I've only had a chance to look at the CUDA and x264 examples. To my eyes the x264 video you linked to looks noticeably better than the CUDA - not a lot, but noticable. The CUDA video has weird stuttering/trailing artifacts on moving objects which almost looks like a reduced frame-rate (highlighted by the red box) and also the stripes on the yellow fish are slightly less defined.



As you say, the bit-rate for both videos is far lower than it should be, and these differences might not be so apparent at higher bit-rates.

You said you used the default settings for both encoders, but these settings are likely to be different - which might explain some of the differences in quality.

Interesting that you comment - "8.3 mb/s is atrocious for 1080p". 6-8Mbit/s is quite typical for the BBC 'HD' channels .
For example, the bit-rate for WALL-E (broadcast over Christmas) averaged out to ~5Mbit/s. I had to double check my figures on that one...
The BBC HD channels have other non bit-rate related 'quality issues' too - but that's for another thread.

[deadrats]

just did 2 more test encodes, for source i replicated the very commonly used x264 hd benchmark found here:

http://www.techarp.com/showarticle.aspx?artno=520

i used the included mpeg-2 source, 30 sec duration, 3950 kb/s, 23.97fps, 720p, 384 kb/s audio and transcoded using both the x264 and the cuda encoder (again japanese version of tmpg express 5), ac3 audio 128 kb/s, mkv, 720p, 23.97, main, 4.1, 3950 kb/s (looked through the testing script, same bit rate, level and profile settings), x264 took 1:20 to finish the encode, cuda took 31 seconds, i for one see no difference between the two encodes:

cuda
http://www.mediafire.com/?evnfks434d28wij

x264
http://www.mediafire.com/?n6gd0gqraadw2bn

[intracube]

just did 2 more test encodes, for source i replicated the very commonly used x264 hd benchmark found here:

http://www.techarp.com/showarticle.aspx?artno=520

i used the included mpeg-2 source, 30 sec duration, 3950 kb/s, 23.97fps, 720p, 384 kb/s audio

Just to correct your typo; the original MPEG-2 video is 15Mbit/s.

x264 took 1:20 to finish the encode, cuda took 31 seconds, i for one see no difference between the two encodes

I can't see any difference between them under normal conditions either. If I import a still from each into the GIMP and heavily process them - there are very slight differences. Insignificant compared to the difference in encoding times, though.

I did notice quite a few glitches in both files, that were at identical places in the video. They're not on the source, so I'm wondering if it's a decoding issue on your computer.

[vhelp]

well, i did see some pixelation in a few fast scenes from the cuba clips, enough to catch my eyes. i also saw slightly less in the x264 verion. either way, these two encoders are difficult to encode with when using their advance settings. i still have a hard time with x264 cli but i manage to get decent encodes using my own aproaches-not yet standardized.

with the presets in these encoders its hard to get solid quality encodes unless you dig a bit deeper inside the params. most my encodes are prob above standards but they work great for my purposes and am satisfied with the quality, plus i can step through each video frame by frame--i wasn't able to do that in any of your test vids.

decoders vary from user to user, so that is another variable factor unfortunately.

i ran out of space on my mem sticks so i could only bring a couple of vids to my desktop video pc to test. i used your 1280x720p clips. those are the ones i saw the pixelations in--the fast scenes.

so, using the same demo clip package you linked to i reencoded the 1280x720p mpeg2 clip down to 720x480p and managed a 7.2mb clip, and i took a look in the same scene but didn't see any pixelation. i didn't have time to step through my test encodes, but feel free to. if i use the standard templates or presets in x264 cli i can't seem to get the same quality--i've given up on them and just use my own test params to get the quality i feel comfortable with. i wish it were that simple, with any h264 encoder but that is not happening at the present. i still consider myself a novice in these things.

heres my test encode for what its worth... i couldn't get the aspect ratio working correclty, the 16:9 param didn't work in this case. but if you play it in kmplayer, just pre the number '6' and that will display correctly. also, my version of vlc doesn't like any of my videos and won't play, perhaps your's will. also, when i demux via tsMuxerGUI to a .264 file, DGAVCIndex has no problems display and playing the video, i use it as my reference gauge tool. . . i think that covers everything for this test vid i made.

-vhelp 5459

[jagabo]

Deadrats, What computer are you running your benchmarks on? You mentioned two in your first post.

By the way, the main reason Intel CPUs run Intel benchmarks so much faster than AMD CPUs is because Intel intentionally disables all MMX and SSE optimizations when running on non-Intel CPUs. Obviously, that doesn't pertain to Quick Sync benchmarks.

[deadrats]

Deadrats, What computer are you running your benchmarks on? You mentioned two in your first post.

By the way, the main reason Intel CPUs run Intel benchmarks so much faster than AMD CPUs is because Intel intentionally disables all MMX and SSE optimizations when running on non-Intel CPUs. Obviously, that doesn't pertain to Quick Sync benchmarks.

the x4 620, 4 gigs ddr2 800, gts 250, 3 1.5tb ultra dense 5400 rpm hdd.

[JNavas]

With respect, your tests were made less relevant by using CBR (constant bitrate) encoding, which can waste bits on low complexity sequences and starve for bits on high complexity sequences.

Good encoders come into their own with VBR (variable bitrate) encoding, which can make possible (with proper tuning of encoding parameters) very good quality output for typical 24 fps video sources at average bitrates of 2200 Kbps for 720p and 4000 Kbps for 1080p.

For example, a sample 720p video encoded with MainConcept H.264 High Profile at 2200 Kbps has bitrates less than 400 Kbps on low-complexity sequences and greater than 10,000 Kbps on high-complexity sequences. By comparison, encoding at (say) 4400 Kbps CBR would waste bits on low complexity sequences and starve for bits on high complexity sequences, resulting in not as good overall quality despite double the file size.

With x264, the recommended approach is CRF (constant ratefactor) encoding, an advanced form of VBR, adjusting the CRF value to suit your own preferences, typically in the range of 18-23 (higher quality the lower the number).

As for how x264 compares to Intel and CUDA at lower bitrates, x264 produces generally produces the best quality for a given bitrate, whereas Intel is far faster, with CUDA not excelling in either category -- see Eighth MPEG-4 AVC/H.264 Video Codecs Comparison

[glenpinn]

With respect, your tests were made less relevant by using CBR

did you realize this thread is 2 years old, the previous poster to yours was in jan 2011 ???

doubt your comments are relevant now days

[JNavas]

did you realize this thread is 2 years old, the previous poster to yours was in jan 2011 ???
doubt your comments are relevant now days

I did, and I respectfully disagree.

[glenpinn]

your comment may have been correct, but i am sorry, you either misunderstood what i said in my post, or i worded it wrong.

it was meant to suggest that it was not really relevant (in my opinion) posting any more comments into a thread that has been dormant for 2 years.

no ill intent was intended, but now your comments will be seen.

cheers

[JNavas]

your comment may have been correct, but im sorry you either misunderstood what i said in my post.
it was meant to suggest that it was not relevant posting that comment after 2 years had past by.
no ill intent was intended, i just thought it was odd someone would actually post back into a thread after it was dormant for 2 years.
now your comments will be seen.
cheers

Thanks for the clarification. While it is true the original conversation has long since concluded, my hope is that the information I provided may be of value to those who now happen on the thread, but (point taken) I should have made that intent clear instead of implying the original conversation was still alive.

[glenpinn]

its all good, now its back out there for all to see once again.

cheers