MPEG-4 AVC/H.264 video codecs comparison

[DmitriyK]

Dear video codec experts,

Moscow State University Graphics & Multimedia Laboratory has finished 6-th H.264 codecs comparison.
It is intended for practical researchers and developers in the field of high-end video compression.

http://compression.ru/video/codec_comparison/h264_2010

We have tested newest implementations of MPEG-4 AVC/H.264 video codecs and compare with XviD (MPEG-4 ASP) and Theora encoders.
One of the main targets for this comparison was to test H.264 encoders for transcoding tasks for Movies and HDTV video content.

Codec that were tested:

• DivX H.264
• Elecard H.264
• Intel® MediaSDK AVC/H.264
• MainConcept H.264
• Microsoft Expression Encoder
• Theora
• x264
• XviD (MPEG-4 ASP codec)
• VP8

Summary report topics:

• Objective measurements (SSIM, PSNR, Average Advantage and etc.)
• Encoding speed
• Analysis of averaged objective results
• Leaders in different areas (Movies, HDTV)
• Options analysis for codecs
• Additional subjective analysis for video codes

  • For psycho-visual enhancement in codecs analysis
  • For fade processing analysis
  • For animation movie compression analysis

• Codecs encoding quality progress over years

Enhancements in comparison to Previous H.264/AVC Comparison:

• Subjective comparison
• New codecs
• New sequences
• New type of special analysis for codecs
• Using natural sequences' special modifications
• Using synthetic sequences
• Not only H.264 Codecs (but also XviD, Theora) were tested

Some examples from the comparison report:

This figure depicts RD-curve for bitrate/quality. Higher curve corresponds better encoding quality. This graph shows quality drop for Theora encoder at 1000kbps.

This figure depicts bitrate handling graph – encoders with good bitarte handling methods has horizontal lines close to 1.0 value. This graph shows strange bitrate handling methods for MS Expression encoder. The more information for it could be found by other graphs analysis.

This figure depicts the progress of the x264 encoder over several years. Y-axis shows encoding quality – encoders with its mark higher than other have better quality. X-axis shows encoding time – encoders with its marks placed to left are faster than other. Therefore encoders in upper-left corner are best – faster and have higher quality than competitors.

More dtailed analysis could be foud at next page

http://compression.ru/video/codec_comparison/h264_2010

Best regards,
Dr. Dmitriy Kulikov,
Head of Video Codec Testing team,
Graphics&Media Lab,
Moscow State University

[Bonie81]

@Dr. Dmitriy Kulikov
Head of Video Codec Testing Team

Nice information.
I knew it already from original web-site while searching for the best H264 encoder.

Thanks!

[DmitriyK]

We have tested another popular video codec - VP8 and have included the results into the comparison report.
You can read it here http://www.compression.ru/video/codec_comparison/h264_2010/appendixes.html#Appendix_8

[lordsmurf]

I think this is worth quoting:

Comments from VP8 Developers

We've been following the MSU tests since they began and respect the group's work. One issue we noticed in the test is that most input sequences were previously compressed using other codecs. These sequences have an inherent bias against VP8 in recompression tests. As pointed out by other developers, H.264 and MPEG-like encoders have slight advantages in reproducing some of their own typical artifacts, which helps their objective measurement numbers but not necessarily visual quality. This is reflected by relatively better results for VP8 on the only uncompressed input sequence, "mobile calendar."

Even with this limitation, VP8 delivered respectable results against other encoders, especially considering this is the first time VP8 has been included in the test and VP8 has not been specifically optimized for SSIM as some other codecs have.

To date, WebM developers have focused on the VP8 decoder performance and are only starting to optimize the encoder for speed. The WebM project has only been underway for three weeks, and we believe that our encoder speed will improve significantly in the near future.

I've noticed these sorts of effects, too, in all kinds of still image and motion image tests in the past decade.

Encoder tests are helpful and results useful. After many different types of tests, you can isolate the best 2-3 performing applications. In some cases, you can find the best tool for the best scenario (i.e., converting MPEG-2 high bitrate source, with animation content, to H.264)

But laymen often take the results from these tests to say something unreasonable. -- i.e., "the best H264 encoder." -- as if it were the only tool anybody should use under all conditions. That's foolish and amateur. So I write not to DmitriyK here, but to all the would-be fanboys who want to use these tests as "proof" that their favorite software is the "bestest ever". It's just not so.

As VP8 points out.

[nitro1973]

Very nice comaprison... gonna read all later! thanks!

[Bonie81]

Thanks for the video codecs comparison. very Informative.

Conclusion:

Free Codec
------------
x264 – one of the best codecs by encoding quality; has very user-friendly predefined presets, as well as many adjustable encoding settings.

XviD – an MPEG-4 ASP codec; its quality could be very close to or even higher than that of some commercial H.264 standard implementations, especially for encoding “Movie” sequences, but not for “HDTV” sequences.

Commercial Codec
--------------------
MainConcept – one of the best codecs by encoding quality; has many encoding settings that can be adjusted. This encoder has a very good bitrate handling mechanism.

DivX H.264 – quite balanced encoder with not very big number of parameters, this fact could be comfortable for users. This encoder is designed as a free sample application for DivX Plus HD compliant video encoding, and is a feature-constrained, for-purpose application.

From personal experience x264 + AC3, XViD + AC3 or XViD + Lame is the best combination, as well.
In audio codec AC3 and Lame are the best of the best.

Thanks to the Developers of x264, XViD, AC3 and lame.

[Bonie81]

In audio codec i missed flac, which is also superb for audio CDs.
Thanks to the Developers of flac, too...


Good Developers, Keep up good works, Cheeeers!