We publish results of

MSU Subjective Comparison of Modern Video Codecs

Goals of our assessment are subjective (visual) comparison of popular videocodecs, comparison of results with objective metrics and subjective assessment technology testing.
  • 2 bitrates (690 kbps and 1024 kbps)
  • 4 tested codecs:
    * DivX 6.0
    * Xvid 1.1.0
    * x264
    * WMV 9.0
  • 4 test sequences
  • 50 experts
  • Testing method SAMVIQ (see description in ITU-T BT.500)
Main comparison parts:

1. Subjective comparison of videocodecs.
2. Comparison of results with results of objective metrics PSNR, VQM and SSIM.

SELECTED MAIN RESULTS:

Following graph illustrates average MOS for each codec and bitrate and its' 95% confidence intervals (MOS is Mean Opinion Score, the higher the better).


On a graph below you can see average MOS for all codecs:


Main idea:
* Looks like XviD extrely needed for better filtering.
* x264 has the same results on 690kbit as WMV 9.0 on 1024 kbit - really good job!

SETECTED METRICS COMPARISON CHARTS:

We measured a number of objective metrics (PSNR, VQM and SSIM). To evaluate their precision, their values were mapped on the scale of MOS (0-9) using a fitting function.





So it's possible to calculate error in % in measurement of PSNR, VQM and SSIM in comparison with visual results (SSIM has better results, VQM - worst).

See detailes in report (32 pages):
* On report page http://www.compression.ru/video/codec_comparison/subjective_codecs_comparison_en.html
* Or in report PDF (852 kB, 32 pages with all details)
http://www.compression.ru/video/codec_comparison/pdf/msu_subjective_codecs_comparison_en.pdf

Enjoy!