Common audio encoder quality comparison

[Premjit V.P.]

Tested few common audio formats by phase cancellation method to grade the quality of audio compressions. It is known that the psychoacoustics model of compression does not allow to conclude the quality of encode by the quantity of data thrown out rather by what quality of data is thrown out. Theoretically a 128 Kbps mp3 file recoded to 160 kpbs should not lose any data for compression, but practically that does not happen.


So I feel when it comes to judging the quality of sound, eyes are more better than ears for inference! ABX listening tests may not be very reliable.


The method of phase cancellation method is well known yet am mentioning it for ready reference. Everybody can perform it in their own system. In a nutshell I will say the encoded waveform should be mixed with the inverse of the actual waveform to know the difference of data left out. Here are the details


In adobe audition multitrack view (or any other sound editor) when the source and encoded are viewed there is a small sync difference. For proper mixing I introduced manually a crust in waveform as an anchor point in the very beginning of the file.
The waveform are now synched by cutting a small portion towards the left of the encoded waveform and moving the crust to match the position of the original waveform. In the fourth image, the vertical bar shows the position the cut was made in the encoded waveform.


Having inverted the source in single track view (Edit View) by Effects->Invert and return to multitrack view Edit-Mixdown to file (all audio clips) is all one needs to do to find out the difference of data thrown out


As an alternative one can have the copy of INVERTED source file in the first track and original source file in the third track and save the document. The encoded file will be placed in the second track and after synching the audio with the third track source as mentioned above, the actual source can be deleted and the mix made and screenshot taken and the audio document reverted for placing the other encoded file in the second track(if necessary). For perfect encodes like wma, ogg, lame mp3 the original file can be opened and edit->mixpaste with invert of the encoded file as shown in the fifth image.


I did not wish to encode any file below 128 Kbps because even good encoders performed only marginally better compared to 128 bit mp3 files. With AAC HE high efficiency mode I could not match the crust of encoded file and hence did not consider it – probably they don’t compress well.



I feel helix mp3 at 320 is the best codec considering the quality of encode and ubiquitous nature of mp3 .
CBR is always better than VBR

The images that you are viewing is the cropped image of left channel (the right being similar is not considered) stacked one above other for easy inference. The images are almost in alignment with other images and can be visually compared to-and-fro in image viewer. Unable to upload lot of files and hence attaching a zipped version.

Software used



1. Adobe audition
2. EZ Audio converter
3. Mediacoder
4. Surcode Standalone encoder
5. DTS HD Master and
6. Arcsoft DTS decoder

[gregalan]

So... You can hear with your eyes ?

Visuals or graphs of the sound waves says very little about the quality of the sound, only your ears can tell you if it sounds good or not.

I've seen this pseudo-science approach before .

First: the psychoacoustics model in mp3 encoders doesn't just "throw data away" it also masks sounds and that changes the waveform when it's decoded so you can't learn much about the sound quality by subtracting the encoded then decoded waveform from the original unencoded waveform.

Second: "CBR is always better than VBR" Where in hell did you get that from ?
It's well known that the best mp3 encoders produce better or equal quality in vbr mode, just look for "recommended Lame settings".
I guess you would also probably say stereo is always better than jointstereo... LOL

[pandy]

"phase cancellation method" can be used only where LINEAR processing is involved - for non-linear processing (and lossy coding is non-linear processing) this method is invalid - same principle is for using white noise or sine pure tones to describe codec capabilities - they can be used only under some limited constrains and you must know what are you doing i.e. you must be able to predict and calculate how coding nonlinearities will affect linear signal...

btw

I can prove that even for more predictable operation on audio signal, "phase cancellation method" will fail even if your ear/brain combination will be happy to hear such audio (i mean in blind test you will be not able to say what is original source and what is processed one).

[jagabo]

Nevermind. I looked at the content of your ZIP file and see that you compensated for phase delays of different codecs. I still agree with others here that looking at waveforms isn't the same thing as listening to audio samples.

[hogger129]

320k MP3 using the LAME encoder always sounded good to me. 256k AAC is better though because it has better compression and I always felt that 256k AAC recordings from iTunes sounded pretty close to transparent with a CD.

So lossy audio I would use AAC (whichever version iTunes is running). For lossless I use WAV. I don't know why but it seems like my playback equipment handles uncompressed better than lossless. Plus WAV is compatible with everything.

[Premjit V.P.]

Eyes can not only hear, but feel too.....

If I need to know which one amongst the hot iron bars is hottest, I just need a digital thermometer. In that sense
I can also feel with eyes!

If a method detects ANY CHANGED,DELETED DATA in a waveform, why should I not partially rely on it? One should not
fully rely on it - for ex. even if an encoder does a lossless encoding job but slightly lowers/increases the amplitude, the result according to method mentioned above will be disastrous. But an encoder is a good encoder only if it is not tweaking any part of the audio and try to be nearest to the source.

The result of the graph may not tell everything. If the result has lot of thin lines it say MANY THINGS, but if it is doing an almost lossless job, IT TELLS EVERYTHING! Take a look at DTS 5.1 Track audio also. I have also does the quicktime TVBR test which I have not mentioned in the post.... they also look more closer to the source!

CBR vs VBR, I was actually referring to lame 320 Kbps. In ABR what I wished was the same quality (I mean thin lines in the result)
as CBR but same/better quality. The file size was only 1996kb which means 1996/7.5 = 266 kbps average per min. I would have wished the encoder to consume more bitrate and be nearer to the source than CBR 320. But I agree the difference for huge size difference i.e, 320*7.5 = 2400-1996=404 kb - almost half an mb (for a minute of audio) is very low.

Pandy, you are better informed than me....

[jagabo]

Essentially, what you're doing is looking at PSNR. This is used all the time as one objective measurement in signal processing. It's a starting point but not the final word in "quality". Nothing new here.

[El Heggunte]

Just for the notes, I am quite surprised that you DIDN'T test stereo DTS, but only 5.1 DTS

It seems to me you still don't know what the Master Audio Suite Encoder is capable of

[pandy]

Pandy, you are better informed than me....

Oh common - don't be so sarcastic, you can't completely ignoring quantization and how quantization error is distributed - especially that high noiseshaping level can give you very nasty wave, nasty spectrum and so pleasurably audio - most of lossy codec will translate LPCM to non PCM domain (even not spatial but frequency domain), during decoding you need convert back this native domain to LPCM and you can do this in many various flavors... just check dithering, noiseshaping then you realize why reverse phase is not useful and this is only tip of the ice berg as in lossy coding there so many other important factors - reverse phase can be used only with simple tones like sine wave however this not how we hear complex audio.

[Premjit V.P.]

El Heggunte Re: Common audio encoder quality comparison
Just for the notes, I am quite surprised that you DIDN'T test stereo DTS, but only 5.1 DTS

I have already done it but did not specify in the post because there are better codecs for stereo (in terms of reach/spread) which can do the same in comparatively lower bitrate. Not that DTS stereo is not a good one, it is not as widespread as other codecs. I did the DTS test with 255, 318 and 768 also. I have the test results and will upload only if you wish to have it.

[pandy]

Judge by your eyes - which one is better?

[Hoser Rob]

Eyes can not only hear, but feel too.....

If I need to know which one amongst the hot iron bars is hottest, I just need a digital thermometer. In that sense
I can also feel with eyes!....

What pompous nonsense.

Anyone who actually knows what they're doing will use signals that actually yield readable graphs. Unlike this garbage.

BTW all listening comparison tests are invalid unless levels are matched by less than 0.2 dB. Quite doable when comparing encoders but easy to screw up too. If you take the same signal and play it back 0.3 dB louder or softer it'll sound different.

[El Heggunte]

......
I have already done it but did not specify in the post because there are better codecs for stereo (in terms of reach/spread) which can do the same in comparatively lower bitrate. Not that DTS stereo is not a good one, it is not as widespread as other codecs. I did the DTS test with 255, 318 and 768 also. I have the test results and will upload only if you wish to have it.

I am too lazy right now, so I will have to quote myself

......
First, DTS audio for Blu-Ray is not limited between only two bitrates, and now there is a freeware DTS encoder (dcaenc), which BTW is far more *comprehensive*, feature-wise, than the old Surcode compressor. Therefore, today anyone can do ABX tests between AC3 and DTS's "Coherent Acoustics" by using stereo sources, which are readily available and inexpensive.

The only apparent problem is, now "NOBODY" cares about running new comparative tests between DTS and AC3.

By «stereo sources, which are readily available and inexpensive», I mean Audio-CD tracks

Also,

The only apparent problem is, now "NOBODY" cares about running new comparative tests between DTS and AC3.

Probably because the only people who would accept the results of such a test already know what the result is going to be, while "golden-eared" idiots continue to spout that DTS is superior because of volume levels, mixing, numbers, and marketing.

Maybe you're right, but IMNSHO it's not enough to confirm that "DTS sucks" ( or rather, that "AC3 sucks, but DTS sucks more" ), it's more important to determine HOW MUCH DTS sucks when compared to AC3 And to be honest, I mistrust all those ancient comparisons "DTS vs. AC3" --- as a well-known Hydrogenaudian already pointed out, ABXing with 5.1 sources is much more difficult (and less conclusive) than ABXing with mono or stereo sources So it would be nice for example, to confirm (or infirm, who knows) that DTS can be perceptually-transparent @ 255kps per channel @ 48kHz (which roughly translates as 1280kbps for 5.1 @ 48kHz).

...... after I've moved this summer, I think it's time I put this goofiness to rest by posting another A/B/X showdown..

Scott

We're still waiting for that, Scott

( and in a brand-new topic, of course )