Listening Comparisons For Audio Codecs At 64kbps

waaka! writes "Hydrogenaudio has just wrapped up a listening test of various audio codecs at 64kbps. Check out the results, where Ogg Vorbis performed quite well, scoring significantly better than WMA, RealAudio and QuickTime AAC, and kept pace with MP3Pro and HE-AAC (AAC with the SBR extensions that MP3Pro uses). Clearly, though, no codec can honestly claim 128 kbps MP3 quality at 64 kbps. The charts at the end show entries for 128kbps LAME MP3 and 64kbps FhG MP3, but these are used as high and low anchors for reference, as MP3 is really out of its league at bitrates such as these."

Comments

Sometimes you don't need high audio quality

[Anonymous Coward]

Sometimes, simple audio clips don't require the highest quality. Function triumphs over high performance hot-rodding.

CD

[Leffe]

Clearly, though, no codec can honestly claim 128 kbps MP3 quality at 64 kbps.

You used to compare against CD quality.

Oh well, times change, I guess it's time to throw all my CDs away and instead store all music in this new exciting digital format.

And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum.

Wait a minute!

[BWJones]

where Ogg Vorbis performed quite well, scoring significantly better than WMA, RealAudio and QuickTime AAC, and kept pace with MP3Pro and HE-AAC (AAC with the SBR extensions that MP3Pro uses)

But from the article: QuickTime 6.3 AAC LC 64kbps, Best Quality

So, what gives?

Streaming audio

[Brahmastra]

I've also noticed sometimes that 28 kbps streaming audio in some sites is much better than 56 kbps audio on other sites

Re:Streaming audio

[Tyrdium]

Yes, but bitrate isn't the only factor in streaming audio. If the latency from the 28 kbps site is relatively low compared to that of the 56 kbit site, it will sound better because it will actually be able to make use of the bandwidth. There was a site on this, but I lost the url. Sorry!

question

[zymano]

Are we at the LIMITS of compression technology ? Is there anything new being worked on by anyone ?

Re:question

[merlin_jim]

Are we at the LIMITS of compression technology ? Is there anything new being worked on by anyone ?

Well we are certainly near the limit of lossless compression. In that there is a well-studied field of computer science (information theory), which provides a framework to determine the theoretically maximum amount of lossless compression possible given a particular data sample, and the best lossless compression algorithms we can come up with are within a small percentage of that figure. FYI, a fundamental tenant of information science is that everything can be reduced to a certain atomic level of representation, and that this atomic piece is the "information" contained within "data"... and that one cannot convey "information" in less space than this atomic piece.

For instance, I've heard that common every day american english conveys approximately 1.2 bits of information per word... meaning that the least redundant approximation of human speech would need that bit rate to represent it.

As far as lossy compression, there might or might not be more work to be done. The problem is that the human ear and the human auditory nervious response are far from being fully characterized, though we do have a good start on it.

The idea of a lossy compression algorithm is to remove pieces of information that the human ear and/or auditory nervous response is not sensitive too... therefore increasing the theoretically possible maximum compression without adversely affecting the signal representation. As we as a species come to characterize these human responses, we will certainly see better codecs coming out. I do however believe that we're rapidly approaching an asymptotic level of understanding where further levels of effort and research into codecs is not economical with regard to expected payoffs...

Re:question

[NonSequor]

Well, of course bzip2 isn't ideal. There are better ways to measure the entropy of a text.

The way Shannon did it was to take the text and show it to a person one character at a time. Before you show the next character you ask the person to guess what it is and record the number of guesses it takes to get it right. When Shannon performed this experiment, he ignored case and punctuation, however, including them would not impact the entropy by much. Using this technique, Shannon obtained an estimate of 1.3 bits per character for the entropy of Jefferson the Virginian by Dumas Malone.

0 - Re:My music days are over

[cra]

In Norway, any copies of any music downloaded for personal use are legal, so as long as I don't share or sell anything I download, I can download as much MP3 songs and DivX movies I want.

So, as I don't have any P2P software running at the moment, the number is zero. :-D

Re:I'll say this

[Wakkow]

"You may like carrying around a cell phone, PDA, and iPod in your pockets, but I want one device that does it all."

Which won't happen until someone designs a better method of inputting data. Keys only get so small before they're not usable and phones can only get so big before it's bulky.

Okay, maybe I'm nitpicking...

[shirai]

Just some relevant data that doesn't appear on the front page is that the test is blind and they do compare the audio to an uncompressed reference. You have to click through to "Return to Roberto's Listening Tests page" to find this information though. Just thought I'd mention this because my first thought was what they are comparing the sound to.

Comparing without a reference reflects how much you like the encoding of the codec, not how accurate it is to the original. For example, if a codec boosts the bass or encodes slightly louder, you may interpret this as better sound. For example, when auditioning speakers, you must always balance the output of the speakers as most people will psychologically prefer the louder (most sensitive) speaker. This does not mean the speakers are accurate however.

At any rate, here is the relevant quote on that page:

One of the most acclaimed methods of comparing codec quality is by performing so-called "Double Blind Listening Tests". In this sort of test, the participant compares various encoded samples against each other and against an uncompressed reference sample. The blind part means that the participant doesn't know which sample was encoded by which encoder. That guarantees there'll be no psychological bias towards his/her favorite codec, or against the codec he/she dislikes.

Note that the quote (and here's the nitpick) suggests that double-blind means that the participant doesn't know which encoder is used. Double-blind means that both the participant and the person running the test don't know. By the way, this is, indeed as accurate as double-blind (since, well, the computer might know but surely doesn't care to influence the results). And I realize he doesn't say "double-blind means" but seems to suggest the definition of double-blind. Anyways, that's just the nitpick. Please don't mod me down for it. It's just an observation and I'm trying to build some Karma!

Wrong settings for Quicktime?

[RalphBNumbers]

As I understand it, the "Best" mode, which they used to encode the QT AAC clips, was actually optimized for audio with sample rates well above CDs' 44.1khz. For audio that originated on CD, the "Better" setting would have been more appropriate. (this setting does seem really unintuitive, I would hope for better from apple)

I wonder if/how this would have affected the scores.

I was surprised to se QT AAC ranked so low after it recently won a similar test among AAC encoders, was that HE AAC encoder not included in the previous test?

You fool yourself...

[Moritz Moeller - Her]

only very few professional and gifted people can distinguish or even recognize a good encoded sound from lossless sound on 100 % top end HIFI systems.

Check out the C't listening test (blind test!) done in 2002 or 2003, which showed that people producing classical music, people finetuning codecs and many others were not consistently able to tell the difference. The best tester was someone with a hearing damage on one ear. The psychoaccustics obviously did not work 100% for him.

BTW: OGG won that test for ~100 kbit and higher bitrates. Even well encoded MP3 with 256 are almost perfect.

Re:I'll say this

[Razor Blades are Not]

Actually, storage isn't the only problem. I wouldn't even say it was a big consideration.

The real problem is in the form. PDAs need a certain amount of screen real estate, and a usable manner in which to input data.
Phones, on the other hand, need only to input a small amount of data, but must have a comfortable way of talking and listening to them.

These aims are not really convergent, since the trend in phones is to smaller and PDAs can't get much smaller without sacrificing even more utility. PDAs will always be too large to comfortably hold against ones head like a handset. If anything, PDAs might get bigger again, with some sort of remote (bluetooth?) headset arrangement to take care of the cell phone interface. At least, that's what I'd like :)

Playing music on them doesn't really make much of a difference either way, since it doesn't change the form of either device significantly.

Speaking of strange bedfellows, I was just given the most useless device - a stop watch with a radio in it. I guess it might be useful to listen to the race broadcast while you're sitting in the pit timing it...