Slashdot Mirror
2nd Multi-Format 128kbps Public Listening Test
technology is sexy writes "Roberto Amorim has launched his latest public listening test evaluating the performance of different audio codecs at 128kbps, among them Apple's AAC implementation (used in iTunes), LAME, Ogg Vorbis fork auTuV, WMA, Musepack and even Sony's Atrac3 format, which is soon to be used in their own music store. Read more on Hydrogenaudio and check out the results of prior tests. As opposed to most evaluations of audio codecs, this is a scientific test adhering to ITU-R BS.1116-1 as much as possible while still allowing everybody to participate."
Because "human auditory capacity" is not fully understood. Sure we can give standard frequency response graph, but most of these codecs take advantage of psycho-accoustic hearing models -- where certain frequencies mask other frequencies in our perception. Since this is a developing field, objective listening tests could really help determine what's working and what's not.
Because "human auditory capacity" is not fully understood. Sure we can give standard frequency response graph, but most of these codecs take advantage of psycho-accoustic hearing models -- where certain frequencies mask other frequencies in our perception. Since this is a developing field, objective listening tests could really help determine what's working and what's not.
From my understanding of MP3 compression and others, the compression protocols take advantage of this frequency masking, so if humans can't hear it, it removes it. It also obviously takes into account frequency ranges of hearing. As a side note, I think it might be neat to be able to compress 30-50% better based on your personal hearing characteristics, but it'd stink if you got old and had to not only wear a hearing aid, but also start collecting MP3's all over again.
Any encoder sounds great if you throw enough bits at it; the trick is sounding good when the bit reservoir is shallow.
Same deal for MPEG-2 encoders, they all look great at 7 Mbit+/sec but the real test is 3-4 Mbit/sec.
Of course, proving the patent-freeness of Vorbis requires searching every single patent with a fine-toothed comb, further indicating how messed-up the whole patent system is at this point.
I just have to wonder how many companies are waiting to pounce on the first major commercial user of Vorbis with a patent suit. (Yes, I know there are commercial users of Vorbis, but none are really big enough to attract patent litigation, especially since none of them are wedded enough to vorbis that they wouldn't be able to just drop it for mp3 support with its well-known and, IMO fairly-reasonable, license fees.)
There are 10 kinds of people: ones who understand ternary, ones who don't, and ones who think this joke is about binary
There may be some merit in attempting to quantify codec properties. Don't think these tests will provide any useful answers. Whatever value the results have will be squandered by the incessant blather of countless audiophiles. They will prattle on with their subjective claims and anecdotes until any credible results fade into the noise. Left with nothing credible on which to base decisions, those who use codecs to publish sound will consider cost/profit and little else.
The purpose of being an audiophile is to discover/invent/imagine minute differences in noise. There is no "best" speaker, codec, bit rate, circuit or recording method. Live performances are flawed because the auditorium will not have been built 2 days prior based on the latest theories of sound propagation and is, therefore, obsolete. No combination of equipment or technology will ever satisfy even the most tolerant audiophile.
I clearly remember the awe we once had for the phenomenal quality of 128Kbps CD quality recordings. It was called "near perfect" and "almost live." Today, 128Kbps is considered laughable crap suitable only for fools that don't know better.
The whole thing is a pseudoscience wannabe filled to the brim with vendors lauding useless gear before gullible fools. I will not participate.
Maw! Fire up the karma burner!
Not possible. All you will get is a bunch of WAV-files, you have no way to tell which file belong to which codec.
.ogg vorbis, an mp4 and 3 flacs. If you want to be biased either for or against mp3/oggvorbis/quicktime itunes AAC, you can.
Check the contents of the sampleXX.zip files; you actually get an mp3, an
SCO employee? Check out the bounty
q6 is imperceptible from redbook in 19/20 samples for me, LAME --alt preset extreme (200-220 kbps VBR for most samples) is better at about 29/30 samples. This was from a double blind computer generated arangement using the same equipment and one listener with good hearing.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
I think that's not the case for many people. Not everybody has high speed internet, many people still have dial-up. Many people want their computers to last a bit longer and a 500 MHz PC with, say, 128 MB RAM en 20 GB disk, is still useful for a lot of things, if only for playing music, writing stuff and using the internet. And besides, portable players are not (easily) available in 200+ GB capacities.
But I admit: when speed and storage are not issues, by all means go for quality. Currently, I use 192 kbps AAC and all my music stores just fine on my iPod. But both my iPod and G5 are very recent purchases and I want them to last for quite a while yet!
Also what kind of codec bias could you possibly be referring to?
Apparently he doesn't realize that this is a double-blind test - meaning neither the listener nor the tester knows what codec is being presented at any given time.
I'm taking the test now (well, not right now, taking a break) and it's about as scientific as I think you could make a public test taken in the home. Yes, the samples get compressed and then put in easily accessible folders with proper file name extensions, but you never know what you're actually listening to when you're running the testing program. All you have is a source file for comparison, then two buttons marked "1" and "2", one of which is the source again, the other a randomized codec. You never know which of the two buttons is the uncompressed source and you also never know which codec you're hearing. The results are also encrypted, so it's not as if you can just go into the results files and look at what codecs you favor.
I suppose someone who's truly got the Ear of the Gods could listen to the samples outside of the testing program, pick various identifiable traits out of each, then listen for those traits in the testing program and vote up or down whatever codecs he or she chose, but that would be exceedingly difficult and more than a little time-consuming. I can't see how it would be worth it, especially as no single test result is going to skew the overall results to any significant degree.
This is the first time I've ever taken a test like this and I am honestly pretty shocked at how good all of these codecs sound. I am having a really hard time even deciding which is the compressed track most of the time, and I consider myself something of an audiophile. I'm even listening in a fairly controlled environment with a good pair of headphones, at a volume loud enough to hear any background noise clearly but below any clipping whatsoever. I will be surprised if any codec really does significantly better than the others consistently when we see the final test results.