Slashdot Mirror
Listening Comparisons For Audio Codecs At 64kbps
waaka! writes "Hydrogenaudio has just wrapped up a listening test of various audio codecs at 64kbps. Check out the results, where Ogg Vorbis performed quite well, scoring significantly better than WMA, RealAudio and QuickTime AAC, and kept pace with MP3Pro and HE-AAC (AAC with the SBR extensions that MP3Pro uses). Clearly, though, no codec can honestly claim 128 kbps MP3 quality at 64 kbps. The charts at the end show entries for 128kbps LAME MP3 and 64kbps FhG MP3, but these are used as high and low anchors for reference, as MP3 is really out of its league at bitrates such as these."
... "Why not just encode it at 384 and be done with it?", consider that PDAs have 64 megs of RAM, cell phones get no better than 56k, and that not everybody has broadband.
...to compensate for everyone's crappy $1.99 computer speakers?
"A great democracy must be progressive or it will soon cease to be a great democracy." --Theodore Roosevelt
Sometimes, simple audio clips don't require the highest quality. Function triumphs over high performance hot-rodding.
You used to compare against CD quality.
Oh well, times change, I guess it's time to throw all my CDs away and instead store all music in this new exciting digital format.
And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum.
I've also noticed sometimes that 28 kbps streaming audio in some sites is much better than 56 kbps audio on other sites
"the purpose of PDAs is not to play music "
Yet they do, and people like to use them for that. Fascinating.
"the purpose of cell phones is not to play music"
Yet the holy grail of mobile computing is to merge the PDA (which can play music) and the cell phone.
You may like carrying around a cell phone, PDA, and iPod in your pockets, but I want one device that does it all.
MP3s are ubiquitous. My computer, DVD player, portable audio player, and car stereo all support it. The same can't be said for other formats.
More to the point, why are all of these competitions at such low bitrates? The differences in quality between various types of audio compression become indistinguishable (and therefore irrelevant) as you raise the bitrate.
I just use good old variable bitrate MP3 and forget about it. Simple and standard.
GeekNights!
Late Night Radio for Geeks!
Its nice to see ogg doing well, but ogg needs to start advertising. Nobody is going to give a shit about ogg if computer companies (apple) is distributing aac and their old napster mp3s don't play on an ogg player. Fact of the matter is that there is a LOT of power behind the MPEG (4, 7, 21) movement.
Have you seen the cuidado project? Have you read what companies publish? Why fight the good fight if you aren't getting the public hooked on it while its still the best option.
hmm
Rob
I saved this thread quite a while ago and I agree with several of the recommendations (notably with the 'Tori Amos' 'Boys for Pele' CD, not that it's the type of music I usually listen to, but I have to admit the production values are outstanding), after all using hyper-compressed (re: other slashdot articles) crappy source material is not that helpful in terms of figuring out how good the various encoders really are...
the thread on google
Personally I rip my own CDs with lame --alt-preset extreme (on said Tori Amos' CD it seems it hovers around 224kbps with -lots- of frames at 256 and 320), for fun I transcoded (I know, transcoding is bad, mmkay?) a few of them to vorbis 48kbps and it's amazing how good they sound at that low of a bitrate.
-- the cake is a lie
I always encoded my MP3s 224 kbps and when iTunes came out I simply continued the tradition.
In any case, I can certainly notice the improvement from MP3s encoded at the same rate.
Still #1 -- Lonely Gay Geek
Are we at the LIMITS of compression technology ? Is there anything new being worked on by anyone ?
You were probably almost fired for listening to such a shitty band, not for sharing MP3s.
In Norway, any copies of any music downloaded for personal use are legal, so as long as I don't share or sell anything I download, I can download as much MP3 songs and DivX movies I want.
:-D
So, as I don't have any P2P software running at the moment, the number is zero.
This message has been ROT-13 encrypted twice for higher security.
Anything below 128k/s (in my opinion) is only good for streaming and embedding. Even 128 is the bare minimum for anything that sounds decent. Are there any comprehensive articles that deal with comparing high encoding rates (192+) of multiple formats?
It should also be noted that it is not recommended using CBR encoding with OGG. It is a native VBR codec that is only forced into CBR for steaming. The quality of CBR is much lower than VBR. It would be very nice to see a comparison that uses VBR for all codecs that stick to the same bitrate range.
The poster offers an interesting interpretation of the results, but only his/her comments support Ogg Vorbis in this case. The numbers tell a completely different story.
The analysis presented leads us to one conclusion: use Lame 128. It's strictly better than all other options. Do not use FhG MP3. Easy.
If you're willing to slip to 4th best encoder, then consider Ogg Vorbis. 4TH BEST. That's hardly the rosey picture painted in the article.
Also, don't be deceived by the "confidence intervals" shown in the graph. They're all drawn to the same widths for each set! At best, this is an approximation. At worst, the author is simply using a program that draws in some uniform (and meaningless) bars. Fear graphs.
"actually, I carry a cell phone, PDA, gameboy advance, *AND* an iPod. You know why? I hate compromising. And like it or not, there aren't any decent "convergence" devices that can do 2 of those things well, let alone 4."
I agree with you from the gaming standpoint of it, but I don't on the rest of it. There's no reason why a single device can't be a PDA, phone, and Mp3 player. The storage is the problem now, and one day in the not too distant future, that'll be fixed.
Why is it that no-one ever quotes their hearing test results when doing these subjective tests? It's just like when my co-worker tells me that watching DVD's on my 1600x1200 resolution screen is not as sharp as his German 100MHz standard television.
The fact that his visual acuity has been compared to Mr Magoo never comes into the equation...
that Joe Schmo out there with the Windows machine will be pretty much sticking to WMA. Sure hardcore audiophiles can tell a difference between formats but the average computer idiot doesn't care.
The saddest part of all is that WMA is a beast that is growing and will be hard to get rid of. Since MS has submitted this format for inspection for widespread adoption, they will continue to force their way into this becoming the de facto standard even though it sucks ass. More importantly, because of the draconian DRM, WMA is the format that the RIAA and other head asswipes at recording labels are drooling over. They could really care less about the quality of their digital music as long as they control the rights management on it.
Until more portable players support formats like ogg, WMA will be an immovable force, the 800 hundred pound gorilla that will be difficult to move.
No trees were harmed in the composition of this; however, numerous electrons were inconvenienced.
Yes, it was a double blind (ABC/HR) test. All the information is available in the links presented. The testers are presented with several groups of two samples, and are asked to subjectively compare the samples and state which one is the original and which is the encoded sample. Furthermore, one group contains two of only the non-compressed sample. Thus, information where one of the non-compressed samples is rated lower than the other can be easily discarded.
Which won't happen until someone designs a better method of inputting data. Keys only get so small before they're not usable and phones can only get so big before it's bulky.
You see, that's why choice and diversity are a good thing.
You like having a dedicated device for everything, some people (me included) like having something small and convenient that'll do a fair job of a lot of things....
However, don't make the mistake of thinking that just because your preference means there is no need for something (in this case low bitrate digital audio) therefore means that no one has any need for it.
Advanced users are users too!
Comparing without a reference reflects how much you like the encoding of the codec, not how accurate it is to the original. For example, if a codec boosts the bass or encodes slightly louder, you may interpret this as better sound. For example, when auditioning speakers, you must always balance the output of the speakers as most people will psychologically prefer the louder (most sensitive) speaker. This does not mean the speakers are accurate however.
At any rate, here is the relevant quote on that page:
Note that the quote (and here's the nitpick) suggests that double-blind means that the participant doesn't know which encoder is used. Double-blind means that both the participant and the person running the test don't know. By the way, this is, indeed as accurate as double-blind (since, well, the computer might know but surely doesn't care to influence the results). And I realize he doesn't say "double-blind means" but seems to suggest the definition of double-blind. Anyways, that's just the nitpick. Please don't mod me down for it. It's just an observation and I'm trying to build some Karma!
Sunny
Be my Friend
Nobody will ever need less than 64kbps of audio.
As an artist that releases mainly online, I found these results very interesting, and thought I'd share my feelings with the slashdot community.
:D
While MP3Pro and Vorbis were good competitors overall, and have a fairly good footprint to boot, I'd have to say that if I'm forced to encode to 64MBit/s, I'd absolutely choose Ahead HE AAC, if I'm judging solely on this comparison (which I am at this point in time...)
Why? Because there was no sample that Ahead HE AAC did POORLY at. MP3Pro and Vorbis (and all the other codecs) each had one or two samples that they just totally choked on, quality-wise. So if I was forced to use a 64 MBit/s codec, it would absolutely be Ahead HE AAC, because while it didn't score highest on every test, and the three codec were virtually tied across the whole competition, I would feel far safer trusting my best digital work to a codec that, according to this test, would have the least chance of representing it particularly poorly.
I wonder how these results compare to higher encoding rates; I could easily imagine that most codecs have a sweet spot, where the encoding quality/bitrate maximizes... it would be interesting to do some research to find this sweet spot.
Anyone want a quick way to slashdot a server?
I am disrespectful to dirt! Can you see that I am serious?!
- Almost ALL of these kids have cell phones. These are middle class kids mind you.
- Almost ALL OF THEM message each other using that miserable little numeric pad.
The issue is that you're too old!Pretty impressive really. I'm trying to change my ways and try using my itty-bitty phone for e-mail and whatnot. I mean gimme a break, I'm 29 and already getting outpaced...
Computer Science is Applied Philosophy
As I understand it, the "Best" mode, which they used to encode the QT AAC clips, was actually optimized for audio with sample rates well above CDs' 44.1khz. For audio that originated on CD, the "Better" setting would have been more appropriate. (this setting does seem really unintuitive, I would hope for better from apple)
I wonder if/how this would have affected the scores.
I was surprised to se QT AAC ranked so low after it recently won a similar test among AAC encoders, was that HE AAC encoder not included in the previous test?
"The worst tyrannies were the ones where a governance required its own logic on every embedded node." - Vernor Vinge
It's worth pointing out that at least MP3Pro and HE-AAC from tested codecs use SBR. SBR is a method (mostly post-process) that allows transmission of lower half of audio spectrum, and have the decoder "guess" what the the other part of the spectrum would have been. While this allows for "cool-sounding" audio at low bitrates, the generated part of the spectrum is not actually an encoded original audio, but rather its "guessed" reconstruction. SBR is also patented.
Search for more info on SBR if interested, like this one.
only very few professional and gifted people can distinguish or even recognize a good encoded sound from lossless sound on 100 % top end HIFI systems.
Check out the C't listening test (blind test!) done in 2002 or 2003, which showed that people producing classical music, people finetuning codecs and many others were not consistently able to tell the difference. The best tester was someone with a hearing damage on one ear. The psychoaccustics obviously did not work 100% for him.
BTW: OGG won that test for ~100 kbit and higher bitrates. Even well encoded MP3 with 256 are almost perfect.
Moritz
Actually, storage isn't the only problem. I wouldn't even say it was a big consideration.
:)
The real problem is in the form. PDAs need a certain amount of screen real estate, and a usable manner in which to input data.
Phones, on the other hand, need only to input a small amount of data, but must have a comfortable way of talking and listening to them.
These aims are not really convergent, since the trend in phones is to smaller and PDAs can't get much smaller without sacrificing even more utility. PDAs will always be too large to comfortably hold against ones head like a handset. If anything, PDAs might get bigger again, with some sort of remote (bluetooth?) headset arrangement to take care of the cell phone interface. At least, that's what I'd like
Playing music on them doesn't really make much of a difference either way, since it doesn't change the form of either device significantly.
Speaking of strange bedfellows, I was just given the most useless device - a stop watch with a radio in it. I guess it might be useful to listen to the race broadcast while you're sitting in the pit timing it...
While Hydrogen Audio did provide the resources to host this test, the real work was done by Roberto Amorim, who organised this monster.
Credit where credit is due.
Use ISO 8601 dates [YYYY-MM-DD]
Ok, first of all, while the test was DISCUSSED on HydrogenAudio, and most of the participants are HA regulars, Roberto Amorim did all the hard work of organizing the test and compiling the results, dealing with complaints, etc. To not give him credit is not very nice.
/.ers seem to be only half-literate (can write but not read). There is a hilarious number of denegrating comments here by people who know nothing about either statistics or psychoacoustic audio compression. ABC/HR type methodology is the standard for comparing the relative quality of audio sources. Also, a great deal of effort went in to assuring that the best settings for the best encoders for each codec were used for the test. A little reading of the pre-test discussions would reveal this. Further, HydrogenAudio is not a club of audiofools who spend zillions of dollars on fancy speaker cable without any science to back it up. It is an objectivist forum. Anyone who makes statements without backing them up (with something like ABC/HR or ABX results) gets flamed HARSHLY. Some of the regluars have PhDs on various audio topics. They know what the fuck they're doing.
Second, there was a 128kbps test a month or two ago (which for some reason got repeatedly rejected when submitted to slashdot). You can see it here. Unfortunately, the results there aren't quite as interesting (it was mostly a big tie). Unfortunately, tests at higher bitrates are difficult because detecting problems at, say, 160kbps often requires well trained ears and good audio equipment.
Third, it's a good idea when commenting on an article to actually read it and click around on a few links to actually have an idea what you are talking about. Many
Fourth, just because you don't have a use for 64k audio, doesn't mean the results are meaningless. Lots of people have small-capacity players, and some codecs can tolerate that bitrate for very casual listening (such as in the car). Lots of streaming audio sources are at this bitrate or lower. Satellite radio is at 64k or lower. Also, it's not a good idea to try to extend these results to other bitrates. MPC for example, isn't even worth considering at 64kbps, but at bitrates over about 140kbps, it will beat the pants off of anything else.
Finally, for those who want to know more, or want their audio collections to sound best, read the FAQs at HA. Many codecs have a preset where they are transparent for the vast majority of samples; usually a VBR setting that averages somewhere between 160 and 200kbps (such as lame --preset standard, mppenc --standard, oggenc around -q5 or -q6).
... "Give me a woman who loves beer and I will conquer the w
I want to give full respect to the people who put all the research into creating these new audio formats. The results are truly phenomenal for 64kbps codecs. It's a fabulous academic demonstration.
However, each is what it is. A 64kbps codec.
I have about $6000 invested in my 2-channel +subwoofer setup here at home, and I consider that moderate compared to what you can truly achieve. I love listening to music, and it is completely remarkable when it is reproduced as realisticly as possible. So I go to painstaking methods to make sure the AC power is clean, the wiring is right, the distortion is low as possible. The signal to noise ratio is far between, with a good amp, and great speakers... I am especially pleased when the recording I am playing on my wonderful system is in the best production quality that it can possibly be.
As amazing as they are, these 64kbit formats are useless on a person like me. I crave LOSSLESS not LOSSY. I might as well be listening to music on a $60 AIWA boombox, since it would sound relatively similar either way. All the subtle beauty and realism of the music is completely wasted with destructive compression.
And for those of you that say it's for portable devices, It's not too unreasonable to get a portable player that plays high streaming VBR mp3s with some nice ~$100-$150 headphones. The small little investment to hear your music from 20hz-20khz flat response with low distortion is worth every single penny.
I simply do not understand the need to take our ever improving technology and lower the quality of the music. If anything, it should be increasing... higher resolutions. 24bit/192khz technologies, and wonderful DSP equalizers, large portable storage devices... they are all realities now, but nobody seems to care but the fanatics like me. I would think that techno geeks would care more about the music they love, but that does not seem to be the case. The only logic that I can fathom to explain why is that perhaps they don't even know what they're missing. I know I didn't, until I actually experienced how good sound quality can be on the right system.
>And seriously, does anyone listen to music encoded at 64 kbps? 128 is the bare minumum.
I have a tiny gadget that I plug into my car stereo so I don't have to lug a CD case around. It holds 192MB of data, so to make the most of it I compress everything to 64kbps. Since the music is normally competing with road noise anyway, it's "good enough".
Through headphones, though, one really can tell the difference.