Slashdot Mirror
Public AAC Listening Test @ ~96 Kbps [July 2011].
The folks at the Hydrogen Audio Forums have for years been benefiting the world with their patience, technical skills, and hyper-focus on sound quality, by comparing the real-world sound of various codecs and bit-rates for audio encoding. Under the scope for the latest public listening test (slated to run until July 27) are the following AAC encoders: Nero 1.5.4; Apple QuickTime 7.6.9 true VBR; Apple QuickTime 7.6.9 constrained VBR; Fraunhofer (Winamp 5.62); Coding Technologies (Winamp 5.61); and ffmpeg's AAC (low anchor).
What, no comparison with LAME? How lame.
Life is not for the lazy.
So some people will say Codec A sounds best. Some will say Codec B sounds best. Some will say that Codecs A and B suck donkey shit and Codec C sounds best. What exactly does this prove?
I'm staying mostly with FLACs. Works for me. The difference between AAC/MP3 and FLAC (and CD player *) my hi-fi allows to hear quite clearly.
(*) Source for AAC/MP3/FLAC is the Squeezebox Touch (via DacMagic) and when compared to the CD player, the difference of sound quality is noticeable. Not out right bad (that would be Squeezebox w/o DacMagic), in fact quite OK, but still far from the proper hi-fi CD player.
All hope abandon ye who enter here.
FFmpeg's AAC encoder is not finished (yet?), and flagged as experimental. Including it in such a test is rather a dubious idea: it is likely to give a bad impression of the whole project.
Having the new vo-aacenc as contender for the Free Software community would IMHO have been more relevant.
I'm no audiophile, though I do take the time (and space) to rip everything I buy to FLAC. What's the intended application of encoding around 96kbps? Most audio streams online passed that mark many years ago. All in all, this seems like a question best answered years ago. Can anyone point me to what I'm missing here?
Can I take the test even if I am not running Microsoft Windows?
[...]and then call "C:\Program Files\Java\jre1.5.0_15\bin\java.exe -jar abchr.jar"[...]
Suuure
Slashdot editors are paid...?
Nobody cares what the CAPTCHA for your post was.
buried in the readme you'll find
You wanted me to donate some of my valuable time, right?
Almost no one can hear a difference between loss-less and any of the codecs at high bit rates (256K+).
Though many think they can, until actually blind tested.
If you can reliably tell the difference in proper blind testing, you are likely have better hearing/perception than 99.9999 % of the population.
I think I have great hearing, but when I did some ABX testing, my ability to distinguish drops off completely by 160 K VBR on MP3s and that is in quiet room with quality headphones straining to ID any difference.
I am skeptical of any golden eared claims these days pooh-poohing modern codecs.
It is impossible to judge audio codecs through subjective tests.
Companies that manufacture loudspeakers have spent hundreds of millions of dollars on audio quality research- not in order to make their speakers better, but to understand the psychology behind the sounds that make people choose speaker A over speaker B in a showroom. They have discovered all sorts of quirks in human psychology and perception that they exploit to boost their sales, and they have little to do with overall 'quality'. Decades of expensive, meticulous, scientifically valid studies are responsible for the range of speakers you find at the average hifi shop, and even when several identical speakers are demonstrated (but the listener is told they are all different) most people will say that speaker number 2 sounds the best.
The same applies to audio codecs. Even if you eliminate all sorts of hardware variables, then just listening to clip A, then B, then C and subjectively deciding which one sounds 'best' is totally unreliable. The results of this type of testing are completely useless. At the very least you would need to set up a triangle test, and to do this properly with 6 codecs in a controlled environment would take a very long time and the results still wouldn't correlate with true 'quality' unless it was repeated many times with different hardware setups.
Ignoring the psychological weaknesses in these types of tests, the playback hardware would colour the sound enough as to make the underlying test - the codec - invalid. The choice of music, the amplifier, the speakers or headphones, and the volume used for playback will all contribute their own distinctive characteristics to the audio so that person A will not be hearing the same test as person B.
Forget codec wars. Just buy a decent pair of earphones.
Putting syrup in coffee is some form of blasphemy.
Maybe if you'd post some actual insight instead of hating on an article without explanation, you'd get modded up. This is an area of technology I personally am very interested in, as there haven't been any large scale listening tests done on AAC since around 2009, if I remember right.
Why aren't they testing to see how things sound encoded in WAV?
That has been my go-to codec for the last 2 decades. MP3 and AAC are inferior in every way. I'll stake my entire bitcoin collection on that claim.
Any schmo can participate. Many - most would not surprise me - have admittedly inferior audio gear, not the least of which being cheap headphone (nevermind loudspeakers). All in all, I say this is a GRAND WASTE OF TIME. Not that I care. It gives them SOMETHING to do. I remember back in the day ... Ahead would always say "wait for our next version" and of course it was buggy as all and still was bad. listening test or not !!
I'm surprised iTunes wasn't in the list. Isn't it one of the more popular encoders?
If I can be modded down for being a troll, can I be modded up for being an orc, or a balrog?
http://physics.nist.gov/cuu/Units/prefixes.html
k is legal, K is ad hoc.
Most audiophiles generally are not interested in low bitrate audio, regardless of the format - this especially applies to lossy formats such as AAC. 96kbps falls into this category, and both TFA and the associated forum thread on that website offers no insight as to why that bitrate was chosen, or why anybody should care. Since I'm left to draw my own conclusions, I think it's possible that as AAC is the default audio format for Apple devices, this might be of interest to a great many people. However, once again, no explanation is given. Being an audiophile myself, there's little advantage to encoding music in 96kbps AAC as disk space hasn't been an issue since the early part of the century: FLAC is used for the better listening experience.
If the public doesn't care?
Hydrogen? Wouldn't that make it sound really high pitched, like helium?
BTW hydrogen is inflammable - don't try this at home folks
I too had this same question awhile back. Why doesn't HA test commonly used codecs at, say, 192kbps or 256kbps?
The answer? the tests fail because nobody can tell the difference. they make for very boring results.
they run the test at 96kbps because they get usable results. people over a wide range of sound systems and hearing conditions can provide usable responses.
What would you do with that data? hard to say. you can't really extrapolate that, say, if codec A is better than codec B at 96kbps, the same will hold true at 192kbps. In fact, I've seen the direct opposite of that in past HA tests, where various codecs trade the lead depending on bitrate.
So "who is 96kbps for?" I don't know. but "why test 96kbps?" that's easy.
I have to assume this is for streaming
Good guess, especially given 5 GB/mo Internet plans. Also the soundtracks for downloadable video games that have to fit into the platform's 40 MB install package limit.
I have no idea what is even being summarized.
I rather strongly suspect once subjected to rigorous double-blinding you might not come back speaking so boldly.
You think /.ers did evidence before make bold claims? You must be new here.
The article is about 96 kbps operation. When you encode WAV at 96 kbps, do you mean downsample the whole thing to 12000 Hz mono 8-bit LPCM? Or do you mean 24000 Hz IMA ADPCM like some DS games use?
streaming.
Rather than typical net snobbery against lossy encoders, the self proclaimed golden ears should really help out, they are the ones that can spot encodes a mile away, they should be able help find really good/bad encodes here.
I found myself humbled when I attempted to help out before. I had a hard time distinguishing anything but the poor encode used as control.
Really guys this is a chance to help out, or recalibrate your preconceptions about how good/bad modern encoders are.
Or would you rather just keep up with the unjustified snobbery?
I've got the perfect two guys with absolute golden ears. They could hear the difference between a digital music master and a perfect copy of that music recorded to another audio workstation. They said the image was "smeared". The rest of us engineers couldn't hear any difference at all. Turns out they were hearing clock jitter from the AES signal system. We did a data copy and that solved it.
Both guys are legally blind, but they can mix audio.
Most of the stuff on
This is not true.
Frequency-domain codecs have known artifacts that CANNOT be eliminated. Pre-echo is probably the best-known example. A sample with heavy percussion or other complex impulses (like audience applause) will stand out like a sore thumb...
Have you tried the test or is are you relying on something you read?
I just downloaded the files mentioned in the main post. There are 20 samples I gave them a quick run through inside ABC-HR.
The low mark (I assume) stands out like a sore thumb.
But for the other 5 samples, the seem quite indistinguishable on a casual listen.
There is one sample with a sharp percussive instrument (castenatas?) and really I can't spot the difference.
And these are 96K files!!
The state of the art is improving all the time.
Music in particular is very dependent on set and setting. Listening to the same piece of music at night in bed with your eye's closed is very different from the same track in the car on the commute to work. This is of course no function of the music itself, but because your mental state and it's resulting perceptions are different in each situation.
Following the same idea, the testing scenario itself can create a less than ideal context for listening. It is a test; an often more stressful and heightened situation. People want to do well. They also are forced to use other parts of their brain for things like interacting with the testing interface itself. They're maybe in front of a computer screen, or in an unfamiliar place with acoustics that their mind has not grown accustom to. This is not the kind of environment that allows for improved audio perception.
I have no doubt that there are some individual with superior hearing that may have real complaints.
But I don't any of them are the people posting above and dumping on compressed audio, without even bother to try a simple blind test when presented to them.
I just did a casual run through of this test and again only the low end control stood out for me.
These are 96K files. The state of the art has certainly moved on.
Absolutely right. I think 96khz data rate is a threshold where you can start hearing degradations. All these people are doing is testing each other's hearing, not the audio files. We know they're flawed.
Heh... "blind test"... why didn't I think of that?
Most of the stuff on
Absolutely right. I think 96khz data rate is a threshold where you can start hearing degradations. All these people are doing is testing each other's hearing, not the audio files. We know they're flawed.
So you add another unsubstantiated, untested proclamation.
The whole point of this is to find low rate encoders that people can't distinguish.
To determine that, you need people to test it.
Instead all we get are baseless opinions and no testing... sad.
And you're doing... what... to find the answer? Whining on slashdot? That's what's sad. I've already got the answer for my compression needs. Do your own testing and let me know how that came out.
Most of the stuff on
It doesn't matter if it is transparent to a machine. You are simply going by something you read as opposed to actually testing how well it works in practice.
If you want to toss around cute quotes, here is one for you:
"In theory, practice and theory are the same. In practice, they are not."
And you're doing... what... to find the answer? Whining on slashdot? That's what's sad. I've already got the answer for my compression needs. Do your own testing and let me know how that came out.
I already did my own ABX testing for my personal needs a few years back and settled into using ~160K VBR.
I also did a casual test of the samples above as well and I am quite impressed at the progress in encoders. 96K AAC is doing quite well.
I am just sad that on slashdot, a place where I expect some respect for scientific method, when people are given the chance to test experimentally how current encoders are working out(and optionally contribute data), they instead resort to baseless assumptions and preconceptions instead of experimentation.
That's 160k VBR using which codec? AAC? That's about where I also stop hearing the differences, so I just go to 256k for round numbers. Well, round numbers for a computer, anyway.
I may also be fooling myself now because age takes a toll on hearing, but there was a time when there was nothing like listening to a studio master tape. Capitol Records was letting us into their vault to make DVD-A disks about 10 years ago. That's when I started noticing aliasing in the audio - it was actually high frequency content in the music beating against the slight 15.7khz tone I was starting to hear constantly but didn't notice.
I just posted a rant to someone else who thought I was making untested, unsubstantiated proclamations about standard red book CDs which may be interesting:
http://slashdot.org/comments.pl?sid=2346324&cid=36867560
I'll stop ranting now and get some sleep.
Cheers.
Most of the stuff on
I wonder if the latest methods for random sampling and noise shaping could produce a CD mix that you would find more acceptable... Or, if it is just the too quiet passages, it sounds like you were trying to dip too far into the on-paper dynamic range of CDs, rather than compressing to something more reasonable and keeping more signal. A vinyl record only keeps those passages by having heavily compressed dynamic range.
I hope they run this experiment right and do it as a double blind trial so there is no bias, which among auidophiles would be inevitable if you told them what each encoder was before they listened to it.
Do you not own a portable music player? Most of them have small disks, very few even use hard drives nowadays. And if you listen to music on a smartphone (or an iPod Touch), your music is also jostling for space with Angry Birds et. al., making this even more of an issue.
Plus there is also the fact that most portable players produce lower quality audio (especially when paired with cheap "going out" headphones and a noisy environment), so there's little point in using lossless codecs in the first place.
I'd personally be quite interested in this, not for my FLAC collection at home, but for my iPhone's music collection on the move.
As a Golden ear myself, I'd happily join the test, but my ISP steadfastly refuses to fit directional single-crystal OFC optic fibre between my home and the exchange, so any listening test that involves bits going over the internet is utterly pointless.
It's not just audio as such that can be encoded at low bitrates, more important is the audio track emdedded in a streaming video. The video is already hogging bandwidth just to look halfway decent, especially if streaming over a mobile connection, so you would want the best low-bitrate audio encoder to go with it.
True confidence comes not from realising you are as good as your peers, but that your peers are as bad as you are.
Podcasts, internet radio and other streaming media maybe ?
given modern music, the codec is irrelevant. sounds shitty anyway. I'm getting old.
I thought 96 kbps was "lo-quality" for internet radio and other streaming audio since at least 2004 or so...
If I wanted to stream audio live from my home connection I could, theoretically, stream it to 700+ users at once even at 128 kbps (A more realistic figure might be simply "hundreds of users" but my point still stands, this isn't 1999 anymore).
So yeah, it can still be interesting for truly massive setups where you have thousands of listeners at once (or you're already using loads of bandwidth for video and every little saving counts) but overall, for most people, testing 96 kbps just isn't all that relevant (besides, most people seem to hate HD video with overcompressed audio, nothing ruins your viewing experience like horrible audio to go with your perfect 1080p video).
Greylisting is to SMTP as NAT is to IPv4
All my music is in 128 Kpbs MP3. Why? So I could fit it all in my 4 GB Creative ZEN. Before that I had a 512 MB Sony and all the music was converted to 64 Kpbs .oma files. With earphones the quality has always been fine enough for me. When I get a new mp3 player with 8 GB or more I guess I gotta go to the library again so I can rip all the music in 256 Kpbs or better quality. Actually what I think as quality is lyrics, rhythm and melody, not the bit-rate.
Um what? Humans can only hear into the low 20KHz range, last time I tried I heard either 20.5 or 21.5KHz [can't recall which] but it was very heavily attenuated so I don't really count it that much. To accurate reproduce that tone you need only encode at ~41-43KHz. The only reason to encode higher than that is so when you're editing/mixing you have more samples to work with and filters are not so aliased. It's the same reason to go for more depth. People really can't hear more than 100dB dynamic range or so [more if you're in a sound-proof chamber... but who is?], and CDs provide 96dB. The reason you want to record in 24 or 32-bits is again aliasing errors.
Once you master the disc 16-bit 44KHz is more than enough. After that it's all up to your speakers, the room, and your ears.
You mean the bits change if the cable isn't Golden enough?
True confidence comes not from realising you are as good as your peers, but that your peers are as bad as you are.
The limit for WiiWare games is 40 MB because the Wii's internal storage is essentially a 512 MB xD-Picture card. Sure, games can be stored on SDHC since Wii Menu 4.0, but the launcher just copies it to internal storage before running it. Likewise, the limit for Xbox Live Indie Games is 50 MB at the lowest price tier.
@ $3000/m. they will.....
jizz up your love life.
Stiffen your pecker
lighten the load on your wallet so you can sit flat on your couch while "pullin' the pud' about music
I think all this dicussion is an example of GET A LIFE......
I use FLAC, AAC MP3 and dont give a shit coz' in the end, the music for me is background and I dont feel the need to do any of the above
sheeshhhhh
One of the codecs I can identify, it sounds like low-bitrate MP3 to me. The rest sound indistinguishable from the reference.
Are all codecs (except one) that good? Or are they messing with us, and they're testing something else entirely?
The 96kHz data rate referenced above is the clock rate of the finished data stream, not the audio sample rate or the upper frequency limit.
In terms of reproducing accurate waveforms, harmonics which extend well beyond what we can hear as pure tones play a very important role. A digital system like a CD may test as perfect in every way but there are subtleties which are selectively compromised to make it possible to create CDs. Recreating supersonic harmonic components is one of the compromises. Back when I was involved with making DVD-A disks, the differences between the 192kHz 24bit PCM stereo tracks from the master and the resultant 44.1kHz 16 bit tracks was astonishing. I don't think there's a measure for "clarity" or "accuracy", but those elements become clear once you've been able to A/B the two systems.
Most of the stuff on
The low anchor encoder is pretty bad, but likewise that is the only one I can detect. Good thing it is there or I wouldn't be sure the test is working. I think all the samples are correct. They are not messing with us.
Chalk it up to some combination of the encoders being that good and our average hearing.
Too bad none of the guys with super hearing were brave enough to give it a shot.
From what I've seen, you get bad karma for being a douche. I post freely, as an opinionated old asshole, and I manage to keep good karma. Am I wrong sometimes? Of course. Do people disagree with me? Often. Am I always pleasant to my fellow posters? Only if I feel like it. Do I think I know it all? Not really, but almost. Opinionated old asshole, very definitely, but with good karma.
May I suggest that you don't be a pedantic douche? Your post right here, for instance, is an instance of whining. "I've got bad karma from "questionable" comments" Bullshit. The moderation system here is nothing to brag about - but in the long run, it works. Think about it . . .
96kbps, NOT 96kHz. Big important difference.
Yes.... using kbps would have been clearer.
Most of the stuff on
streaming.
Especially in the light of bandwidth caps for mobile devices.
Fandroids hate facts.
I thought 96 kbps was "lo-quality" for internet radio and other streaming audio since at least 2004 or so...
You've hit the nail on the head. In the old days of MP3, 96kbps was considered a "low quality" bitrate. We're now many years later, and various encoders have matured to the point where some of us feel that it's worth testing to see how they fare.
96khz
96kbps. We're talking about encoding bitrate, not sample rate.
As a Golden ear myself, I'd happily join the test, but my ISP steadfastly refuses to fit directional single-crystal OFC optic fibre between my home and the exchange, so any listening test that involves bits going over the internet is utterly pointless.
Mod up for humor, please.
Someone already beat you to it. What was I thinking?
Most of the stuff on
What difference does the encoding make if the source material is crappy to begin with? I do not mean crappy like (insert favorite type of music to hate here), I mean crappy like you can not even really distinguish the instruments from each other? I am told this is due to "loudness wars" but I really have no idea. There is very little clarity in modern music so the encoding really does not matter very much.
strike
"Someone needs to talk to the tree of liberty about its ghoulish drinking problem." by ohnocitizen
then go ahead and listen to your 128 Kb/s MP3s. Even with my cheap (really cheap) headphones I can tell between a 320 Kb/s MP3 and a FLAC. The artifacts you hear are similar to those you get in albums with heavy dynamic range compression, such as those by Oasis. Does that mean I would get a perfect score at a blind test? No, of course not, but I would be willing to bet I would surpass the 50%.
That being said, the difference is heard mostly while comparing MP3s and FLACs, trying to find faults. I don't really find MP3s repugnant. I find bitrates as low as 160 Kb/s very listenable, but being I have got the storage capacity and the bandwidth, why not get the highest possible quality? After all, even a subliminal gain is a gain.
golden ears arent really required when youve listened to the same song 1000 times.
You have a 100 mbit/s upload ?
Yes, it's not particularly expensive or uncommon in Sweden.
Greylisting is to SMTP as NAT is to IPv4
Hehehehe sweden that explains it :D.
...
Iirc your high speed infrastructure was built for home schooling, eg videoconfering with teachers because of
the enormous distances between people and the various snow blizzards ?
But to stay on topic wouldn't a very good low bitrate codec be awesome to enhance the codecs used in phones, walkie-talkies, etc
Right now they use codecs designed for their purpose but not a lot of further development of them has been done that i know of.
Iirc your high speed infrastructure was built for home schooling, eg videoconfering with teachers because of the enormous distances between people and the various snow blizzards ?
Nah, it's mostly that there was a big push with government-funded backbone buildouts as well as the fact that various laws force what you might consider cooperation and resource sharing.
But to stay on topic wouldn't a very good low bitrate codec be awesome to enhance the codecs used in phones, walkie-talkies, etc ...
Right now they use codecs designed for their purpose but not a lot of further development of them has been done that i know of.
Well sure, but my point was more that I, like many "regular" users am more interested in finding the best codec in terms of the bitrate at which you have a "perfect" sound (as opposed to finding which codec performs the best at a specified bitrate).
Heck, even 128 kbps mp3 sounds pretty decent these days since encoders aren't as lame (yay for puns!) as they used to be. Still, I tend to stick to 192+ kbps for music otherwise I do tend to hear minor differences when listening on a good stereo. For my iPhone I could probably just encode everything as 128 kbps mp3 and not hear a difference but at home, seated on the couch and listening actively there are definitely songs that even the best mp3 encoders seem to have issues with at bitrates of 160 kbps or lower, nothing like it was a few years ago but storage and bandwidth have long been cheap enough that to me this isn't an issue. But yeah, for things like phones and walkie-talkies low bitrate performance is a good thing although I'd rather want to know what the performance is like at 24, 32, 48 and possibly 64 kbps in that case...
Greylisting is to SMTP as NAT is to IPv4