Slashdot Mirror
1/3 of People Can't Tell 48Kbps Audio From 160Kbps
An anonymous reader writes "Results of a blind listening test show that a third of people can't tell the difference between music encoded at 48Kbps and the same music encoded at 160Kbps. The test was conducted by CNet to find out whether streaming music service Spotify sounded better than new rival Sky Songs. Spotify uses 160Kbps OGG compression for its free service, whereas Sky Songs uses 48Kbps AAC+ compression. Over a third of participants thought the lower bit rate sounded better."
I don't care if 99% of the population cant tell the difference between the two, I can and I want all my audio to be 320Kbps
I would be more impressed if the same encoding format was used. I think both samples should have been ogg or aac and not a mix. If comparing aac at 48 and 160 are the results different? Same goes for ogg at 48 and 160?
People who can't tell the difference have a 50-50 chance of getting it right. Therefore we can deduce that over *two-thirds* of the population can't tell the difference, by adding in the inferred members who couldn't tell, but guessed right.
Do it with 48kbps AAC vs. 160kbps AAC, or 48kbps OGG vs. 160kbps OGG, and you might have something meaningful.
Or, 48kbps AAC vs. 48kbps OGG, and 160kbps AAC vs. 160kbps OGG, if you want a flamewar...
Village idiot in some extremely smart villages.
If the higher compression audio had simply used this $500 Denon ethernet cable, the results would have been different:
http://www.usa.denon.com/ProductDetails/3429.asp
But seriously, can you make a sweeping statement like "People can't tell 48k audio from 160k" if you're also switching compression technologies? OGG vs. AAC is a whole article on it's own, you just muddy the waters by making this about the compression rate.
This is just a new version of the old megahertz myth of the CPU wars. Two different 2GHZ processors from different manufacturers are not equal, we all finally figured that out for the most part, right? Now we've moved onwards... to the Kbps myth?
>> 1/3 of People Can't Tell 48Kbps Audio From 160Kbps
Correction: Over a third of participants thought the lower bit rate sounded better.
Those are not the same thing. To find out how many people thought they sounded exactly the same, I would have to RTFA.
There are a lot of things to mention in this article. They are using VERY high end hardware that can interpolate the sound and cause sound clipping (which makes things sound metallic) to be minimized. They also didn't mention what songs were chosen. A lot of music is mastered to sound good on poor quality speakers and thus the 48 Kbps may actually not be the limiting factor.
At least there going to be a new reason to sell audio snake oil now.
To elaborate: in my testing, I took a couple of random tracks (two Coulton rock tracks and two classical Christmas tracks, both FLAC), and encoded them at 96k, 128k, 160k, and 192k ogg vorbis, then played them each into their own wav file, then distributed the re-encoded wav files and a wav generated straight from the flac (all with randomized filenames) to the people who wanted to take part in the test. There was a statistically significant (although not universal) recognition that the 96k was the worst. There was a correlation on the 128k track, but not a statistically significant one (I may want to do this again with a larger sample size). And the 160k, 192k, and original tracks were as good as random.
Most people hear 128k and think, "How can a person possibly not get *that*?" But that's really a stereotype from the olden days. There's a huge difference between a 128kbps fixed-bitrate mp3 and a 128kbps VBR ogg. VBR makes a *huge* difference.
Mr. Wizard... why is this place called the Cave of Hopelessness?
The summary is quite misleading.
It sounds like 100% of the participants could tell the difference between the two encodings, just 1/3 of the people thought the more simple, clean, highly compressed version sounded better. 2/3 of people thought the high bitrate version sounded better.
When choosing compression, the better way to go is to shoot for transparency versus the uncompressed source, not which audio sounds better to your ears.
That's why ABX is the industry standard for compression comparison, not a simple AB test as in this experiment.
Blessed are the pessimists, for they have made backups.
this summary is misleading. they were asked to choose which they thought sounded better. the listeners DID notice a difference between the two, and for some reason 1/3 of the participants enjoyed the lower bitrate version better. perhaps it had less harsh high tones or something about it was more pleasurable to them... that doesn't mean that the higher bitrate didn't honestly sound more accurate to the source material. Perhaps uncompressed audio should have also been incorporated into the test. If they still choose the lower bitrate over uncompressed, then it's clear that some listeners prefer the song with the changes inherent to compression.
this was a very unscientific study, with a very small sample size, and really shouldn't be front page on slashdot.
frog blast the vent core
This wasn't a proper repeated ABX double-blind listening test, nor an ABC-HR, but just a single-trial single-blind AB for each person, with one track and no hidden reference. Pathetic and unscientific, and definitely shouldn't be presented as a valid listening test, given how susceptible audio research is to error.
Dear C|Net: If you're going to do a listening test, please don't just do something that'd get you laughed at on (then banned from) Hydrogenaudio. It's easy to do it properly - Hydrogenaudio have been doing it for years, and that's how the encoders are tuned. Doing it wrong tells you nothing of value.
Previous, proper ABX double-blind listening tests have proved that Vorbis -q5 (using AoTuV b5.5), which is what Spotify use, is perceptually transparent on almost all listeners on almost all audio. Meanwhile, 48kbps AAC-HE+SBR with a good encoder is best-in-class for its bitrate at the moment, but is very poor at some sounds which spectral band replication tends to make too prominent or artificial; electronic music encodes well, but classical most certainly does not. It almost always is distinguishable in ABX, although it ranks moderately highly in ABC-HR, especially for its bitrate, on untrained listeners. It's not even remotely a competitor to Vorbis -q5, though (or LAME 3.98 -V2 for that matter).
Want research sources? Hydrogenaudio listening tests, and/or peer-reviewed papers conducted using similar/the same methodologies. Want to contradict those? Do your tests properly first.
While, as you say, most people have crap speakers/headphones, so anything above 128kbps is largely a waste, there is one major reason to do it anyway.
If you ever upgrade your hardware, dealing with all your old, low-quality, tracks is a pain. You can re-rip, or suffer through, or throw them all away and get new ones; but it is a hassle. With storage so cheap these days, you might just want to include a little extra, in case you upgrade later.
I used to sell audio equipment as a teenager and I recall different people had different ideas about what constituted quality audio. Some people liked deep muddy base, other people liked loud midranges, etc.. I think the study's conclusion is all wrong... it's not that people can't tell the difference, it's that people sometimes prefer the lower quality bitrate. Personally, I just want things to sound representative of the real-life equivalent. :)
No, just the headline is massively misleading.
The article actually states that people (a) could hear the difference (b) thought the lower bit rate stuff sounded better.
The key being that the two were encoded with two totally different codecs.
Today's low-bitrate MP3/AAC will be tomorrow's vinyl.
I firmly believe that you prefer what you're accustomed to hearing in the first place. Most kids today have grown up hearing nothing better than highly-compressed FM or low-bitrate MP3 music. They don't know anything better, and given the option of hearing better music, perhaps even uncompressed, with a much larger dynamic range and noise floor, they'll gravitate to what their ears and brain have been trained to appreciate.
Tomorrow's world will have "128Kbps MP3 Afficionado" publications extolling the virtues, "warmth", and "naturalness" of the low-bitrate MP3. And audiophiles will pay top-dollar for crippled hardware and overcompressed, undersampled music tracks.
"we tested with Billie Jean"
I don't hate that song.. but as a testing ground for music hardware/software, it sucks. And you should always test with different types of music.
Also, small sample size (16), only 1 song in 2 versions, presumably always in the same order, on hardware that has nothing to do with what everybody uses (does that lessen or worsen compression characteristics ?), no control group (wanna bet that with 2 exact same versions, song A or song B consistently comes out on top ? Coke and Pepsi worked that one out long ago). No indication how responses were collected (group ? interviewer ? biased ?).
made me chuckle. amateurs.
The Cloud - because you don't care if your apps and data are up in the air.
As long as they're not passing around the MP3s, or never want to upgrade their stereo system 128 is fine. If you ever want to do either of those, 128kbps MP3s will not be good enough.
Give me Classic Slashdot or give me death!
I was all excited that my new car came with a satellite radio reciever, then bitterly disappointed with the sound quality and didn't subscribe. I'm considering replacing it with an HD reciever, once I hear one to find out what "HD" really means. Satellite radio is utter crap for sound quality.
CD quality mp3 is 320kbps.I can understand not being able to tell 48kbps from 160kbps (especially when a different codec is used for each, the quality of the codec and the configuration of it is key). It's hard to tell the difference between crap and sh*t. The test is only meaningful as a bitrate test if the same codec and encoding settings are used. Otherwise it's apples to oranges. The bitrate isn't nearly as important as how it's encoded unless both streams are done exactly the same way (except for bitrate).
This test smacks of Apple fanboism. Do a real bitrate test using the same codec and settings (outside of bitrate) and I guarantee you'll get better listener accuracy.
Why on earth would you do a bitrate test with two different codecs unless the test was really marketing propaganda for one of the codecs? /filed in the Apple marketing bullshit drawer
This is a codec test, not a bitrate test. As a "can a user tell the difference between these bitrates" test, the results are completely worthless. It's more like a "AAC rulez! look a 48k AAC stream sounds as good as a 160kbps Ogg stream!" /barf
Don't kid yourself. It's the size of the regexp AND how you use it that counts.
Then the summary is misleading in presenting this as a comparison of bitrates. The article is really comparing the audio quality of the two services.
Based on the article, the testing seems to have very little in the way of meaningful results.
A single instance of a single song with two different encoders given to listeners who hear "more bass" as a quality where the results were so close to split (two people shy of 50/50).
To gather meaningful data: songs must be switched quickly: you should go through a variety of materials (it's worth noting that some compressions have more trouble with certain types of sounds than others), and (ideally) there should be a reference from which to work.
The goal of compression, in theory at least, is to maintain meaningful fedility. Yes, that means that "the part we notice most" is most important: but that's no excuse for causeing "a pleasent error" better than "correct reproduction".
Of course, I've never tested these encoders. It's possible that the lower bitrate encoder did a better job.
There's no news here. The HE AAC codec (called AAC+ in the Coding Technologies implementation, and now called Dolby Pulse after Dolby's acquisition) is a highly advanced spectral band replication codec, and can be pretty darn transparent down to around 48 Kbps. That there was about a 2:1 preference for the high bitrate Ogg in a highly nonscientific small sample size test like this is a yawner. http://en.wikipedia.org/wiki/HE_AAC
My video compression blog
When listening to audio in a car, there is a problem with background noise. The sound pressure level in an average car at 70 mph will be in the region of 70 dB. The threshold of pain is at a sound pressure level of about 120 dB. So in a car there is a maximum dynamic range of no more than 50 dB and in reality far less.
As CDs spread from homes to cars, CDs which made use of the full dynamic range available in the technology were difficult to hear clearly (and painlessly) as either the loud passages were too loud for most passengers or the quiet passages would be inaudible. So they adapted to the new common audio setup and began mastering CDs with a smaller dynamic range. Since in-car listeners are a major audience, many FM stations also gean using automatic dynamic range compression technology. Not all radio stations do -- BBC Radio 3 (in deference to its listeners' preference for better audio setups) does not use it in the evening.
As much as I enjoy my iPods, one of my greatest fears related to it was that an increasing dominance of portable, digital audio players with earbuds would ignite something akin to "Loudness Wars -- Round Two" in terms of selling audio pre-recorded at lower bit rates.
I paid to get my TV ISF calibrated. It looks amazing. But if you brought it inside a Best Buy and sat it next to their other TVs your average Joe would think it looks like crap.
The TV manufacturers increase the amount of blue to make things appear brighter. People's faces turn green so they up the amount of Red. Then they over-sharpen which introduces artifacts and over-contrast which creates banding.
Encoding audio in a lossy format no doubtingly does the same thing. They make sure the music still "pop"s to the point where it is exaggerated causing the music to "sound" better.
The people who say that 48Kbps sounds better than 160 would probably say the same thing compared to the original.
(although not as low as 46kbps) and reached the same conclusion. Most people vastly overestimate their ability to distinguish tracks encoded at different bitrates. And I've seen study after study that backs this up. This includes self-professed audiophiles, the original authors of particular tracks of music, and so forth.
This is true. Mostly.
On most material, you cannot hear the difference; however, every once in a while (though rarely), there will be a song which not even a 320 kbps mp3 can encode properly[1], and there'll be distortion on cymbals or applause or a snare drum or a weird synth. If you don't know how the original is supposed to sound, you won't notice anything strange, but if you do and if you can recognize that "mp3 sizzle", which is far easier for those of us who have been dealing with mp3s ever since the days of early FhG and Xing at 112/128 kbps, the track or even the album is ruined entirely. And I do mean entirely. There's no point in listening to it anymore because you know it's flawed and you'll just be spending time trying to listen to the sound waves instead of music.
For example, today I listened to a straight FLAC encode of an Armin van Buuren live set / album from Ibiza 2008 (or something) and several tracks were totally messed up - the "sizzle" was perfectly clear, as AvB either used mp3s directly, or burned mp3s to CDs.
[1] I remember ripping a Cranberries album in the late 90s, when lossless container formats didn't exist and the best encoder was the original Fraunhofer, which couldn't deal with Dolores and her band at all, even with 256 kbps. The situation has since improved greatly (though it's not anywhere near perfect), but we have the opposite circumstances: those were the days of 1-2 GB hard drives and .wav lossless was just out of the question, whereas today we have 1-2 TB drives and 300-500 MB for a FLAC is a drop in the storage ocean. I don't care about mp3s at all anymore. I buy a CD, I rip it and encode to FLAC - what the hell else am I going to spend disk space on?
Since the 16 subjects were asked "which sounds better" and were not given an alternative "there's no difference" then it's actually possible that 12 of the 16 thought there was no difference, and so they randomly picked A or B. And 6 picked A.
So it's possible that only 25% could tell the difference and selected the higher bit rate.
Great study. Very Scientific.
>> They simply don't have the mental acuity to care.
In other words, the ipod users. The ones who just think how cool it looks, not how good it sounds.
e.g. iriver kicks ipod ass any time when it comes to sound quality. So does Sony (I hate them too, but this is my experience).
The human memory for details seems to be measured in seconds. One big issue in blind-testing speakers is to make sure that you can switch almost instantly between two volume-matched pairs.
Playing to the end of a song then listening to it again is not going to yield the best objective results; although it does say something subjective pretty strongly.
This 'test' seems rather lacking. It doesn't note if the AAC is HE or LC. That can have a very big impact on quality as HE takes more processing power but delivers much better quality at low bitrates. Each codec would also have it's quirks and 'tricks' that establish it's strong and weak points. Some people will simply like one aspect of a codecs compression methods over another, whether that pertains to filterout out high frequency, chopping out repetitive or white noise that is typically not heard, or whatnot.
The fact that they also only tested 16 people should tell the rest of the story. It's not even remotely a good sampling of users and considering the source, it probably consists of users who are 'in the know' about compression techniques and what to listen for.
I would be very interested in a larger study with a random sampling of the users of these two services, with a much larger study group to see what it shows.
Variable bitrate produces a file where the average is the given bitrate, but it changes the bitrate at any given point based on need. So for example, any parts of the song that are silence will be encoded at very low bitrates, and complicated parts will be encoded at higher bitrates. If you encode with a fixed bitrate, you're wasting space encoding simple parts and not encoding the complicated parts as high as you should.
So for your second question we should turn it around into "is 128 kbps VBR better than 192 kbps fixed?", and the answer is that it depends on the song, but you probably wouldn't be able to tell the difference anyway. A more interesting question is "is 128 kbps VBR better than 128 kpbs fixed" and the answer is yes, always.
Of course, unless you're some kind of audio god (and I mean, completely inhuman), you probably can't guess what bitrate you need anyway, which is where quality-based encoders come in. If you want to encode a file in ogg vorbis, you just say what quality you want, and it figures out what the average bitrate needs to be to encode it.
CDs are encoded at 44.1 kHz, 16-bit stereo. If you do a little math, this comes out to 1.4 Mbps, meaning that to get you audio down to such a low bit rate you need to eliminate 29 out of every 30 bits. If anybody out there is incapable of hearing the difference, they need to go get a hearing test right away, as that level of compression is EXTREMELY destructive to the quality of the audio.
To the haters: You can't win. If you mod me down, I shall become more powerful than you could possibly imagine
You blame the people, I blame the editors.
There is no study here, from TFA (which itself is barely longer than TFS), sixteen people were asked to state which song clip they thought sounded better. I'm surprised the results were better than 50/50. From TFA, all listeners could *tell* a difference and the report was on which one they *prefer*
Really, there's nothing to see here.
+1 Disagree
Hertz and bits are independent. You can sample something at 150kHz but then compress it to 14kbps. You can likewise, sample something at 14kHz and then encode it at 1.5Mbps.
There was absolutely no (none, zero) discussion on the sampling rate and I was correcting that.
And the equipment they use. The chain is never stronger than its weakest link. There is no point in testing say 24bit@96kHz uncompressed if the audio equipment cannot deliver it.
If the only way you can tell the difference is in a "controlled environment" by someone with "listening experience" (whatever that means0, then the difference is completely irrelevant, both to the average person and to the marketplace.