Slashdot Mirror
Music Listeners Test 128kbps vs. 256kbps AAC
notthatwillsmith writes "Maximum PC did double-blind testing with ten listeners in order to determine whether or not normal people could discern the quality difference between the new 256kbps iTunes Plus files and the old, DRM-laden 128kbps tracks. But wait, there's more! To add an extra twist, they also tested Apple's default iPod earbuds vs. an expensive pair of Shure buds to see how much of an impact earbud quality had on the detection rate."
Apple's iTunes store--in partnership with EMI--is now hawking DRM-free music at twice the bit rate of its standard fare (256Kb/s vs. 128Kb/s) and charging a $0.30-per-track premium for it. We're all for DRM-free music, but 256Kb/s still seems like a pretty low bit rate--especially when you're using a lossy codec.
So we decided to test a random sample of our colleagues to see if they could detect any audible difference between a song ripped from a CD and encoded in Apple's lossy AAC format at 128K/s, and the same song ripped and encoded in lossy AAC at 256Kb/s.
Our 10 test subjects range in age from 23 to 56. Seven of the 10 are male. Eight are editors by trade; two art directors. Four participants have musical backgrounds (defined as having played an instrument and/or sung in a band). We asked each participant to provide us with a CD containing a track they considered themselves to be intimately familiar with. We used iTunes to rip the tracks and copied them to a fifth-generation 30GB iPod. We were hoping participants would choose a diverse collection of music, and they did: Classical, jazz, electronica, alternative, straight-ahead rock, and pop were all represented; in fact country was the only style not in the mix. (See the chart at the end of the story for details.)
We hypothesized that no one would be able to discern the difference using the inexpensive earbuds (MSRP: $29) that Apple provides with its product, so we also acquired a set of high-end Shure SE420 earphones (MSRP: $400). We were confident that the better phones would make the task much easier, since they would reveal more flaws in the songs encoded at lower bit rates.
METHODOLOGY
We asked each participant to listen with the Apple buds first and to choose between Track A, Track B, or to express no preference. We then tested using the SE420's and asked the participant to choose between Track C, Track D, or to express no preference. The tests were administered double-blind, meaning that neither the test subject nor the person conducting the test knew which tracks were encoded at which bit rates.
The biggest surprise of the test actually disproved our hypothesis: Eight of the 10 participants expressed a preference for the higher-bit rate songs while listening with the Apple buds, compared to only six who picked the higher-quality track while listening to the Shure's. Several of the test subjects went so far as to tell they felt more confident expressing a preference while listening to the Apple buds. We theorize that the Apple buds were less capable of reproducing high frequencies and that this weakness amplified the listeners' perception of aliasing in the compressed audio signal. But that's just a theory.
LEAVE IT TO THE OLD FOGEYS
Age also factored differently than we expected. Our hearing tends to deteriorate as we get older, but all three of our subjects who are over 40 years old (and the oldest listener in the next-oldest bracket) correctly identified the higher bit-rate tracks using both the Apple and the Shure earphones. Three of the four subjects aged between 31 and 40 correctly identified the higher bit-rate tracks with the Apple earbuds, but only two were successful with the Shures. Two of three under-30 subjects picked the higher-quality tracks with the Apples, but only one of them made the right choice with the Shures. All four musicians picked the higher-quality track while listening to the Apples, and three of the four were correct with the Shures.
Despite being less able to detect the bit rate of the songs while listening to the Shure SE420 earphones, eight of 10 subjects expressed a preference for them over the Apple buds. Several people commented on the Shure's ability to block extraneous noise. While listening to the SE420s, one person remarked "Wow, I'd forgotten that wood-block sound was even in this song." Another said "The difference between the Shure earphones and the Apple earbuds was more significant than the difference between the song encoded at 128Kb/s and the one recorded
8/10 Picked High Bit Rate with Apple Headphones
6/10 Picked High Bit Rate with Shure Headphones
100% certainty that 10 people sample-set is too little for a Yes-No experiement.
that article doesnt provide enough data to make any conclusions.
maybe they should go back to statistics 101
comment directly in my journal
I agree with your statement that audiophiles don't use "professional" equipment, but I disagree with your statement that studio monitors will give you the sound that the recording engineer intended. This is because, as you imply, there is a distinct difference between accurate speakers and good-sounding speakers, and recording studios use accurate speakers, while consumers, even audiophiles, are better off with good-sounding speakers.
If you working in a recording studio, you want accuracy at all costs. You must hear everything distinctly, because you need to make important decisions based on what you hear. If "it sounds great" is all you are getting from your speakers, you won't make those tough decisions (more cymbals, different reverb, more compression on the vocals, or whatever.) You'll just leave it alone and it won't be as good as it could be. However, those extremely accurate speakers that are perfect for recording studio use are NOT pleasant for casual listening. Everything is too crisp and sharp, and they will tend to make you want a break from all that detail.
When I'm working on a mix in the studio, I want everything in very crisp detail so I can make judgments; when I'm listening to the final product, I want the music to "hang together" and present itself to me as a coherent whole. There are other differences between studio monitors and "normal" speakers (for example, consistency of frequency response) but this relatively subjective factor of detailed sound vs. coherent sound is one of the more important ones I have experienced.
The recording engineer did not intend for you to listen to the music on studio monitors. Studio monitors are a tool with a specific use, and that use is not everyday listening. The attributes of a good studio monitor just don't match up with the attributes of a good audiophile speaker. This is why audiophiles buy certain kinds of speakers, and recording engineers buy other kinds. I've been lucky enough to own both kinds of speakers, and I've tried using them for the wrong purpose with less-than-stellar results. Mixes made on good-sounding speakers are inconsistent on other speakers, and music played through accurate speakers isn't as pleasant to the ear.
My truck is like a series of tubes.
Hydrogen audio forums provide a lot of very good information including well designed double blind comparisons between codecs and bits rates. See this page for details and the links to other testing sites. http://wiki.hydrogenaudio.org/index.php?title=List ening_Tests
All in all an excellent resource for any serious listener.
Did they randomize which track was played first? If not they can throw out the whole thing. It's a well-established phenomenon than given a choice between two roughly equivalent items people will overwhelmingly choose the second one. That's how, for instance, Pepsi always did so well in the "Pepsi Challenge."
First, to those who made comments about 128k encoding, you may be thinking of mp3. (Or maybe not, who knows...) From what I've heard, AAC, Vorbis, and AC3 all sound better than mp3 at similar bitrates.
Second, I remember there was a comment on Slashdot awhile back, before they actually came out with these, and I want to confirm... Apparently, CDs are recorded at a certain physical bitrate/frequency, and there are digital masters which are at a higher rate... it's late, so I'm not entirely coherent, but think of it as somewhat equivalent to resolution of a DVD (quality of video is proportional to resolution (HD vs normal) and bitrate). The point was that 256k may actually sound better than a CD, since it comes from a better source than a CD.
If so, this whole test is BS, since they did not do a comparison of CD vs 128k (either iTunes-DRM'd or custom-ripped) vs 256k (un-DRM'd, from the iTunes store). Specifically, I'd want to hear 256k vs CD. But at the same time, I don't know if any iPod, or specifically the one they are using, would be able to handle the higher resolution. If not, you'd have to specifically check your soundcard, too.
And finally, again vaguely remembering this from a Slashdot comment (so correct me if I'm wrong), but there was some comment about "The 30c may seem small, but imagine buying a whole album at these prices..." And I seem to remember that a full album is always $9.99. Still high compared to, for example, the minimum you'd pay for a FLAC-encoded album at MagnaTune, but if you're buying a whole album (and if that's accurate), you may as well just opt for un-DRM'd -- especially if it sounds better than a CD (which would probably cost more anyway.)
But then, of course, I'd like to hear a much bigger study, with more rigorous controls in place. 10 people is just not enough, no matter how you set it up.
And personally, if I had any money to spend on music, I'd be buying un-DRM'd stuff. But probably not from iTunes -- not till there's a web interface (or at least an API) that doesn't require me to download their client software. After all, if I'm buying a single file, the point of the client software is to implement the DRM, and if I buy the un-DRM'd version... Not that it shouldn't also work in the iTunes client, but it'd be nice for it to work natively in AmaroK, or just in a browser.
Don't thank God, thank a doctor!
"And as much as we dislike DRM, we just don't think DRM-free tracks alone are worth paying an extra 30 cents a track for.. It would be crazy to pay that premium if you're going to buy the entire album. We'd be more excited if Apple increased the bit rate even further, or--even better--if they used a lossless format."
First off, I've yet to see a lossless formula that WORKS. And by works I mean is easily convertible into mp3/aac so I can use it on a portable player I already own, able to be used. I've seen APE and FLAC, both are too much hassle, and the APE files I got were in japanese. Here's a little fact, Ape doesn't necessarily know how to correctly encode Japanese into ID3, end result? Buffer overflow, bad data. Oh and if they work? They are larger than mp3s and AAC. Lossless codec means all the data has to remain, trust me, that's not a good thing when coupled with all the other little hassles it has.
Second. It'd be crazy to spend 99 cents just to license your files so that you can only use as Apple approves. Paying money to crack the music so I can use it as I want is illegal according to them so why am I paying the money to get locked into their plan. However DRM free music is easily worth 1 dollar and 30 cents because it's mine (It AAC but I can live with that). I don't have to ask permission to use it in another player, I don't have to ask permission to convert it to a data format I choose. Personally I'm fine with 192 for most recordings, I'm not an audiophile, I'm just a listener. If you want the highest grade data, or are an audiophile you'll be buying CDs or fully lossless data, you're not going to fuck with iTunes anyways.
Btw their other idea is to get rid of the apple iBuds and get quality recievers. Hint, This is what got the less interchangable results? I don't exactly see why getting a "higher quality" headset would be desirable if it creates less of a difference instead of more of a difference between two bit rates. Higher quality means I should hear everything. If you are asking people "can you hear the difference" they already should be listening as hard as they can. The theory they try to explain it with doesn't make much sense. They are telling us 30 cents doesn't make a difference but they are trying to sell us on dropping 400 bucks on noise reducing headsets you can get for around 100 if you're clever. Hell they are EARBUDS!!! So far I've notice two things about earbuds. They are uncomfortable, and they are worthless compared to my headphones. If you're talking about noise reducing earbuds just be smart buy a good set of headphones.
Overall a throw away article, I'm still only going to buy DRM less music (I expect you out there to do the same, I'm assuming 30 cents won't kill you, but that's your choice) and hope to soon if Apple ever put the DRMless music out, and had the music I listen to (so far not really). I'm assuming you all are STILL buying music like you are going to. The only mind's this article changes is the cavemen hiding under the rock who still scream "ahhh cds bad", and he's still trying to figure out our compooters, so showing him the internet might not be smart just yet.
AAC is supported on everything except cheapo Taiwan guys having MS sponsor them (with Wmedia).
All my files on my Nokia 9300 Symbian phone are AAC even including ringtones embedded in device ROM. The other, "real" phone of mine, not anything close to smart (SE K700i) has everything in aac too. In fact thanks to AAC's better compression, I can use its 46mb flash memory for music.
"(Advanced Audio Coding) An audio compression technology that is part of the MPEG-2 and MPEG-4 standards."
http://www.answers.com/aac
As long as MPEG committee builds the standards, AAC is a standard. All players can't support AAC because the companies making them can't afford or won't get the AAC license. Not because it is only an Apple standard. Zune has AAC because it is iPod competitor and MS also have money to afford that extra format.
Whole confusion comes from the container format of mpeg-4 is built on the quicktime standard I guess. Now there are people who thinks mpeg4/h264 is Apple only format while their new HD-Satellite receiver probably has it in a chip.
The propetioary standards are the ones you mentioned. WMA (if we ignore their propaganda) and ATRAC3 which Sony really missed a huge chance by not opening it.
Classical music usually has a wide dynamic range whilst most of the rest doesn't. The audio engineer working on a pop track runs everything through an audio volume-level compressor, bringing every sound to more or less the same volume level. In classical music it is quite normal to play certain things at the level of a whisper.
This means that most of popular music never uses the digital bits representing these low-volume whispers but confines itself to loud shouts and blaring synths, so a lot of the 'bandwidth' on a CD is wasted because of it. Classical music on the other hand uses most of the available bandwidth thanks to the sane use of audio level compressors. When this wideband signal is to have its data compressed then it requires a lot more storage space than the popular music would.
All rites reversed 2010
They are talking bitrate not encoding. Sampling records the amplitude at the sampling point. Quiet stuff (especially under louder stuff) becomes muddy as the distance between sampling points increases. Consider a violin section playing a high note very softly with tremolo. You get really fast changes that change the overall wave very slightly. Looking at the wave on an oscilloscope it's very clear, and hearing it live you feel and hear the effect.
When recorded digitally (in mono) if the samples are too infrequent the net effect of the string's high-pitch and quiet trem on the wave will end up sounding like there were no strings playing and the orchestra was out of tune, or the trem will sound like a sustained note.
If we look at the output of the sampling on an oscilloscope, the result will have the same basic shape, and if you stood far away with the two oscilloscope pictures next to each other you'd say they were the same. Increasing the bitrate will increase the "focus" (for lack of a better word) of sampled oscilloscope image.