Music Listeners Test 128kbps vs. 256kbps AAC
notthatwillsmith writes "Maximum PC did double-blind testing with ten listeners in order to determine whether or not normal people could discern the quality difference between the new 256kbps iTunes Plus files and the old, DRM-laden 128kbps tracks. But wait, there's more! To add an extra twist, they also tested Apple's default iPod earbuds vs. an expensive pair of Shure buds to see how much of an impact earbud quality had on the detection rate."
Apple's iTunes store--in partnership with EMI--is now hawking DRM-free music at twice the bit rate of its standard fare (256Kb/s vs. 128Kb/s) and charging a $0.30-per-track premium for it. We're all for DRM-free music, but 256Kb/s still seems like a pretty low bit rate--especially when you're using a lossy codec.
So we decided to test a random sample of our colleagues to see if they could detect any audible difference between a song ripped from a CD and encoded in Apple's lossy AAC format at 128K/s, and the same song ripped and encoded in lossy AAC at 256Kb/s.
Our 10 test subjects range in age from 23 to 56. Seven of the 10 are male. Eight are editors by trade; two art directors. Four participants have musical backgrounds (defined as having played an instrument and/or sung in a band). We asked each participant to provide us with a CD containing a track they considered themselves to be intimately familiar with. We used iTunes to rip the tracks and copied them to a fifth-generation 30GB iPod. We were hoping participants would choose a diverse collection of music, and they did: Classical, jazz, electronica, alternative, straight-ahead rock, and pop were all represented; in fact country was the only style not in the mix. (See the chart at the end of the story for details.)
We hypothesized that no one would be able to discern the difference using the inexpensive earbuds (MSRP: $29) that Apple provides with its product, so we also acquired a set of high-end Shure SE420 earphones (MSRP: $400). We were confident that the better phones would make the task much easier, since they would reveal more flaws in the songs encoded at lower bit rates.
METHODOLOGY
We asked each participant to listen with the Apple buds first and to choose between Track A, Track B, or to express no preference. We then tested using the SE420's and asked the participant to choose between Track C, Track D, or to express no preference. The tests were administered double-blind, meaning that neither the test subject nor the person conducting the test knew which tracks were encoded at which bit rates.
The biggest surprise of the test actually disproved our hypothesis: Eight of the 10 participants expressed a preference for the higher-bit rate songs while listening with the Apple buds, compared to only six who picked the higher-quality track while listening to the Shure's. Several of the test subjects went so far as to tell they felt more confident expressing a preference while listening to the Apple buds. We theorize that the Apple buds were less capable of reproducing high frequencies and that this weakness amplified the listeners' perception of aliasing in the compressed audio signal. But that's just a theory.
LEAVE IT TO THE OLD FOGEYS
Age also factored differently than we expected. Our hearing tends to deteriorate as we get older, but all three of our subjects who are over 40 years old (and the oldest listener in the next-oldest bracket) correctly identified the higher bit-rate tracks using both the Apple and the Shure earphones. Three of the four subjects aged between 31 and 40 correctly identified the higher bit-rate tracks with the Apple earbuds, but only two were successful with the Shures. Two of three under-30 subjects picked the higher-quality tracks with the Apples, but only one of them made the right choice with the Shures. All four musicians picked the higher-quality track while listening to the Apples, and three of the four were correct with the Shures.
Despite being less able to detect the bit rate of the songs while listening to the Shure SE420 earphones, eight of 10 subjects expressed a preference for them over the Apple buds. Several people commented on the Shure's ability to block extraneous noise. While listening to the SE420s, one person remarked "Wow, I'd forgotten that wood-block sound was even in this song." Another said "The difference between the Shure earphones and the Apple earbuds was more significant than the difference between the song encoded at 128Kb/s and the one recorded
8/10 Picked High Bit Rate with Apple Headphones
6/10 Picked High Bit Rate with Shure Headphones
100% certainty that 10 people sample-set is too little for a Yes-No experiement.
that article doesnt provide enough data to make any conclusions.
maybe they should go back to statistics 101
comment directly in my journal
Clearly these test are inadequate, or at least they haven't disclosed enough information on the testing conditions. As any true audiophile knows, headphone performance is strongly affected by atmospheric conditions; I'll bet that if they had bothered to maintain proper water vapor saturation levels in the test facility the complete the inadequacy of the ear buds would have been obvious to everyone involved, because sensory receptors (hair cells) in the human ear only achieve full sensitively under controlled conditions.
No doubt they also failed to account for magnetic field alignment; the flaws of low bit rate reproductions are much easier to perceive when the listener is not aligned with Earth's natural axial vectors. The solenoidal force lines ruin the high band pass attenuation of any digital audio and will make both low and high bit rate reproductions equally poor, so naturally there wasn't a strong correlation among the test subjects.
Idiots.
</sarcasm>
Lurking at the bottom of the gravity well, getting old
Judging by the comments from the six people who actually got to read the article I'm glad it got slashdotted before I wasted my time on it.
Three Squirrels
Yeah, well I used to have a gf who claimed to be a Scientologist, and she gave over $40k to the church. She states that some alien is responsible for blowing up volcanos that created humans, but you know what... the bitch is just wrong.
w00t
The big difference that the 256 Kb/s + DRM-free option makes for me is that now I'll buy albums from iTunes Store. Previously I would use iTunes to buy one to three tracks if there was some artist I liked but didn't want a whole album from. But usually I order the CDs online for $8 to $14, rip them to AAC at 192 Kb/s, and put the disc away to collect dust on my overflowing CD rack. Now I can get higher quality cheaper and faster.
Yes, ideally I would rip all my music to a lossless format. And ideally everything would be available on SACD at 2822 KHz rather than 44.1 KHz CDs. But that's just not practical with my 500+ album collection. It'd fill up my laptop's hard drive real quick and allow me to put only a fraction onto my iPod.
I'm also disappointed that the article only tested the tracks on iPods with earbuds. Most of my listening is on a decent stereo system fed from my laptop. Ripping is about convenience, not portability. I only use my iPod when riding the Metro or an airplane. With all the outside noise the bitrate doesn't matter.
And being DRM-free isn't just a matter of idealism. I get frustrated when I go to burn an MP3 CD for my car and discover that one of the tracks I selected is DRMed. Sure there are ways to get around it, but it's just not worth the bother.
AlpineR
I agree with your statement that audiophiles don't use "professional" equipment, but I disagree with your statement that studio monitors will give you the sound that the recording engineer intended. This is because, as you imply, there is a distinct difference between accurate speakers and good-sounding speakers, and recording studios use accurate speakers, while consumers, even audiophiles, are better off with good-sounding speakers.
If you working in a recording studio, you want accuracy at all costs. You must hear everything distinctly, because you need to make important decisions based on what you hear. If "it sounds great" is all you are getting from your speakers, you won't make those tough decisions (more cymbals, different reverb, more compression on the vocals, or whatever.) You'll just leave it alone and it won't be as good as it could be. However, those extremely accurate speakers that are perfect for recording studio use are NOT pleasant for casual listening. Everything is too crisp and sharp, and they will tend to make you want a break from all that detail.
When I'm working on a mix in the studio, I want everything in very crisp detail so I can make judgments; when I'm listening to the final product, I want the music to "hang together" and present itself to me as a coherent whole. There are other differences between studio monitors and "normal" speakers (for example, consistency of frequency response) but this relatively subjective factor of detailed sound vs. coherent sound is one of the more important ones I have experienced.
The recording engineer did not intend for you to listen to the music on studio monitors. Studio monitors are a tool with a specific use, and that use is not everyday listening. The attributes of a good studio monitor just don't match up with the attributes of a good audiophile speaker. This is why audiophiles buy certain kinds of speakers, and recording engineers buy other kinds. I've been lucky enough to own both kinds of speakers, and I've tried using them for the wrong purpose with less-than-stellar results. Mixes made on good-sounding speakers are inconsistent on other speakers, and music played through accurate speakers isn't as pleasant to the ear.
My truck is like a series of tubes.
Both kinds of music were missing: They had neither Country nor Western music in the test.
What is America coming to?
Excuse me, but please get off my Pennisetum Clandestinum, eh!
Hydrogen audio forums provide a lot of very good information including well designed double blind comparisons between codecs and bits rates. See this page for details and the links to other testing sites. http://wiki.hydrogenaudio.org/index.php?title=List ening_Tests
All in all an excellent resource for any serious listener.
Actually, I notice a huge difference between 128K and 192K when listening to classical music. For music that doesn't contain the brashness of percussion or brass instruments, the distortion at lower encoding levels is fairly good; however, brass instruments (including brass cymbals) in particular are unbearably distorted when 128K is used but come across rather cleanly when >192K is used. I've finally accepted that a variable rate between 224K and 320K is where I need to encode my tracks in order to make them as close to the original CDs as I can tolerate without using the actual CDs.
The Overrated mod is for reversing inappropriate, positive mods, not for voicing disagreement with a post.
At 192k I start not to be less annoyed.
I don't think that means what you think it means.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
So, you would be happy with my codec, which replaces hip-hop with Bach played on original instruments? That sounds best to me...
The Red Book standard unfortunately went with 44.1 (for some esoteric reason having to do with syncing with an analog video standard or something back in the 80s).
;-).
Huh, you're right...
http://www.cs.columbia.edu/~hgs/audio/44.1.html
I always assumed that 44.1kHz was chosen because they took the necessary (Nyquist) sample rate to be able to record up to 20kHz (40kHz), and added a bit for good measure. There's always been that rumor that the time length of a CD was chosen to be able to fit Beethoven's Ninth Symphony, so I always figured they knew they wanted 16 bit, and a length of about 74 minutes, and just picked the >40kHz sampling rate that would get them there with that fancy new "CD" technology that was being developed. I'm happy to know that we're all using 44.1kHz for an even stupider reason
ZuluPad, the wiki notepad on crack