Slashdot Mirror
Music Listeners Test 128kbps vs. 256kbps AAC
notthatwillsmith writes "Maximum PC did double-blind testing with ten listeners in order to determine whether or not normal people could discern the quality difference between the new 256kbps iTunes Plus files and the old, DRM-laden 128kbps tracks. But wait, there's more! To add an extra twist, they also tested Apple's default iPod earbuds vs. an expensive pair of Shure buds to see how much of an impact earbud quality had on the detection rate."
Apple's iTunes store--in partnership with EMI--is now hawking DRM-free music at twice the bit rate of its standard fare (256Kb/s vs. 128Kb/s) and charging a $0.30-per-track premium for it. We're all for DRM-free music, but 256Kb/s still seems like a pretty low bit rate--especially when you're using a lossy codec.
So we decided to test a random sample of our colleagues to see if they could detect any audible difference between a song ripped from a CD and encoded in Apple's lossy AAC format at 128K/s, and the same song ripped and encoded in lossy AAC at 256Kb/s.
Our 10 test subjects range in age from 23 to 56. Seven of the 10 are male. Eight are editors by trade; two art directors. Four participants have musical backgrounds (defined as having played an instrument and/or sung in a band). We asked each participant to provide us with a CD containing a track they considered themselves to be intimately familiar with. We used iTunes to rip the tracks and copied them to a fifth-generation 30GB iPod. We were hoping participants would choose a diverse collection of music, and they did: Classical, jazz, electronica, alternative, straight-ahead rock, and pop were all represented; in fact country was the only style not in the mix. (See the chart at the end of the story for details.)
We hypothesized that no one would be able to discern the difference using the inexpensive earbuds (MSRP: $29) that Apple provides with its product, so we also acquired a set of high-end Shure SE420 earphones (MSRP: $400). We were confident that the better phones would make the task much easier, since they would reveal more flaws in the songs encoded at lower bit rates.
METHODOLOGY
We asked each participant to listen with the Apple buds first and to choose between Track A, Track B, or to express no preference. We then tested using the SE420's and asked the participant to choose between Track C, Track D, or to express no preference. The tests were administered double-blind, meaning that neither the test subject nor the person conducting the test knew which tracks were encoded at which bit rates.
The biggest surprise of the test actually disproved our hypothesis: Eight of the 10 participants expressed a preference for the higher-bit rate songs while listening with the Apple buds, compared to only six who picked the higher-quality track while listening to the Shure's. Several of the test subjects went so far as to tell they felt more confident expressing a preference while listening to the Apple buds. We theorize that the Apple buds were less capable of reproducing high frequencies and that this weakness amplified the listeners' perception of aliasing in the compressed audio signal. But that's just a theory.
LEAVE IT TO THE OLD FOGEYS
Age also factored differently than we expected. Our hearing tends to deteriorate as we get older, but all three of our subjects who are over 40 years old (and the oldest listener in the next-oldest bracket) correctly identified the higher bit-rate tracks using both the Apple and the Shure earphones. Three of the four subjects aged between 31 and 40 correctly identified the higher bit-rate tracks with the Apple earbuds, but only two were successful with the Shures. Two of three under-30 subjects picked the higher-quality tracks with the Apples, but only one of them made the right choice with the Shures. All four musicians picked the higher-quality track while listening to the Apples, and three of the four were correct with the Shures.
Despite being less able to detect the bit rate of the songs while listening to the Shure SE420 earphones, eight of 10 subjects expressed a preference for them over the Apple buds. Several people commented on the Shure's ability to block extraneous noise. While listening to the SE420s, one person remarked "Wow, I'd forgotten that wood-block sound was even in this song." Another said "The difference between the Shure earphones and the Apple earbuds was more significant than the difference between the song encoded at 128Kb/s and the one recorded
8/10 Picked High Bit Rate with Apple Headphones
6/10 Picked High Bit Rate with Shure Headphones
100% certainty that 10 people sample-set is too little for a Yes-No experiement.
Test confirms the generally known (but debatable) points:
1. Not many can detect the improvement of higher kbps
2. Expensive earbuds are way better than the default ones.
3. 128kbps AAC isn't all that bad.
Most of the stuff on
that article doesnt provide enough data to make any conclusions.
maybe they should go back to statistics 101
comment directly in my journal
"Eight of the 10 participants expressed a preference for the higher-bit rate songs while listening with the Apple buds, compared to only six who picked the higher-quality track while listening to the Shure's."
I don't buy this. I have a friend who claims to be an audiophile - and he is - with sound equipment worth well over $40,000. He states that the more expensive and professional your gear is, the easier it is to spot low quality music.
So the article contradicts with his statement, and I have to agree with him on this one. Logically speaking, professional speakers should produce results far closer to the source than the ones that aren't.
Full Tilt
Clearly these test are inadequate, or at least they haven't disclosed enough information on the testing conditions. As any true audiophile knows, headphone performance is strongly affected by atmospheric conditions; I'll bet that if they had bothered to maintain proper water vapor saturation levels in the test facility the complete the inadequacy of the ear buds would have been obvious to everyone involved, because sensory receptors (hair cells) in the human ear only achieve full sensitively under controlled conditions.
No doubt they also failed to account for magnetic field alignment; the flaws of low bit rate reproductions are much easier to perceive when the listener is not aligned with Earth's natural axial vectors. The solenoidal force lines ruin the high band pass attenuation of any digital audio and will make both low and high bit rate reproductions equally poor, so naturally there wasn't a strong correlation among the test subjects.
Idiots.
</sarcasm>
Lurking at the bottom of the gravity well, getting old
Judging by the comments from the six people who actually got to read the article I'm glad it got slashdotted before I wasted my time on it.
Three Squirrels
Despite what Apple charges for a set of its replacement buds, the earphones that come with 90 percent of the digital media players on the market are throw-away items--they're only in the box so you'll have something to listen to when you bring the player home.
I'm a musician. I've recorded and released an album (sorry for the shameless plug but it's only to put my post in context - honest). I own expensive studio earphones, have experience mixing and mastering etc.
I don't own a 5th generation iPod but I do own an iPod Shuffle that has since stopped playing MP3s. It still works as a storage device and I still have the headphones. I kept on to the headphones because I prefer them over all other ear buds I have. They don't beat the studio headphones, but I would not consider them "throw aways". I found they're pretty good quality and I began using them with all of my portable devices. I would generally agree that most ear buds that come with cd players and probably many other mp3 players are of relatively low quality, but I was very impressed with the ones that came with the iPod Shuffle. I will never throw them away.
Me, personally, what I find unsatisfying about compressed music is that the treble is the first thing to go, and even at high bit rates AAC and MP3 each seem to just make all cymbals, brushes, triangles, and synthetic tones in the high registers sound equally like white noise.
I found a tonality frequency setting in LAME that seemed to cure this problem, but neither iTunes nor ITMS seems to let you adjust or purchase based on this issue.
Perhaps not everyone is sensitive to this, but maybe there are other settings or aspects of compression that other people are sensitive to which I am not...leading one to the possible conclusion that compressed music might be made better by personalizing each rip to the hearing response of the listener rather than compromising on an average human hearing model.
w00t
Hmmm I took the pepsi challenge and chose gif.
The unexpected age results (that older people were better at telling the difference for the bitrates) may well be a consequence of music choice. Each subject picked their own music, and it is very clear that these quality differences are more noticable in some types of music than in others. The first time I played an iTunes purchased classical piece on a cheap component stereo system, I thought something was broken. I hadn't noticed a problem with most popular music, but I find some jazz and most classical digitized at 128bps un-listenable on my low-end component stereo.
Prime numbers are exactly what Alan Greenspan says they are -S. Minsky
One of their key ideas was having the participants submit music they were intimately familiar with. Unfortunately, they should have taken the idea to its logical conclusion: having each participant tested only with songs they submit. Also, they could have at least published the statistics on how participants performed on the song they submitted.
I find it easy to tell the difference between say lossless or even 320 and 128/192 when listening to music I'm very familiar with. But give me a set of random songs I've never heard before and I'd have a much harder time. You don't have to be an audiophile - you just have to be paying attention.
My grievance with low bit rates and/or inferior sound equipment is simply that you won't know what you are missing. And I'm not one of those gold-plated cable audiophiles either -- my "serious" listening equipment is the Etymotics ER4s with a headphone amp. Used for lossless songs, of course.
The big difference that the 256 Kb/s + DRM-free option makes for me is that now I'll buy albums from iTunes Store. Previously I would use iTunes to buy one to three tracks if there was some artist I liked but didn't want a whole album from. But usually I order the CDs online for $8 to $14, rip them to AAC at 192 Kb/s, and put the disc away to collect dust on my overflowing CD rack. Now I can get higher quality cheaper and faster.
Yes, ideally I would rip all my music to a lossless format. And ideally everything would be available on SACD at 2822 KHz rather than 44.1 KHz CDs. But that's just not practical with my 500+ album collection. It'd fill up my laptop's hard drive real quick and allow me to put only a fraction onto my iPod.
I'm also disappointed that the article only tested the tracks on iPods with earbuds. Most of my listening is on a decent stereo system fed from my laptop. Ripping is about convenience, not portability. I only use my iPod when riding the Metro or an airplane. With all the outside noise the bitrate doesn't matter.
And being DRM-free isn't just a matter of idealism. I get frustrated when I go to burn an MP3 CD for my car and discover that one of the tracks I selected is DRMed. Sure there are ways to get around it, but it's just not worth the bother.
AlpineR
We want an experiment designed to identify the codec that will make our music sound the best.
That's it. "best" Not "like the original", which is a poor substitute for "best".
If the original was tainted by phono record pops and clicks, and the codec made that go away, great! We really don't care about "like original". We care about the best.
Your experiment measures something that, while mildly interesting, isn't what we need to determine.
It seems obvious to me they do NOT know what they were doing.
RTFA or not. ( Guess which I chose? )
10 subjects is hardly enough to prove ANYTHING, other than that they
have no idea how to perform a remotely rigorous scientific analysis.
You can expect 2 idiots, 3 to be biased, 4 to be honest, and 1 to lie.
I think 100 would begin to scratch the surface. I'm not trying to be
a snarky science dick, this is self evident. This is epinion.com bullshit.
Show me 10 people who have ipods and I'll show you 5 aol users. (lol)
If you spend a huge amount of money in a particular thing, you have a vested interest in that thing. Much of the audiophile world falls into this category. One example of this is audiophiles who buy expensive power cords for their amps etc, which plug in to the wall. I'm not sure what they think is behind the wall...
I have NO accurate speakers. Instead I cut even more costs and just have a few separate stereos with different speakers hooked up. I use my high-quality Shure studio headphones for recording, then when I'm done, I play it back on all three systems, and I note just how it sounds on each system, so I know over a wide range of speakers/amplifiers (from car amp to hosue speakers, car speakers to house amp, etc.) what I can expect to hear. I listen to it as if I'm hearing it out of Joe Sixpack's home stereo rig, so I know what the average consumer is likely to hear. Saves me money (pawn shops serve my purposes VERY well,) and works out pretty damned well in the end. Also, one thing I don't do is compress the fuck out of my music either. No compressors, thank you. If my meter clips red, I lower the main until everything pans out. I hate metal music nowdays that SOUNDS LIKE IT'S IN ALL CAPS due to the compressors used to sustain a louder volume level without clipping the meter.
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
This test, to a large extent, tells us about the output of the codecs, rather than tell us about the differences between 128k/256k encoding. For a really meaningful test, we must ensure that each song was encoded using the exact same settings.
I can create 256k MP3's which sound worse than 128k MP3's, both from the same WAV. There are a large number of customizations you can use in the encoding process which can really affect the output.
Both kinds of music were missing: They had neither Country nor Western music in the test.
What is America coming to?
Excuse me, but please get off my Pennisetum Clandestinum, eh!
Hydrogen audio forums provide a lot of very good information including well designed double blind comparisons between codecs and bits rates. See this page for details and the links to other testing sites. http://wiki.hydrogenaudio.org/index.php?title=List ening_Tests
All in all an excellent resource for any serious listener.
Actually, I notice a huge difference between 128K and 192K when listening to classical music. For music that doesn't contain the brashness of percussion or brass instruments, the distortion at lower encoding levels is fairly good; however, brass instruments (including brass cymbals) in particular are unbearably distorted when 128K is used but come across rather cleanly when >192K is used. I've finally accepted that a variable rate between 224K and 320K is where I need to encode my tracks in order to make them as close to the original CDs as I can tolerate without using the actual CDs.
The Overrated mod is for reversing inappropriate, positive mods, not for voicing disagreement with a post.
Did they randomize which track was played first? If not they can throw out the whole thing. It's a well-established phenomenon than given a choice between two roughly equivalent items people will overwhelmingly choose the second one. That's how, for instance, Pepsi always did so well in the "Pepsi Challenge."
At 192k I start not to be less annoyed.
I don't think that means what you think it means.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
http://en.wikipedia.org/wiki/Advanced_Audio_Coding
Outside of Apple, the biggest supporter seems to be Sony. Both the PS3 and PSP (and newer Sony phones and Walkmans) can play it, and it's the default codec for the CD ripping ability of the PS3. So the DRMless iTunes songs will benefit PS3/PSP owners quite a bit allowing them to buy songs from iTunes and use them on their machines.
AAC is also the standard audio format of MPEG4/H.264 video.
First, to those who made comments about 128k encoding, you may be thinking of mp3. (Or maybe not, who knows...) From what I've heard, AAC, Vorbis, and AC3 all sound better than mp3 at similar bitrates.
Second, I remember there was a comment on Slashdot awhile back, before they actually came out with these, and I want to confirm... Apparently, CDs are recorded at a certain physical bitrate/frequency, and there are digital masters which are at a higher rate... it's late, so I'm not entirely coherent, but think of it as somewhat equivalent to resolution of a DVD (quality of video is proportional to resolution (HD vs normal) and bitrate). The point was that 256k may actually sound better than a CD, since it comes from a better source than a CD.
If so, this whole test is BS, since they did not do a comparison of CD vs 128k (either iTunes-DRM'd or custom-ripped) vs 256k (un-DRM'd, from the iTunes store). Specifically, I'd want to hear 256k vs CD. But at the same time, I don't know if any iPod, or specifically the one they are using, would be able to handle the higher resolution. If not, you'd have to specifically check your soundcard, too.
And finally, again vaguely remembering this from a Slashdot comment (so correct me if I'm wrong), but there was some comment about "The 30c may seem small, but imagine buying a whole album at these prices..." And I seem to remember that a full album is always $9.99. Still high compared to, for example, the minimum you'd pay for a FLAC-encoded album at MagnaTune, but if you're buying a whole album (and if that's accurate), you may as well just opt for un-DRM'd -- especially if it sounds better than a CD (which would probably cost more anyway.)
But then, of course, I'd like to hear a much bigger study, with more rigorous controls in place. 10 people is just not enough, no matter how you set it up.
And personally, if I had any money to spend on music, I'd be buying un-DRM'd stuff. But probably not from iTunes -- not till there's a web interface (or at least an API) that doesn't require me to download their client software. After all, if I'm buying a single file, the point of the client software is to implement the DRM, and if I buy the un-DRM'd version... Not that it shouldn't also work in the iTunes client, but it'd be nice for it to work natively in AmaroK, or just in a browser.
Don't thank God, thank a doctor!
I'm new at Slashdot (just pointing it out so don't shoot me). I've been a fan of music all of my life. Sometimes it's just obligatory to own a lost track from some obscure dance album and all you find is some lousy MP3 that's chewed up and spat back out. But then again, you've got the track what's to complain about? Sure, it could've been better quality, but you still have it and that's the point. If I rip my own cds I use lame with -V2 because I like the quality, it sounds better do me than -V4, but sometimes you're just happy to hear the song, even if it's on the radio. And what shitty quality is that? (no DAB)
"And as much as we dislike DRM, we just don't think DRM-free tracks alone are worth paying an extra 30 cents a track for.. It would be crazy to pay that premium if you're going to buy the entire album. We'd be more excited if Apple increased the bit rate even further, or--even better--if they used a lossless format."
First off, I've yet to see a lossless formula that WORKS. And by works I mean is easily convertible into mp3/aac so I can use it on a portable player I already own, able to be used. I've seen APE and FLAC, both are too much hassle, and the APE files I got were in japanese. Here's a little fact, Ape doesn't necessarily know how to correctly encode Japanese into ID3, end result? Buffer overflow, bad data. Oh and if they work? They are larger than mp3s and AAC. Lossless codec means all the data has to remain, trust me, that's not a good thing when coupled with all the other little hassles it has.
Second. It'd be crazy to spend 99 cents just to license your files so that you can only use as Apple approves. Paying money to crack the music so I can use it as I want is illegal according to them so why am I paying the money to get locked into their plan. However DRM free music is easily worth 1 dollar and 30 cents because it's mine (It AAC but I can live with that). I don't have to ask permission to use it in another player, I don't have to ask permission to convert it to a data format I choose. Personally I'm fine with 192 for most recordings, I'm not an audiophile, I'm just a listener. If you want the highest grade data, or are an audiophile you'll be buying CDs or fully lossless data, you're not going to fuck with iTunes anyways.
Btw their other idea is to get rid of the apple iBuds and get quality recievers. Hint, This is what got the less interchangable results? I don't exactly see why getting a "higher quality" headset would be desirable if it creates less of a difference instead of more of a difference between two bit rates. Higher quality means I should hear everything. If you are asking people "can you hear the difference" they already should be listening as hard as they can. The theory they try to explain it with doesn't make much sense. They are telling us 30 cents doesn't make a difference but they are trying to sell us on dropping 400 bucks on noise reducing headsets you can get for around 100 if you're clever. Hell they are EARBUDS!!! So far I've notice two things about earbuds. They are uncomfortable, and they are worthless compared to my headphones. If you're talking about noise reducing earbuds just be smart buy a good set of headphones.
Overall a throw away article, I'm still only going to buy DRM less music (I expect you out there to do the same, I'm assuming 30 cents won't kill you, but that's your choice) and hope to soon if Apple ever put the DRMless music out, and had the music I listen to (so far not really). I'm assuming you all are STILL buying music like you are going to. The only mind's this article changes is the cavemen hiding under the rock who still scream "ahhh cds bad", and he's still trying to figure out our compooters, so showing him the internet might not be smart just yet.
Back when the great audiophile debate was between CD and vinyl, New Scientist magazine put a load of audiophiles to the test by playing them the same piece of music from CD and then from Vinyl and asked them to identify which version was from which media and describe the differences between them.
What they didn't tell them was that they simply played the same CD track twice so any differences they thought they heard were purely a result of their own perception fantasies; it didn't stop them from describing in some detail how the two tracks varied though.
Input error. Replace user and press any key to continue.
AAC is supported on everything except cheapo Taiwan guys having MS sponsor them (with Wmedia).
All my files on my Nokia 9300 Symbian phone are AAC even including ringtones embedded in device ROM. The other, "real" phone of mine, not anything close to smart (SE K700i) has everything in aac too. In fact thanks to AAC's better compression, I can use its 46mb flash memory for music.
"(Advanced Audio Coding) An audio compression technology that is part of the MPEG-2 and MPEG-4 standards."
http://www.answers.com/aac
As long as MPEG committee builds the standards, AAC is a standard. All players can't support AAC because the companies making them can't afford or won't get the AAC license. Not because it is only an Apple standard. Zune has AAC because it is iPod competitor and MS also have money to afford that extra format.
Whole confusion comes from the container format of mpeg-4 is built on the quicktime standard I guess. Now there are people who thinks mpeg4/h264 is Apple only format while their new HD-Satellite receiver probably has it in a chip.
The propetioary standards are the ones you mentioned. WMA (if we ignore their propaganda) and ATRAC3 which Sony really missed a huge chance by not opening it.
Nearly all music is recorded and processed at 48kHz. The Red Book standard unfortunately went with 44.1 (for some esoteric reason having to do with syncing with an analog video standard or something back in the 80s). So, there's at least a down-conversion from 48 to 44.1, which isn't the end of the world but you lose some fidelity in the process since its really hard to do that "right" (and its only been recently that people have stopped using langrangian techniques and used truncated sinc functions or polyphase filters to do a decent job without it taking 50 forevers)
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON