Posted by
Hemos
on from the learn-more-about-it dept.
bullitB writes "For fans of the world wide patent conspiracy's latest audio format, the latest double blind AAC encoder comparison test results are in. If nothing else, this suggests much of the complaints regarding the iTunes Music Store's lossyness might be unfounded."
Discussion
by
doofusclam
·
· Score: 5, Informative
You can discuss this test with the author and others at http://www.hydrogenaudio.org
Re:iTMS music does NOT sound lossy
by
doofusclam
·
· Score: 3, Informative
Absolute rubbish.
Maybe you don't understand the nature of the tests?
FWIW, with the Norah Jones track 'Creepin In' (not used in this test) I can not only ABX every codec bar musepack, I can also spot the aac and mp3 variants because of the way they degrade.
Being a medical student, I assume you understand basic psychoacoustic principles?
Re:iTMS music does NOT sound lossy
by
misterpies
·
· Score: 5, Informative
I can tell you're a medical student and not a doctor, because most doctors are much more realistic about the limits of their knowledge. "Scientific" limits on the capabilities of biological systems are often wrong because of some overlooked or unknown factor.
A famous example of this is that for many years scientists could not work out how bees could fly. Their wings were too small, muscles too weak and bodies too heavy. It turned out that bees were able to use previously unknown aerodynamic effects to generate more lift than our previous "knowledge" allowed. Another example is that many birds of prey have visual acuity better than the laws of optics, applied to their eyes, would seem to permit. It turned out that the visual signal processing in their brains is so advanced that birds can actually 'see' features that are below the resolution limit of their eyes.
Similarly, we shouldn't be too dogmatic about what humans can and cannot hear. MP3s (and presumably AACs) compress music by suppressing parts you "can't" hear, not because they're outside your range of hearing but because the brain, assuming those parts should be there, fills in for them even when they're absent.
But it may be that you can't hear something consciously but still tell that it's not there. For example, there was a news story a week or so back showing that people could somehow tell when a picture had changed by the removal of an item in it, even though consciously they could not explain what the difference was - it just 'felt different'.
So, if someone claims to be able to tell the difference between 1 128Kb AAC and a CD, test that claim in a double-blind experiment. Only when he fails the test can you say he was imagining things.
-- The author of this post asserts his moral rights.
I'm not a doctor, but I play one on television
by
AtariAmarok
·
· Score: 4, Informative
It's either that, or "Damn it, Jim, I'm a doctor, not an audiophile!"
It is clear you have no idea what you are talking about. Just because you can't tell the difference does not mean others can.
The people who are "able to tell" just happen to have more sensitive hearing. I'm probably not one of them, but I have known several, including someone who cannot listen to CD's because there is a whine on all of them associated with the digital nature (this same guy does not like going into Radio Shack because of the noise made by their security system.)
Just because you are not a sensitive-eared audiophile does not mean everyone has the same cloth-ears as you do.
-- Don't blame Durga. I voted for Centauri.
Re:I'm not a doctor, but I play one on television
by
Golias
·
· Score: 5, Informative
The parent to this post was modded down for being rude to the grandparent, I suppose... but the point was correct. "The Digital Sound," as we used to call it back in the 80s, turned out to be the result of poor D/A conversion, poor error correction, and amplification hardware that was tweaked to compensate the shortcomings of LPs. Like I mentioned on another audio-related thread last week, a $300 Rotel CD player connected to a modern high-end stereo will sound as good or better when compared to a $3000 air-suspended, laser-guided turntable. (Especially after the LP has been played a few dozen times.)
Re:AAC is an aberration that should go away
by
DiscoOnTheSide
·
· Score: 4, Informative
AAC is a standard format. Perhaps you've heard of the people that made it... Dolby? Ring a bell? Just about anything doing with excelence in audio comes from them. I can see four things in my shoebox of a dormroom that has their logo on it.
I also find that most of the people that are so violently against DRM (in any form) are the people who would be analy raped by the RIAA/MPAA if they raided your house. I find that the DRM used in the iTunes store is fair, and more or less barely noticeable. Don't have a player that can play AAC? Buy one or shut the fuck up. You bought a player that doesn't do what you want it to, thats no one's fault but your own. Thats like buying DVDs then bitching because your VCR won't play them.
Grow up.
-- Viva La Revolucion! Buy a Mac!
Unfortunately...
by
lotsofno
·
· Score: 5, Informative
Winamp 5.02's encoder (which got a lot of help from hydrogenaudio's own Menno, a FAAD AAC-decoder developer/Ahead MPEG4 developer) wasn't included in the listening test because of a bug they found before testing.
Too bad, too. I would've loved to have seen how it compared.
Sounds good to me
by
kherr
·
· Score: 4, Informative
I had all of my CDs ripped to 192kbps MP3. When iTunes came out with AAC, I did a bunch of rip testing. I ripped from Donald Fagen's The Nightfly in a bunch of formats and bitrates. I found, for my personal preference, that 128kbps AAC was at least as good as 192kbps MP3, if not better. So I reripped all of my CDs to 128kbps AAC and got more songs onto my 5GB iPod. Now I'm on to ripping all of my old vinyl to AIFF, eventually to end up as AAC. Huzzah!
Re:Error bars
by
patman600
·
· Score: 5, Informative
As a high school senior in a Statistics AP class currently studying confidence intervals and hypothesis testing, I think you are missing something. At the beginning, he clearly states your point: "One codec can be said to rated better than another codec with 95% confidence if the bottom of its line segment is at or above the top of the competing codec's line segment." The author gives which one is in first place, but announces at the beginning the requirements for a clear winner. And the author seems to me to be requiring at least half an interval of difference to even say that, much of the time he says they are tied.
Re:iTMS music does NOT sound lossy
by
dmdimon
·
· Score: 3, Informative
Being a medical student who has a particular interest in this stuff, you have to know that due to compression method AAC, MP3, ATRAC, etc generates artifacts during compression. I mean due to "harmonising". Furthermore, there exists some "dynamic range compression" So I have to tell that:
a) if you are listening in APPROPRIATE conditions and on APPROPRIATE sound system (mean amplifyer&speakers&QUET room) you easily can distinguish compressed from flat by dynamic range. Possibly you can't tell what's difference is, but you CAN hear it. b) it's not so hard to generate a soundwave that will compress very bad (you easily can point difference) at any bitrate.
Example: get audiotest CD, compress it, listen to what you get.
Which 78s sound best, RCA or Columbia?
by
dpbsmith
·
· Score: 1, Informative
I have to admit to being puzzled as to why people spend so much time and energy trying to determine the relative merits of lossy formats.
I find the music I've downloaded from iTMS perfectly acceptable; ditto the music I hear on my car's factory-equipment FM receiver. That doesn't mean I can't tell the difference between them and better sound.
Actually, I've been transferring my LP's to CD... and recently I've been converting the CD's to.mp3 format with iTunes. The first recording I compressed, using some middle-of-the-road "high-quality" setting, happened to be a recording I liked because of the warmth of the violins. Remember, this is an old LP... digitized on a $250 consumer-grade CD recorder. To my utter astonishment, I could instantly tell the difference. After some experimentation, I upped everything to the max, encoded at 256k bits/second... and could still tell the difference.
On the other hand, with popular music (e.g. the Beatles) and some classical recordings, I couldn't.
The point is, if I can hear the difference between a CD and an.mp3 at 256k bits per second, that tells me that the difference in quality is NOT in some rarefied, golden-ears, territory. It's like the difference between a CD and FM radio.
So, why even bother to agonize over minute differences in an imperfect format when a) upping the bit rate does far more to improve quality than fussing over which format is best; b) the mileage varies so much depending on program material; c) they're obviously inferior to CD sound to begin with?
Isn't it a little like arguing over which electrically-recorded 1950s 78's sound better... RCA or Columbia?
Re:You may have a point
by
BorgCopyeditor
·
· Score: 5, Informative
They may be pricks (or trolls), but they have some good points. If you can get past the annoying exterior, you might find some good information on these issues by googling for "audiophile" in rec.audio.pro, a group populated primarily by very good recording engineers. These are guys (mostly) who got where they are through both excellent (and aesthetically attuned) hearing and scientific knowledge of how audio works at every point in the signal chain. To watch them dismiss, with unimpeachable arguments and long experience, the claims of "sensitive" audiophiles can be instructive. I speak as one who has been schooled.
-- Shop as usual. And avoid panic buying.
Re:Audiophile opinion
by
node+3
·
· Score: 2, Informative
The claim of CD quality isn't for MP3 at 128kb/s, it's for AAC at 128kb/s. It's interesting that you said that 192kb/s is what you prefer, since that's the MP3 bit rate that 128kb/s AAC is said to equal.
Look at Roberto's previous listening tests
by
waaka!
·
· Score: 3, Informative
You might find the graphs for a previous listening test interesting if you want to see how AAC stacks up against other codecs.
A warning to potential HA posters
by
waaka!
·
· Score: 4, Informative
For good reasons, the posters on Hydrogenaudio don't take kindly to people making unfounded assertions about which codecs are better, so if you're going to argue with them, think twice and ABX first. You will be, after all, arguing with many audio developers, e.g. people who make contributions to LAME, people who've tuned the Vorbis encoder, and a surprising number of people who work for Ahead (makers of Nero, of course).
Just a minor detail to mention here. Dolby licenses two different versions of their AAC codes. iTunes, when encoding for end users, uses the Dolby Consumer codec (affordably licensed by Apple in Quicktime 6). The Itunes Music Store uses the Dolby Professional codec (which would not be affordable to license in iTunes).
Thus, AACs coming out of the iTunes Music Store have a higher quality at the same compression rate than the same songs you rip and convert on your own copy of iTunes.
-- "It is our blasphemy which has made us great, and will sustain us, and which the gods secretly admire in us." - Zelazny
Whoa there! Look at the error bars...
by
Jon+Abbott
·
· Score: 2, Informative
I have looked at these sorts of test results before, and used to take them as the truth. That was before I started taking a Design of Experiments grad class, and have some evidence to the contrary. For what it's worth, I'm the top performer in the class right now.
The issue I have is with the error bars. These are the vertical lines above and below the mean of each encoder. Like the beginning of the report says, "One codec can be said to rated better than another codec with 95% confidence if the bottom of its line segment is at or above the top of the competing codec's line segment." This is very much true for these sorts of statistical tests -- if the error bars overlap, that indicates that the means of the two groups are statistically identical. One could always adjust their confidence interval to a lower percentage, but 95% is quite often the standard.
Note how many of the plots in this test have overlapping error bars. In the first plot, for example, all of the encoders tested have overlapping error bars. The results drawn from this plot should be that no encoder was measurably different than any other encoder -- not that iTunes won, like the results say. (Note: I own a Powerbook G4, and am typing this post on it right now, and I love Apple. I just don't like bad statistics, that's all)
The results given in many of the plots are based strictly on the means of the samples, and not the error bars, which are actually more important in this case. Do not trust them. Interpreting the plots with the logic stated at the beginning of the article is the only statistically sound method (that I know of). I hope this sheds some more light on these tests...
Slashdot Mirror
You can discuss this test with the author and others at http://www.hydrogenaudio.org
Absolute rubbish.
Maybe you don't understand the nature of the tests?
FWIW, with the Norah Jones track 'Creepin In' (not used in this test) I can not only ABX every codec bar musepack, I can also spot the aac and mp3 variants because of the way they degrade.
Being a medical student, I assume you understand basic psychoacoustic principles?
here
l
c =1 3464&
http://www.rjamorim.com/test/64test/results.htm
and here
http://www.hydrogenaudio.org/index.php?showtopi
I can tell you're a medical student and not a doctor, because most doctors are much more realistic about the limits of their knowledge. "Scientific" limits on the capabilities of biological systems are often wrong because of some overlooked or unknown factor.
A famous example of this is that for many years scientists could not work out how bees could fly. Their wings were too small, muscles too weak and bodies too heavy. It turned out that bees were able to use previously unknown aerodynamic effects to generate more lift than our previous "knowledge" allowed. Another example is that many birds of prey have visual acuity better than the laws of optics, applied to their eyes, would seem to permit. It turned out that the visual signal processing in their brains is so advanced that birds can actually 'see' features that are below the resolution limit of their eyes.
Similarly, we shouldn't be too dogmatic about what humans can and cannot hear. MP3s (and presumably AACs) compress music by suppressing parts you "can't" hear, not because they're outside your range of hearing but because the brain, assuming those parts should be there, fills in for them even when they're absent.
But it may be that you can't hear something consciously but still tell that it's not there.
For example, there was a news story a week or so back showing that people could somehow tell when a picture had changed by the removal of an item in it, even though consciously they could not explain what the difference was - it just 'felt different'.
So, if someone claims to be able to tell the difference between 1 128Kb AAC and a CD, test that claim in a double-blind experiment. Only when he fails the test can you say he was imagining things.
The author of this post asserts his moral rights.
It's either that, or "Damn it, Jim, I'm a doctor, not an audiophile!"
It is clear you have no idea what you are talking about. Just because you can't tell the difference does not mean others can.
The people who are "able to tell" just happen to have more sensitive hearing. I'm probably not one of them, but I have known several, including someone who cannot listen to CD's because there is a whine on all of them associated with the digital nature (this same guy does not like going into Radio Shack because of the noise made by their security system.)
Just because you are not a sensitive-eared audiophile does not mean everyone has the same cloth-ears as you do.
Don't blame Durga. I voted for Centauri.
You mean FLAC.
FAAC is another AAC codec.
AAC is a standard format. Perhaps you've heard of the people that made it... Dolby? Ring a bell? Just about anything doing with excelence in audio comes from them. I can see four things in my shoebox of a dormroom that has their logo on it. I also find that most of the people that are so violently against DRM (in any form) are the people who would be analy raped by the RIAA/MPAA if they raided your house. I find that the DRM used in the iTunes store is fair, and more or less barely noticeable. Don't have a player that can play AAC? Buy one or shut the fuck up. You bought a player that doesn't do what you want it to, thats no one's fault but your own. Thats like buying DVDs then bitching because your VCR won't play them. Grow up.
Viva La Revolucion! Buy a Mac!
Winamp 5.02's encoder (which got a lot of help from hydrogenaudio's own Menno, a FAAD AAC-decoder developer/Ahead MPEG4 developer) wasn't included in the listening test because of a bug they found before testing.
Too bad, too. I would've loved to have seen how it compared.
I had all of my CDs ripped to 192kbps MP3. When iTunes came out with AAC, I did a bunch of rip testing. I ripped from Donald Fagen's The Nightfly in a bunch of formats and bitrates. I found, for my personal preference, that 128kbps AAC was at least as good as 192kbps MP3, if not better. So I reripped all of my CDs to 128kbps AAC and got more songs onto my 5GB iPod. Now I'm on to ripping all of my old vinyl to AIFF, eventually to end up as AAC. Huzzah!
As a high school senior in a Statistics AP class currently studying confidence intervals and hypothesis testing, I think you are missing something. At the beginning, he clearly states your point: "One codec can be said to rated better than another codec with 95% confidence if the bottom of its line segment is at or above the top of the competing codec's line segment." The author gives which one is in first place, but announces at the beginning the requirements for a clear winner. And the author seems to me to be requiring at least half an interval of difference to even say that, much of the time he says they are tied.
Being a medical student who has a particular interest in this stuff, you have to know that due to compression method AAC, MP3, ATRAC, etc generates artifacts during compression.
I mean due to "harmonising".
Furthermore, there exists some "dynamic range compression"
So I have to tell that:
a) if you are listening in APPROPRIATE conditions and on APPROPRIATE sound system (mean amplifyer&speakers&QUET room) you easily can distinguish compressed from flat by dynamic range. Possibly you can't tell what's difference is, but you CAN hear it.
b) it's not so hard to generate a soundwave that will compress very bad (you easily can point difference) at any bitrate.
Example: get audiotest CD, compress it, listen to what you get.
I have to admit to being puzzled as to why people spend so much time and energy trying to determine the relative merits of lossy formats.
.mp3 format with iTunes. The first recording I compressed, using some middle-of-the-road "high-quality" setting, happened to be a recording I liked because of the warmth of the violins. Remember, this is an old LP... digitized on a $250 consumer-grade CD recorder. To my utter astonishment, I could instantly tell the difference. After some experimentation, I upped everything to the max, encoded at 256k bits/second... and could still tell the difference.
.mp3 at 256k bits per second, that tells me that the difference in quality is NOT in some rarefied, golden-ears, territory. It's like the difference between a CD and FM radio.
I find the music I've downloaded from iTMS perfectly acceptable; ditto the music I hear on my car's factory-equipment FM receiver. That doesn't mean I can't tell the difference between them and better sound.
Actually, I've been transferring my LP's to CD... and recently I've been converting the CD's to
On the other hand, with popular music (e.g. the Beatles) and some classical recordings, I couldn't.
The point is, if I can hear the difference between a CD and an
So, why even bother to agonize over minute differences in an imperfect format when a) upping the bit rate does far more to improve quality than fussing over which format is best; b) the mileage varies so much depending on program material; c) they're obviously inferior to CD sound to begin with?
Isn't it a little like arguing over which electrically-recorded 1950s 78's sound better... RCA or Columbia?
"How to Do Nothing," kids activities, back in print!
They may be pricks (or trolls), but they have some good points. If you can get past the annoying exterior, you might find some good information on these issues by googling for "audiophile" in rec.audio.pro, a group populated primarily by very good recording engineers. These are guys (mostly) who got where they are through both excellent (and aesthetically attuned) hearing and scientific knowledge of how audio works at every point in the signal chain. To watch them dismiss, with unimpeachable arguments and long experience, the claims of "sensitive" audiophiles can be instructive. I speak as one who has been schooled.
Shop as usual. And avoid panic buying.
The claim of CD quality isn't for MP3 at 128kb/s, it's for AAC at 128kb/s. It's interesting that you said that 192kb/s is what you prefer, since that's the MP3 bit rate that 128kb/s AAC is said to equal.
You might find the graphs for a previous listening test interesting if you want to see how AAC stacks up against other codecs.
For good reasons, the posters on Hydrogenaudio don't take kindly to people making unfounded assertions about which codecs are better, so if you're going to argue with them, think twice and ABX first. You will be, after all, arguing with many audio developers, e.g. people who make contributions to LAME, people who've tuned the Vorbis encoder, and a surprising number of people who work for Ahead (makers of Nero, of course).
Just a minor detail to mention here. Dolby licenses two different versions of their AAC codes. iTunes, when encoding for end users, uses the Dolby Consumer codec (affordably licensed by Apple in Quicktime 6). The Itunes Music Store uses the Dolby Professional codec (which would not be affordable to license in iTunes). Thus, AACs coming out of the iTunes Music Store have a higher quality at the same compression rate than the same songs you rip and convert on your own copy of iTunes.
"It is our blasphemy which has made us great, and will sustain us, and which the gods secretly admire in us." - Zelazny
I have looked at these sorts of test results before, and used to take them as the truth. That was before I started taking a Design of Experiments grad class, and have some evidence to the contrary. For what it's worth, I'm the top performer in the class right now.
The issue I have is with the error bars. These are the vertical lines above and below the mean of each encoder. Like the beginning of the report says, "One codec can be said to rated better than another codec with 95% confidence if the bottom of its line segment is at or above the top of the competing codec's line segment." This is very much true for these sorts of statistical tests -- if the error bars overlap, that indicates that the means of the two groups are statistically identical. One could always adjust their confidence interval to a lower percentage, but 95% is quite often the standard.
Note how many of the plots in this test have overlapping error bars. In the first plot, for example, all of the encoders tested have overlapping error bars. The results drawn from this plot should be that no encoder was measurably different than any other encoder -- not that iTunes won, like the results say. (Note: I own a Powerbook G4, and am typing this post on it right now, and I love Apple. I just don't like bad statistics, that's all)
The results given in many of the plots are based strictly on the means of the samples, and not the error bars, which are actually more important in this case. Do not trust them. Interpreting the plots with the logic stated at the beginning of the article is the only statistically sound method (that I know of). I hope this sheds some more light on these tests...
Slashdot's first reaction to VMware