Slashdot Mirror


Latest AAC Encoder Comparison Results

bullitB writes "For fans of the world wide patent conspiracy's latest audio format, the latest double blind AAC encoder comparison test results are in. If nothing else, this suggests much of the complaints regarding the iTunes Music Store's lossyness might be unfounded."

22 of 173 comments (clear)

  1. Error bars by thesp · · Score: 5, Insightful

    As a physicist, I'd just like to draw everyone's attention to the error bars on these charts. For the majority of the tests, it's possible to draw a horizontal line through the 95% confidence intervals of nearly all the points.

    Hence, the conclusions declaring clear winners/losers in these cases are invalid. If 99% confidence intervals were used (which gives a better statistical test), I feel that no clear winners or losers would be drawn.

    Be careful with these sort of studies - even though the author has used confidence intervals, he has failed to use them to infer the proper conclusions.

    That said, it's awfully nice to see error bars on this sort of website. Simple data points give such a false sense of precision, I find...

  2. Re:Lossy is lossy by gl4ss · · Score: 4, Insightful

    *(Anybody burned them and then ripped them to mp3 from the CD? I can imagine that would be less lossy than straight transcoding.)*

    if every device was perfect it would be bitwise identical. and what in the world led you to believe otherwise? if you ran it through analog form in some point you might get 'smoother' sound or something but that's just it and self deceit.

    that being said, if you buy music for an mp3 player, buy it in mp3. or rip it yourself from a cd, or just get high enough bitrate it doesn't matter for your golden ears if you code it from one format to another.

    better yet buy from some independends that are willing to provide both formats. or fuck, just encode good old amiga mods.

    --
    world was created 5 seconds before this post as it is.
  3. Re:Audiophile opinion by doofusclam · · Score: 4, Insightful

    Ha! The audiophiles are nutters, it's true.

    But this is different. Bear in mind 'CD quality' ('Red Book' audio) has been established as a base line for the last 10 years or so. Lossy compression degrades this quality by variable amounts depending on what codec is used, what the source material is and at what bitrate it is compressed. The reason for so much development on these codecs isn't to find an audio nirvana, but to minimize the loss from the source material.

  4. Re:I'm not a doctor, but I play one on television by Golias · · Score: 5, Insightful
    I used to think the same way as your friend, because in the 80s and early 90s there did not seem to be any chance that digital sources could sound as good as my favorite LPs.

    Then I heard what good CD players can sound like, and realized that the harshness of CD audio had nothing to do with resolution, and everything to do with component makers cutting corners. Your friend might make the same discovery, if he goes to a good listening room with an open mind.

    --

    Information wants to be anthropomorphized.

  5. Re:Audiophile opinion by Amiga+Lover · · Score: 5, Insightful

    Oh there's definitely a difference in pure analogue vs cd quality vs lossy compression codecs. Just taking a compression ratio down to 96kbit will make most listeners wince when their favourite tracks are played

    I think that's part of it for many people. We might not hear parts of the music just like we may not "see" parts of a video clip on the first run round, but after 3 years listening to Louis Armstrong direct from CD, hearing him on 128kbit MP3 can be harsh. Humans learn and learn well, and the repitition of that playing guarantees we'll hear things that we're not meant to! or rather, things that we don't need to in order to identify a particular artist and recording. But we don't just listen to something to identify it, we listen to enjoy. That's different.

    Most of the time 128kbit is fine for me. 192kbit for the things I'm familiar with.

  6. Re:Lossy is lossy by fok · · Score: 4, Insightful

    Why would you go from AAC to MP3?

    To load it in your mp3 player?

    --
    \m/
  7. Re:AAC is not a standard format by EricWright · · Score: 3, Insightful

    I somewhat understand your angst, but the following is a bit ridiculous:

    "I find that the DRM used in the iTunes store is fair, and more or less barely noticeable"

    It's far from fair, since it requires the kludgey solution of burning to CD and then ripping to an actual usable format in order to make use of your OWN files on your OWN machines.

    If you got a track from iTMS, you MUST have downloaded it with iTunes. Thus, you have a solution for using the original file on your machine (Windows and Mac). Don't complain about the lack of Linux support. It's apple's baby and they can do what they want with it.

    I'm not saying you have to like AAC, or support its right to exist, but if you knowingly buy an iTMS track, caveat emptor.

    Also, I'd like to know what rights Fairplay has denied you? The right to share music with all your friends? Copyright law already forbids that. Fairplay only enforces it. Your example of AAC to CD to MP3/WMA/etc. claims that you have lost the right to directly convert AAC to another format. I hardly find that overly restrictive, considering the alternatives (only one machine, only one portable music player, limited times burning the track, etc.)

  8. Re:iTMS music does NOT sound lossy by mhoward736 · · Score: 5, Insightful

    Bollocks! BUT...

    If I listen to an MP3 or AAC file on my computer with its sound card and speakers (SB Audigy, Boston Accoustics) I can tell the difference between the same (160kbs) MP3 file and a (128k) AAC - the AAC sounds better. I can't tell the difference between the AAC and a WAV file however.

    If I move up to my ($$$$) home stereo then I can easily tell the difference between the compressed and non-compressed versions of the same song. AAC still sounds better to me than MP3 however.

    The difference here is money and environment, my office is a noisy place with all the computers etc running. My listening room is quiet and I spent a lot of time setting the stereo up so that its at its best.

    I have not looked at OOG or any of the other formats so I can't comment on the relative merits of them.

  9. Perspective by UnknowingFool · · Score: 5, Insightful
    Not to quash anybody's personal opinion of AAC, there are a few things that we should note.

    1) Most people can't tell the difference between formats that are similiar in performance.
    2) Some people actually can tell the difference.
    3) Some people are just posers who can't tell the difference but say they can.
    4) Lastly, most people don't really care as long as it is convenient to use either format.

    --
    Well, there's spam egg sausage and spam, that's not got much spam in it.
  10. Do you ask a car mechanic... by SnowDog74 · · Score: 5, Insightful

    For the chemical formula for internal combustion? No. Likewise, I wouldn't go to the local audiophile shop to ask them about audio engineering-related issues. After being accosted by about ten salespeople in ten minutes at a local audio store that sells everything from Sony ES, to Krell, Wadia, Sunfire and the like... I caught a sales rep in a bold-faced lie. I was looking for a receiver without many bells and whistles, and he tried steering me towards Denon. When I asked why Denon is "better", he replied, "Because they focus solely on making audio components unlike Sony." I chuckled and asked him to explain to me the funadamental difference between the sample & hold buffers on a Sony DAC vs. a Denon DAC... Naturally, he had no clue what I was talking about. The "double-blind" survey is somewhat misleading... but that being said, it's clearly not measuring which format is superlative... it's only measuring people's perceptions. And people were pretty much even on those various formats. The study in question just shows that people cannot consistently tell the difference between AAC formats. Now, I've read articles in audiophile magazines that insisted that SACD (Super Audio CD) was brilliant in comparison to CD. And every one of those articles was a load of crap. Fundamentally, even the most "discriminating" audiophiles cannot tell the difference between 16-bit, 44.1kHz PCM (Pulse Code Modulation - e.g. AIFF, WAV, in the computer world) and the 1-bit, 2.7GHz DSD bitstream of SACD... nevermind the minute differences betweeen various AAC-enabled codecs. Hell, I would challenge anyone to be able to tell the difference between 16-bit PCM and MPEG-4 AAC. The AES (Audio Engineering Society) has stated that MPEG-4 AAC is perceptibly indistinguishable from uncompressed 16-bit, dual channel PCM (e.g. CD-DA spec audio).and I would wager any experienced audio engineer's pair of ears (my own included) against any consumer "audiophile" any day of the week. My advice to the idle rich? Don't buy the $45,000 pair of speakers... instead buy yourself better hearing and some common sense. My personal preference? MPEG-4 AAC. As a content creator intensely familiar with a variety of media standards including AES, NTSC, ISO, ITU-R/CCIR, etc. I believe MPEG-4 w/AAC (not Quicktime MPEG-4, mind you, but straight MPEG-4) is the superlative format for compressed audiovisual media. However, for critical listening, only uncompressed audio is the way to go. The general rule of thumb is that higher bitrates are preferable over higher sampling frequencies. Frequency response roll-off is what you want to avoid. But in order to support the higher bitrates, you need a D/A Converter (DAC, Digital-to-Analog Converter) with an effective sample-hold buffer that can crunch the necessary data to make an accurate conversion of the digital source. That being said, I'm going to begin digitally remastering my own compositions soon... and go straight from the 24-bit master to a 24-bit multichannel DVD-Audio format. Why? Even an audiophile deafened by the sound of their money burning a hole in their wallet can actually tell the difference between my 24-bit master recordings and the dithered 16-bit CD audio.

    1. Re:Do you ask a car mechanic... by GlassHeart · · Score: 2, Insightful
      The "double-blind" survey is somewhat misleading... but that being said, it's clearly not measuring which format is superlative... it's only measuring people's perceptions.

      And here I was thinking that the whole point of lossy audio compression was about throwing away information that people could not hear. :)

      The format that throws away the least audible information (as determined, in fact, by "measuring people's perceptions"), other things (encoder complexity, file size, etc) being equal, is the winner.

      for critical listening, only uncompressed audio is the way to go.

      Somehow I don't think 99-cent files that can be downloaded in seconds was meant for "critical listening". I'm not saying you're wrong, just that you may want to evaluate a product or technology against its stated goals, not against your personal goals. Apple has stated that it basically breaks even at 99 cents per song, so what would it mean for their business model if you increase their bandwidth expenses tenfold?

    2. Re:Do you ask a car mechanic... by valkraider · · Score: 3, Insightful

      All the "formats" sound the same when played through my fairing speakers on my motorcycle - at 75mph...

      However, when I am in my car playing them off my iPod->TapeDeckConvertor, well - all the formats still sound the same.

      At home on my expensive stereo, I can easily tell the difference between CD and AAC. The AAC is the one playing off my iPod, and the CD is stored in a corrigated box in the basement...

  11. Re:Lossy is lossy by Anonymous Coward · · Score: 2, Insightful

    You completely missed my point. My MP3 player wasn't obsolete when I bought it. And an AAC player won't be obsolete if I buy it today. But all formats are obsoleted eventually. What happens when I'm stuck with a load of AAC files, and FormatX is the popular one? I have the choice of hunting down a player that still supports AAC, or transcoding and getting a load of artifacts, or throwing away my music.

  12. Don't use ANOVA here by fscmj · · Score: 4, Insightful

    In this analysis he presents the results of an ANOVA (Analysis of Variance). This reminds me of the saying: "If your only tool is a hammer, every problem looks like a nail". The tool the author is using is ANOVA and so he is trying to force the data into that form. Hovever, the assumptions of any ANOVA is that the data is independent, normally distributed, and has constant variance. The response ranges from 1.0 to 5.0. I haven't taken the test myself and do not know if users were allowed to select non-integer values or not but even if they were we can see from the graphs he does present that most of the responses were near the upper bound of 5.0. This type of response is clearly not normally distributed. ANOVA is faily resistant to departures from normality but one would need to fully explore the degree of the departure before placing any weight to the confidence intervals presented. My gut feeling here is that it is highly skewed and will present confidence intervals smaller than what they should be (data is forced to be artifically close due to the upper bound and having so many people report values close to that upper bound). The data can probably be viewed as independent but it must be recognized that this is an assumption. Constant variance departures may be a problem as higher responses are less variable than middle responses (due to the upper bound again). It would probably be much better to use a non-parametric test alternative to ANOVA such as a Kruskal-Wallis. Scope of inference: This is not a random sample from any population and as such cannot be interpreted to represent anything more than the perceptions of the respondants themselves. -chris

    1. Re:Don't use ANOVA here by ff123 · · Score: 2, Insightful

      The intervals in the rating scale are 0.1 steps, which is close enough for argument's sake.

      And ANOVA is a robust method as you've commented, so it's probably reasonable to assume normality. In any case, the raw data is available for any stats weenies to play with, and there are a couple of more conservative methods besides the Fisher LSD readily available to try, if anyone has an uncontrollable urge.

      I point you to:

      http://ff123.net/friedman/stats.html

      where you can run a non-parametric analysis of the raw data using a web form if you like. Or download the program to correct the pvals for multiple comparisons using Tukey's Honestly Significant Difference instead of the Fisher LSD (either paramteric or non-parametric).

      And if you want to really geek out, you can do a bootstap resampling method with different methods of correcting the pvals for multiple comparisons:

      http://ff123.net/bootstrap/

      But let's not lose the forest for the trees here. The blocked ANOVA/Protected Fisher LSD used for the test provides a reasonable (if not the most conservative) summary of the results.

      A couple of more important weaknesses of the test are (and they're related):

      1. Only 12 samples were used. Although this is probably close to the practical limit, more samples always provides a more comprehensive picture. The results are suggestive, but not definitive.

      2. The selection of the samples makes a difference. If you listen mainly to classical for example, this test may not be representative for you.

  13. Its all about the file size by acomj · · Score: 3, Insightful

    People agonize because they want the best quality / size ratio. for example if aac encodes at 128 Kbps as well as mp3 at 192 Kbps you can fit more tunes on you harddrive in the same space and be happy. This isn't critical on computers with they're large hard drives, but for portables] players it matters. People also don't want to rip all there CDs multiple times.

    In someways your right, people should pick a bit rate/format that works for them and not worry about it. but this is slashdot..

  14. Re:Sounds good to me by Anonymous Coward · · Score: 1, Insightful

    >Sounds like what I did. Then I played my iPod through my home stereo. Yikes! Now I'm re-ripping to 256 MP3. Like you said, though, it's personal preference and what you're using the files for and everybody's different.

    So you jumped from 192kbps MP3 to 128kbps AAC then back to 256kbps MP3... Did you at least try AAC at higher bitrates?

  15. Re:Lossy is lossy by Snowspinner · · Score: 3, Insightful

    Yeah, I looked at many of those players.

    Then I bought an iPod, because it was actually well designed - it had an interface that actually lends itself to playing music. I want an mp3 player that I can change the song on while driving - that is, one that's fast to use, and is forgiving of being put down halfway through a menu and picked up again five minutes later because the traffic got bad.

    And, frankly, though I could carry an mp3 player larger than an iPod around, if I don't have to, I'm just as happy not to. I mean, yeah, a larger thing would fit in my pocket, but I'd just as soon also put in my gloves, hat, scarf, phone, pen, GBA, candy bar, and whatever the hell else I'm carrying on a given day.

  16. Objectively comparing formats by Experiment+626 · · Score: 2, Insightful

    These kinds of tests comparing codecs always seem to be something involving playing two versions of the song and asking someone which in their opinion sounded better. Isn't there a more quantitative way to measure the effect of the lossy compression? For instance:

    1. Start with the digitized CD recording
    2. Make a copy of it
    3. Compress the copy with the codec to be analyzed
    4. Do a lossless uncompression of the MP3 or whatever it is back to CD-resolution
    5. At each sample point (44k per second) on the resulting track, compare the 16-bit sample value of the compressed-and-back version to the control version.
    6. Sum the absolute value of the differences across all sample points.

    In other words, whichever codec introduces the least error into the track in a closed loop encode and decode test did the best job of faithfully reproducing the original signal. No subjective human testing required. You might have to tweak it a bit (say, sum the squares of the error or something) but would an approach like this work to settle the codec debates, or is there a fundamental flaw in this technique?

  17. Re:AAC is not a standard format by Anonymous Coward · · Score: 1, Insightful

    Linux support for Apple DRM:
    http://www.videolan.org

    Why do people act like VLC does not exist in discussions about Apple DRM?

    Have Apple zealots been told by God Jobs to not talk about it so the RIAA doesn't find out?

    Fair Use and DRM are mutually exclusive.

  18. Re:Lossy is lossy by Snowspinner · · Score: 3, Insightful

    Somehow, I don't think that car speakers and highway noise are a viable solution for audio quality too.

    I mean, you're driving. If you're being safe, your sound quality is going to be hampered by the fact that your stereo isn't louder than the sounds your car and the road are making.

  19. Re:AAC vs. AAC not the issue by Anonymous Coward · · Score: 1, Insightful

    They allow people to preview 30 seconds of a song. Figuring an average song is 3 minutes, I'd say most of their bandwidth is squandered by previews.

    Hehe, Akamai, land of infinite bandwidth =D