Slashdot Mirror


Hydrogenaudio AAC Listening Test Results

caffeine_monkey writes "Hydrogenaudio's AAC public listening test, previously posted on Slashdot, is now over and the results are in. The test compared five codecs at 128 kbps, including Psytel, Nero, Sorenson Squeeze, QuickTime, and FAAC. The winner? 'QuickTime is a clear winner, performing much better than the competition. Sorenson Squeeze, Psytel AACenc and Nero are tied, with Sorenson slightly higher than the others. Faac is clearly the worst.'"

17 of 306 comments (clear)

  1. Microsoft by x0n · · Score: 1, Insightful

    I knew that subject would get your respective attentions, but... the obvious question is, why wasn't WMA v8/v9 included in the test?

    - Oisin

    --

    PGP KeyId: 0x08D63965
    1. Re:Microsoft by C_nemo · · Score: 4, Insightful

      Because comparing apples to oranges is never a bright idea. The comparison is between AAC codecs not a few AAC codecs vs WMA.

  2. Real competition absent by weeble · · Score: 2, Insightful

    While it may be relevant to have a comparison between different AAC encoders the trial would have much more relevance to real life if it had included ogg and mp3.

    A little extra work for a lot more sense in the results.

    Have fun
    Xander

    --
    Slashdot Beta should die a painful death.
    1. Re:Real competition absent by viktor · · Score: 5, Insightful

      Actually I do not agree with you at all. Comparing different encoders for a certain format is a complete test. In this particular case, the goal was to find out which AAC encoder is the best. The results answered that question.

      As an Open-Source (and open standards) advocate it is easy to look at such a test and say "but Ogg/MP3/whatever is better anyways, they should have included them". Actually saying so leads nowhere at all.

      If you want a comparison between Ogg, AAC, WMA and MP3, then do such a comparison. But this was not a test to find our which format/algorithm was the best, it was a test to find out which AAC encoder was the best, which is also what the test answered.

      Not every test in the world should always include all and every variant of the test subject in question. Just because you're comparing the quality of carrots from different farmers, you shouldn't automatically be expected to include potatoes in the comparison just because there are people that prefer potatoes over carrots. If the test question was "which is the best vegetable", then omitting potatoes could be considered careless.

      And, in very much the same way, if the test question is "which is the best AAC encoder", including MP3-or Ogg-encoders would be just as wrong.

      That's my view, at least.

      /Viktor...

    2. Re:Real competition absent by deuce868 · · Score: 2, Insightful

      No, this is more like comparing the quality of service at 4 ford dealerships, or perhaps testing performance kits on a ford from four different companies. As everyone else is stating...other formats, or brands of cars, are not relevent to this at all.

    3. Re:Real competition absent by bill_mcgonigle · · Score: 4, Insightful

      And to further your argument, if the question is 'which is the best sounding format, aac, ogg, or mp3?' then this test is a prerequsite. To compare the formats, one must pick an encoder to encode the samples. The only fair way to do this is to use the best encoder from each group. This test shows that it would be unfair to use faac, for example, to encode the AAC samples.

      So, the self-righteous open-standards advocates (no denegration intended) ought to be organizing a test just like this article talks about for ogg and mp3 if there's not a suitable prior study.

      Only then can they properly organize the audio-format bake-off.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
    4. Re:Real competition absent by Anonymous Coward · · Score: 1, Insightful

      That's great, but what about using optimal configurations for the test.

      1. For all codecs they were using a CBR configuration.
      2. For Nero they were using "Low Complexity" AAC. I have performed my own test with Nero vs Quicktime and I can say without a doubt in my mind that Nero with "High Efficiency" AAC will definately outperform Quicktime. (Unfortunately you are then relegated to using NeroMix to play the AAC back, because Quicktime does not decode High Efficiency AAC properly.)

      .

  3. be careful by borgdows · · Score: 3, Insightful

    Do not confuse FAAC and FLAC (the lossless audio format from Xiph).

    Btw, why does this test compare only proprietary formats and not free (as speech) formats like Ogg Vorbis or FLAC ?

  4. Re:Things I've heard from Audiophiles... by VCAGuy · · Score: 3, Insightful

    Hey! I resemble that remark! Well, not really; but as someone who works with pro audio, I must say that most self-professed "audiophiles" are indeed full of it. [Oh, and by the way, if the cables really make a difference you need to A) get out more, and B) switch to XLR balanced analog or AES/EBU digital connections.]

    --
    Q: "Why do sound techs say 'check 1, 2'?"
    A: "Cause if they could count any higher they'd be lighting techs."
  5. Why is parent at -1? by Anonymous Coward · · Score: 2, Insightful

    Its so fucking true! Everytime an audio format gets mentioned all the simpering open sores twats come out squealing 'what about OGG? pleeeeeeeeeeeeeeaaaaaaasee mummy pleeasssee dont ignore me'

    SHUT THE FUCK UP ABOUT OGG YOU PASTY FACED FAT HIPPY CUNTS

  6. Re:Things I've heard from Audiophiles... by e1618978 · · Score: 4, Insightful

    About half that stuff makes sense to me. Many of the things you wrote seem reasonable and valid - I don't see what you are making fun of. For example, many pre-amps (not mine) change character during loud transients. Opera singers seem to jump out at you when they hit the really high/loud notes. A lot of the quoted statements make perfect sense - it is just that they are having trouble explaining their experience, and you are taking the quotes out of context to make them look stupid. Slashdot is not capible of discussing high end audio - stick to your MP3 players and surround sound computer speakers.

  7. Re:Why should that be a surprise? by wfberg · · Score: 2, Insightful

    "mostly audiophile gear is a jerk-off session of wannabee music expert wealthy bast^H^H^H^H guys who can afford to spend large anounts of money on equipment"

    And this differs from computer hardware discussions how?


    Computer hardware discussion usually discuss measurable quantities such as Mhz, fps, Mbps, etc. The more subjective stuff (CISC vs. RISC) is usually backed up by benchmarks (with all sides accepting that ALL benchmarks are bullshit).
    It is rare to hear an audiophile even mention the word "frequency", let alone relate this to any numbers ("high" frequencies? wotsat?).

    Now, if you're talking about software, that's when geeks get religious! ;-)

    --
    SCO employee? Check out the bounty
  8. Re:Things I've heard from Audiophiles... by 91degrees · · Score: 2, Insightful

    Depends on whether you mean the difference between a couple of bits of wire and a good quality cable, or the difference between decent quality cable, and ludicrously expensive cables with loads of whizzy buzzwords. After a certain level, the amount of improvement you can get even with a superconductor is so far below the threshold of hearing that it becomes pointless.

  9. Re:interesting by Anonymous Coward · · Score: 1, Insightful

    I use FLAC, but I've done comparisons between QT AAC and FhG & LAME MP3. At the 128 kps range the AAC quality increase is more than a tad better.

    And with the iPod marketshare you can't discount AAC for compatibility and then turn around and suggest something like OGG.

  10. Re:Things I've heard from Audiophiles... by makohund · · Score: 2, Insightful

    Yeah, I was going to say the same thing. I think audiophiles that go for that kind of crap are total nutjobs, but some of those lines aren't bunk.

    There were a few somewhat valid critiques of music passages (like the different ranges of bass frequencies being there or not). But I think the point here was that the nutjob was atrributing the fact to having a $1000 cable or not.

    Different tubes do sound different, though. Talk to any musician that uses amplifiers... stick different tubes in and it will sound like a different amp. It should be less of an issue w/ stereo equipment (since instrument amp tubes are PURPOSELY colored) but I'm sure it still holds true. (That doesn't mean it is enough to really care about.)

    Same goes for pre-amps. Push them hard, and they will sound different. Certain freqeuncies the pre is more efficient with MAY become slightly more pronounced than others. (The better the pre, the less this should be true.) They will also distort. Some times clearly distorting, sometimes just sounding like a really hot signal. But more to the point... you shouldn't be pushing a preamp hard to begin with, unless you actually want distortion. (Not uncommon with instruments. Dumb with a stereo. If you need more volume, you turn up the power amp, not blast the pre.)

    Lastly, if these looneys truly want their "perfect listening experience" and "totally faithful reproduction" the last thing they need to do is blow $20,000 on a fancy stereo. They need to call up the studio that mastered the recording and find out what the mastering engineer used for monitoring while he worked. (Probably more than one system, but they'll have a primary reference.) That is what they need to buy.

    Good powered reference monitors aren't cheap, but they aren't insanely gratuitously expensive either. If accuracy was what these people really wanted, that's what they need to get.

    And if they think they'll do better with equipment that is "better" than the monitoring equipment the engineers at the studio used to create it... just walk away. They are so full of shit and so far gone you'll never get any logic through their head.

  11. Wait. by pclminion · · Score: 2, Insightful
    They asked people to rank things on a subjective scale, and then they did statistics with the results? I'm sorry, but that's a scientific faux-pas. The moment they tried to compute a standard deviation, they became pseudoscientists. Sorry, the "hard" stats are worthless. The only even possibly meaningful thing would be a histogram of the rankings.

    Just to make it clear why you can't do this: There is a certain incremental difference in subjective quality, that will cause listener A to rank something 4 instead of 3, for example. Will that same incremental difference in subjective quality case listener B to also rank something 4 instead of 3? We don't even know if the scale is linear, much less whether the different increments are even the same for different listeners.

    Example: on the Kuro5hin blog site, if I really like a comment I will rate it 5. If I hate it, I will rate it 1. I never use the ratings in between. Thus, sometimes a 1 is bad, and sometimes a 1 is really bad. The same could happen here. Even if different listeners perceive similar subjective changes in quality, they might assign different changes in rank to those changes in quality.

    God, this entire thing is bullshit, now that I think about it.

    1. Re:Wait. by pclminion · · Score: 2, Insightful
      Are you saying that you can't perform statistical analysis of subjective data?

      Yep. Here's my 1-5 scale:

      1: Base ranking
      2: 1.668 times better than 1
      3: 5.881291 times better than 2
      4: 2.5877 times better than 3
      5: 100000 times better than 4

      Except for when they play that certain song I like, in which case my entire ranking system is subconsciously skewed. In fact, if they play Metallica, I will rank it 2 no matter how subjectively good it sounds. Because Metallica just sounds like shit.

      And here's Johnny's scale:

      1: Base ranking
      2: 2.4 times better than 1
      3: 1000 times better than 2
      4: 16.8523 times better than 3
      5: 1.0000001 times better than 4

      Would you like to see Suzie's scale? Do you see why this is bullshit yet?