Audio Format Listening Tests Concluded

Pointing to the conclusions of this listening study, nullity writes: "The results are interesting, and show a high variation in the performance of the various codecs on different musical styles. Ogg seems to work well on dance music, WMA8 on chamber music, etc."

Comments

Re:Didnt read the article yet but....

[mccalli]

it seems that music you listen closely too sounds better with WMP, and fast, not listened to music sounds ok w/ ogg.

Well....not quite. There's a different frequency distribution between electronic, pop acoustic and classical music.

Specifically, electronic music, which most dance stuff is, has a very flat frequency distribution. See this for yourself - load your favourite media player, siwtch on the graphic equaliser graph and watch how basically nothing happens except in the mid-range.

Now try again with an orchestral piece. There will be much more variation, though in most it will tend towards the top end.

Now try again with rock. Tends towards the bottom and top, with middle frequencies missing.

Keep going with any format you feel like mentioning...you'll get the same.

Actually, this is a striking example of how recording techniques can ruin sound as well. Take a look at the Apollo 440 album - Gettin' High on Your Own Supply. A good mixture of guitars and electronics, right? Well, look at the frequency graph again. See how virtually every guitar frequency variation has been cut out: this music was recorded digitally, mostly using samples by the looks of it. The normal variations you'd associate with having guitars play live are all filtered out, and the graph goes back to the flat digital sound again.

Cheers,
Ian

Re:No Control

[Anonymous Coward]

Here, there is no control - crappy experiment.

The participants could have just been scoring on "this is different to the unencoded track, therefore it must be worse".

So put a copy of the unencoded track as a test track and see if it gets marked down (and also, of course do NOT tell the participants that it is there).

Umm.. did you bother to read about the testing methodology before coming to your grand conclusion here?

ABC/HR.. as in ABC/Hidden Reference... as in, there is a copy of the original track included as a hidden reference on every single trial.

The users are given 2 sliders per sample laid out on a panel. The samples are loaded in random order. On the sliders for each sample, one slider is for the original sample, and one is for the encoded. These are also randomized per sample. The user does not know which is which. If they happen to rate the original sample less than 5.0 (highest rating, meaning it should be transparent), then their results are disregarded entirely for that sample.

Re:Outiers skewing the results?

[jonathan_ingram]

If you read the thread on HydrogenAudio (which is the message board where most of these tests / codecs are discussed), you'll find the following information from Monty, the lead developer of Vorbis:

Ogg had a very low bitrate (in the forties) on all the classical samples, which is the way it should have been (Classical solos with their deep noise floors and simple harmonics are relatively easy). But the real reason Ogg scored so low in both (and Beauty Slept as well) was a) the tuning behind noise normalization is still not perfect. This is the very first release of that feature, and the test found flaws b) also the first release of new, more aggressive stereo modes and I think that they too need more analysis infrastructure driving them.

I expect Ogg's performance on Liszt and Bach to be very subpar NN performance. The poor performance on BeautySlept and Waiting was most likely insufficient stereo analysis. Ogg had the infrastructure to win those four samples, but the encoder didn't know how to do it yet (because I didn't know it would be necessary).