Slashdot Mirror
AAC Put To The Test
technology is sexy writes "Following the increasing popularity of AAC in online music stores and the growing amount of implementations in software and hardware, the format is now being put to the test. How well does Apple's implementation fare against Ahead Nero, Sorenson or the Open Source FAAC at the popular bitrate of 128kbps? Find out for yourself and help by submitting the
results. You can find instructions on how to participate here. The best AAC codec gets to face MP3, MP3Pro, Vorbis, MusePack and WMA in the next test. Previous test results at 64kbps can be found here."
I note that in the 64 Kbps test, they used the AAC-LC encoder from QuickTime 6.0. This was a pretty darn lousy one, lacking any ability to specify a sample rate at a given data rate, and had poor quality. The current version of QuickTime 6.3 (for Windows and MacOS X), has a much improved, more flexible AAC-LC encoder, so if they did that test today AAC would likely rank higher.
If using the Apple encoder, encode in "Better" mode with 16-bit source, and in "Best" mode with source that's more than 16-bits per sample (and hence isn't a CD rip). Support for mastering from 24-bit when running in "Best" is one of the reasons why the AAC-LC files as part of iTunes sound so good.
My video compression blog
Not possible. MP3 by its very nature is a lossy encoding scheme, hence there will always be artifacts when you pass the audio through the encoder. You may not be able to hear the quality change (even after passing the files over and over and over through the encoder) but you will be generating noise.
As far as your original question, it all comes down to file portability. It takes people a bit longer to send a 65 meg wav to their friends, compared to a 6.5 meg mp3.
There *are* lossless codecs like FLAC and SHN, but they generally achieve between 10 - 30% compression.
Religion is a gateway psychosis. -- Dave Foley
Granted it was probably mostly marketing bluster, but Steve Jobs did claim that Apple is encoding the original master recordings when they're available.
You tell me how "whilst" differs from "while," and I'll stop calling you a pretentious jackass.
With 6000 participants, the double-blind public test results were:
- Ogg
- WMA
- RealAudio
- Mp3Pro
- MP3
- AAC (Sic!)
Of course, this was crazy, with AAC even behind MP3, but these really were the results...First, read this:
.zip files.
http://www.ff123.net/abchr/abchr.html
This describes the program and testing methodology used here, which, btw, is based on widely accepted perceptual testing conventions. And yes, by the scientific community. These are the same techniques used by the scientists that do the research and development on these formats. Please note the references at the bottom of the page.
1. Wrong, the MP4 files are already encoded and created for the user, stored in the
2. Wrong, the Hidden Reference (ABC/*HR*, please read the page at the first link), ensures that if the user honestly cannot tell the difference but thinks that one exists (placebo), and rates the original lower than one of the encoded versions, that their results are discarded.
3. This is where the statistics come in. With enough listeners, the "noise" gets weeded out of relevant results. Most past tests using this methodology have been shown to provide highly relevant and fairly uniform results when all the data is factored together.
An open call to the masses is the only way to measure the perception of the masses, and if the test is performed properly (which it is in this case), then it *is* scientific.
Next time, please read up a little more on what is happening before jumping to all sorts of incorrect conclusions.
As was previously mentioned on Slashdot, a highly regarded German magazine called C'T dedicated an article to a similar comparison of various audio compression codecs last year.
.WAV recordings containing 3 short excerpts from various CD music tracks (pop, classical and jazz) that had previously been encoded by 6 popular codecs, each at both 64 Kbit/s and 128 Kbit/s (or as close as possible for VBR-only encoders). For verification of the results, 2 of the recordings came directly from CD and had not gone through any encoding process. Because the .WAV files were all the same size, there was no way for the listener to know which encoder had been used on a particular file. Participants were asked to rank their preferences among these files. The encoders included MP3, MP3PRO, Ogg Vorbis, WMA, RealAudio and AAC.
They created fourteen different
Over 6000 people downloaded those tracks and submitted their preferences. Unfortunately, the results of that test were only published in print and I haven't been able to find an online version of it. A few noteworthy results are below however.
The percentages indicate how many people put a particular codec at a particular ranking:
MP3 64 KBit/s
1st place: 1 %
2: 1%
3: 1%
4: 1%
5: 2%
6: 4%
7th place: 90%
As might be expected for the oldest codec, almost everyone agreed that the file that had been run through MP3 at 64 Kbit was the worst sounding of all. At 128 KBit however, listeners were clearly divided on whether MP3 sounded worse or better than others:
MP3 128 Kbit/s
1: 11%
2: 14%
3: 15%
4: 15%
5: 16%
6: 16%
7: 14%
Now the AAC results. At 64 Kbit, it was ranked a slightly below average performer:
AAC 64 KBit/s
1: 7%
2: 12%
3: 17%
4: 26%
5: 22%
6: 14%
7: 2%
What's interesting is that at 128 Kbit/s, more people ranked AAC the worst sounding encoder than any other codec in the test including MP3!
AAC 128 KBit/s
1: 11%
2: 11%
3: 13%
4: 12%
5: 14%
6: 14%
7: 26%
Not surprisingly, the files that had been read directly from CD without any encoding steps done in between got the best rankings of all. Ogg Vorbis did very well indeed and came in second overall.
This is what the cdparanoia faq has to say about ripping...
I can play audio CDs perfectly; why is reading the CD into a file so difficult and prone to errors? It's just the same thing.
Unfortunately, it isn't that easy. The audio CD is not a random access format. It can only be played from some starting point in sequence until it is done, like a vinyl LP. Unlike a data CD, there are no synchronization or positioning headers in the audio data (a CD, audio or data, uses 2352 byte sectors. In a data CD, 304 bytes of each sector is used for header, sync and error correction. An audio CD uses all 2352 bytes for data). The audio CD *does* have a continuous fragmented subchannel, but this is only good for seeking +/-1 second (or 75 sectors or ~176kB) of the desired area, as per the SCSI spec.
When the CD is being played as audio, it is not only moving at 1x, the drive is keeping the media data rate (the spin speed) exactly locked to playback speed. Pick up a portable CD player while it's playing and rotate it 90 degrees. Chances are it will skip; you disturbed this delicate balance. In addition, a player is never distracted from what it's doing... it has nothing else taking up its time. Now add a non-realtime, (relatively) high-latency, multitasking kernel into the mess; it's like picking up the player and constantly shaking it.
CDROM drives generally assume that any sort of DAE will be linear and throw a readahead buffer at the task. However, the OS is reading the data as broken up, seperated read requests. The drive is doing readahead buffering and attempting to store additional data as it comes in off media while it waits for the OS to get around to reading previous blocks. Seeing as how, at 36x, data is coming in at 6.2MB/second, and each read is only 13 sectors or ~30k (due to DMA restrictions), one has to get off 208 read requests a second, minimum without any interruption, to avoid skipping. A single swap to disc or flush of filesystem cache by the OS will generally result in loss of streaming, assuming the drive is working flawlessly. Oh, and virtually no PC on earth has that kind of I/O throughput; a Sun Enterprise server might, but a PC does not. Most don't come within a factor of five, assuming perfect realtime behavior.
To keep piling on the difficulties, faster drives are often prone to vibration and alignment problems; some are total fiascos. They lose streaming *constantly* even without being interrupted. Philips determined 15 years ago that the CD could only be spun up to 50-60x until the physical CD (made of polycarbonate) would deform from centripetal force badly enough to become unreadable. Today's players are pushing physics to the limit. Few do so terribly reliably.
Note that CD 'playback speed' is an excellent example of advertisers making numbers lie for them. A 36x cdrom is generally not spinning at 36x a normal drive's speed. As a 1x drive is adjusting velocity depending on the access's distance from the hub, a 36x drive is probably using a constant angular velocity across the whole surface such that it gets 36x max at the edge. Thus it's actually spinning slower, assuming the '36x' isn't a complete lie, as it is on some drives.
Because audio discs have no headers in the data to assist in picking up where things got lost, most drives will just guess.
This doesn't even *begin* to get into stupid firmware bugs. Even Plextors have occasionally had DAE bugs (although in every case, Plextor has fixed the bug *and* replaced/repaired drives for free). Cheaper drives are often complete basket cases.
Rant Update (for those in the know):
Several folks, through personal mail and on Usenet, have pointed out that audio discs do place absolute positioning information for (at least) nine out of every ten sectors into the Q subchannel, and that my original stateme
Do you even lift?
These aren't the 'roids you're looking for.
It's rather obvious that you haven't bothered to read anything about the test, the program used, etc.
The test *is* blind, and it is based on widely accepted perceptual testing techniques. It uses hidden references (references to the original vs the encoded sample, on a per sample basis in which the user is not aware of which is which, thus if they rate the original as being worse than the encoded version, their result is discarded) as a control. The program devised has been developed by someone who has taken the time to do the proper research, read the appropriate papers and other sources, discuss the idea with developers of many different audio codecs (LAME, Vorbis, PsyTEL AAC, etc). The technique here works, and has been used many times before. It's not simply some amateurish scheme that someone who knew nothing about the appropriate sciences dreamed up simply because he wanted to find out if "Person A liked Audio B".
The complete results can be found in issue 19/2002 of Heise's offline magazine C't. Along with the online public test, some 'experts' (such as some music producers, hobby listeners, a singer, and a young student and choir singer) were consulted.
In the online public test, the 64 kBit/s comparison yielded
The parent's results were the ones for 128 kBit/s. The eight experts compared the codecs on 160 kBit/s as well, with much more varying results (not much of a surprise). But on average, the results were
As I said, those were an average, with the individual results of the eight experts strongly deviating. Ogg was placed once 1st, once 2nd, twice 3rd and 4th, and once 5th and 7th. (One had actually placed the plain wave reference 5th...)
I disagree on the 160 vs 256 kbps statement. I listen to mostly rock and punk, so I took a Thursday song, which is kindof in the middle of the two genres, and encoded it at 32, 48, 56, 64, 96, 112, 128, 160, 192, 256 and 320 kbps. I wanted to encode my whole CD selection (350 cds) at a bitrate that I couldn't hear the differance, and a bitrate that I could stream at decently. For streaming, 56 was the magic number. Any less and it sounded like crap, any more, and my DSL line couldn't host 2 streams at once. For music, 192 was good, but I could still hear the mp3 compression. I find that bass tends to get distorted in mp3s, and once I went to 256 this seemed to go away. I did all these tests with an audigy2 under windows XP, using Lame with q=9. Playback was through the Infinity HTS-20 Speaker System.
-- the computer doesn't want any beer, no matter how much you think it does. NEVER, EVER feed your computer beer.