Slashdot Mirror
Audio Format Listening Tests Concluded
Pointing to the conclusions of this listening study, nullity writes: "The results are interesting, and show a high variation in the performance of the various codecs on different musical styles. Ogg seems to work well on dance music, WMA8 on chamber music, etc."
In some circles, it's believed that Ogg Vorbis is the future of lossy format music. You get higher quality in less harddisk space than with MP3, plus it's a far more open codec allowing more customisation with less legal risk.
Steven Woston
Lead Programmer, J-j-j-julius SoftwareI guess grip will have to use Genre info from CDDB to decide what to encode the the files as now. I wonder if you coudl set up something to optimize individual tracks. Like scan a wav and pick the best codec for the frequencies used in the audio.
Why not fork?
64Kbps is where the flaws of a codec are truely exposed. It's a great median between being too high to produce much results and too low where everything completely falls apart. You may not think any of this has any relevance to you as you're encoding above 128Kbps, but it actually does make a difference when you stress your encoder with a difficult piece of music.
However, if the difference between sounding 'good' and sounding 'accurate' mean little to you, as someone who'd make an argument of 64Kbps tests being worthless would, then you really aren't the intended audiance of such tests. You can merrily use any of those encoders at 128-192Kbps without ever really noticing or caring much.
I, personally, would like to see OGG1.0, MP3 Pro and WMA8 take on some real tough to beat codecs such as Dolby's AAC High-Complexity Mode (which no AAC freely available encoder supports, including QuickTime) and Sony's ATRAC3. But, that'd be kinda moot, because most people out there do not have access to those toys.
For now, I'm content to just watch people hop around and proclaim whatever they want as king of audio formats while sticking to 256Kbps Fraunhoffer MP3 (archival purposes) and 192Kbps LAME HQ MP3 (general usage) as something both widely supported and pratically indistinguishable from the source. Even if AAC-HC and ATRAC3 were freely available, it'd take an awful large effort to wean people off of MP3 so far as support base and to migrate them to a new format. New P2P programs, new players/plug-ins (in some cases) and new hardware players. Not gonna happen for a while.
I noticed a number of confused posters here... The tested codecs were AAC/MP3PRO/OGG/WMA, not MP3. Had mp3 been tested, it would have lost every round as all of the tested codecs are vastly superior to plain MP3 at this bitrate.
It also should be noted that the only two samples that WMA beat OGG at (indeed the only ones that it didn't totally flop on) were two very simple samples that are demonstrations of two differnt weaknesses in the current revision of vorbis. Orignally the results page had some very interesting commentary from Monty on this, but it looks like it got pulled.
With the exception of those two samples, OGG clearly won. Even including those, it was only beat out by MP3PRO by a small margin. When you factor in that MP3PRO isn't available at anything but such low bitrates and that it's substantially more propritary then MP3, it seems like pretty much a no-contest.
How would the codecs compare in bitrate given a minimum quality requirement. Say eg at least 99.9% of the samples produced when decoding must match the original wave with 99.9% accuracy, at what bitrate can this requirement be met by the various codecs (which is smaller). And a nice graph of some sort for various musical styles.
Considering that different codecs do better at different music w/ different frequency spreads, who else thinks that the next generation of audio codecs will be multi-modal; in effect, be several codecs in one. Then have each codec specialize on certain types of music. Perhaps even have them run in an advanced mode where they do a frequency analysis of whole songs, rather than just using genre, to automatically select the best codec for the job. Perhaps even use different codecs for different sections of the song. That would definitely help songs like Bohemian Rhapsody and orchestas with movements, etc.
Would this be too time consuming to implement or what?
BlackGriffen
Just a side note about the frequency distribution of different styles of music:
The reason why classical music generally compresses better is because the frequency distribution of the sound of natural instruments like for instance string instruments (including the human voice) is harmonic. This means that the sound spectrum consists mainly of a superposition of peaks at the base frequencies of the instruments played and their corresponding harmonics at higher frequencies.
If you were to make a two dimensional spectral analysis of a such sound recording with the time axis to the right, the frequency to the top and the amplitude as the color intensity of the point you would see a lot of wiggling lines at
regular distances. (BTW: this would make a great visualization plugin for xmms)
Since audio compression algorithms also make such a spectral analysis and after that discard some of the information below a threshold they can
reproduce a mainly harmonic spectrum easier than that found in pop or rock music, which is much more complex and more "noisy" because of the
use of distorting amplification and all kinds of
percussion.
Holger
Looking at the data, it looks the two samples where Ogg performed poorly ended up being encoded at a significantly smaller average bitrate than any of the other encoders.
The table at the end lists LiszBMinor with an average ogg bitrate of 45 and BachS1007 with an average bitrate of 47. Since the other codecs encoded those samples at a bitrate 64 or higher, this may explain the results.
The results may point to a flaw in Ogg's VBR login rather than in the lossy compression scheme.
OVERALL RANKINGS (12 SAMPLES)
mp3pro 49.00
oggq0 44.00
ogg64 40.00
wm8 24.00
aac 23.00
The AC above me speaks the truth. mp3PRO has no hope of gaining enough market share to become a worthy competitor. It's a very proprietary extention to MP3. OGG being open source and free (as in beer) has clear advantages for hardware vendors (where it really counts). Lets hope the codec is easy to embed into portable products.
I want my Portable OGG CD Player! I'll buy the first one that comes out. Could you imagine? Twice the capacity of normal players and it STILL sounds better (or same capacity truly indistinguishable from CD -- at only 128k). Right now I have to encode my mp3's at ~180-220kbit to get something acceptable. =/
I haven't seen anyone else mention this yet. At the end, he gives a table of the bitrates for each song for each codec. The one with the greatest variation appears to be oggq0. I noticed that for the songs where that codec did well, the bitrate was much higher, and where it did poorly, it was much lower. I don't realy understand how the bitrate is chosen, but as I understand it, the encoder chooses it automatically somehow, right? I wonder how effective that really is.
I had several obvious choices for the AAC encoder: Psytel, the Quicktime, and Liquid Audio 5 (I hadn't looked into LA6). Liquid Audio 5 is another FhG low complexity encoder, but lowpasses at a lower frequency than the Quicktime. The Psytel encoder is worse-sounding than the Quicktime at 64 kbit/s. I did try to choose the best AAC implementation available to me (I do not have access to the latest and greatest implementations).
It's possible I could have set up the experiment as a Latin Square, and randomized which codecs any individual was comparing, but my home-grown statistical tools are not up to that task. That is, I can only perform balanced analyses, where N is the same for every codec.
ff123