Vorbis And Musepack Win 128kbps Multiformat Test

technology is sexy writes "After 11 days of collecting results Roberto Amorim today announced the results of his 2nd Multi-Format listening test: Vorbis fork AoTuV scored the highest and ranks as the winner together with open source contender Musepack closely followed by Apple's AAC implementation and LAME MP3, which improved markably since last year thanks to further tunings of its VBR model done by Gabriel Bouvigne. Sony's ATRAC3 format ranks last after WMA on the third place. The suprising success of AoTuV (compared to last year's performance of Xiph.org's reference implementation) shows the potential of Vorbis and possible room for further tuning and improvments. Take a look at the detailed results and their discussion at Hydrogenaudio.org."

Comments

Striving for innovation

[MrIrwin]

So given Microsofts stated goal to bring us innovative technology, they should throw in the towel and ship OggVorbis and derivatives with Windows, right?

Re:Striving for innovation

[Anonymous Coward]

its important to point out two things especially:
WMA brought clearly worse quality than (good old) MP3 at 128kbps

itunes AAC brought clearly better quality than WMA at 128kbps

so why should anyone even a minute consider buying crap quality wma encodes at napster, coca-cola, walmart or however the wma-based music stores are called?

on the legal way -> itunes is better
on the illegal way -> even old mp3 (next to vorbis or aac) is better

Re:Striving for innovation

[Anonymous Coward]

The Microsoft-published PC version of Halo uses Ogg Vorbis for all of its audio - so it's not as if there hasn't been a precendent. :-)

Re:Striving for innovation

[Anonymous Coward]

so why should anyone even a minute consider buying crap quality wma encodes at napster, coca-cola, walmart or however the wma-based music stores are called?

Because many people can't discriminate the difference between a cannon shot and a muffled fart and will take whatever crap MS feeds them.

Re:Striving for innovation

[badasscat]

on the legal way -> itunes is better
on the illegal way -> even old mp3 (next to vorbis or aac) is better

Illegal?? How is ripping my own CD's to MP3 illegal?

I have ripped all of my CD's to 320k max VBR MP3's using LAME (with EAC as a front-end). There's nothing illegal about this, and based on this listening test I'm quite confident that all of my music sounds at least as good, and probably better (in some cases probably significantly better) than if I'd re-purchased those same songs through iTunes (or ripped them to 128k AAC). And I have max compatibility among devices, with no DRM.

If this test proved anything, it's that Apple's and MS's claims that their codecs sound better than MP3's recorded at twice the bitrate is a load of bunk. All the codecs are at least comparable at the same bitrate, but a few are a little bit better than the others. In the end, for me it comes down to compatibility. I have no idea if the next player I buy will be an iPod, so why would I want to tie myself down to that player, especially when other codecs sound just as good, bit for bit?

btw, I tried taking this test, and honestly, I gave up because most of the time I could not even tell which was the original and which was the compressed version of the song. In my opinion any of these codecs yields more than acceptable sound quality. I see in a few cases one or another codec was significantly worse than the others on specific tracks (I probably just didn't get that far) but in most cases the high scores here (in the high 4's for the most part) show that all of these codecs do a good job at producing compressions nearly indistinguishable from the original even at 128k. Given a higher bitrate, they'd do just that much better.

But does it matter?

[Gothmolly]

When everyone gets an iPod, dood, or the WinFooTunes player that you get with your Dell only works with WMA, or your in-dash CD player only groks 128kbps MP3s, whats the practical application of the other codecs? It's nice that we propeller-heads on Slashdot can smirk while we rip everything to FLAC and write custom Perl apps to transcode-on-the-fly to our wireless enabled MythTV box, but for John Q. Drone^H^H^H^H^HConsumer, none of this matters.

So how do we get the word out? How do we start the revolution? Open-Source hardware?

Re:But does it matter?

[Liquid-Gecka]

I don't think we have to "get the word out." Most cool tech innovations make it into the mainstream if they are really good enough. I remember buying a CD-RW (4x2x2) for $400. Everybody thought I was stupid for spending so much on a piece of hardware. Later I spend $300 on a 64MB MP3 player. The guy at the desk told me that I shouldn't get a MP3 player because changing the media was really hard. Yet most people now have both of these gadgets. If Vorbis is license free and simple enough to put on a hardware chip then it will slowly gain support and slowly people will begin to see it.

Re:But does it matter?

[mukund]

You have a point. There are devices however such as from iRiver which can play Ogg Vorbis and the winning encoder/codec in the Slashdot story AoTuV seems to be just an encoder fork which is bitstream compatible with Ogg Vorbis.

Re:But does it matter?

[millahtime]

There is a difference to this. At the time there was no alternative to the CD-RW. There are many compressed sound types that are being marketed.

Re:But does it matter?

[Liquid-Gecka]

At the time there where many large format media types available. Tape, Jazz, Zip, portable HD all where in there prime. CD-RW discs couldn't be read in most CD-R drives and most earlier audio systems wouldn't read CD-R discs. There wasn't a big advantage to buying a CD-RW drive over other solutions. Zip drives where $100 and the discs where $2, when I bought my CD-RW discs where like $20 and CD-R discs where $.50, or $1.25 with a jewel.

Re:But does it matter?

[ErroneousBee]

Vorbis's problem is that it suck batteries dry in about 2/3rds the time it takes MP3 to do it. Thats only based on my iRiver, but I think the Rio Karma gets similar figures.

But what do I know, my first player was a POS Nex ia.

Re:But does it matter?

[Anonymous Coward]

Sooner or later, Venture capitalist will start to pick support of OSS projects. In fact, I am guessing that in about a years' time, one them will have the bright idea to create a contest to fund the best ideas. At that time, we will see a number of OSS ideas and projects make it big.

Re:But does it matter?

[true_majik]

oohhh yes, i remember the day my brother and i bought a cd-r/rw...close to $500. we polled our money and saved enough to get the RW. at first, there WERE separate cd-r and cd-rw models to choose from. now-a-days burners are RW as a standard.

Re:But does it matter?

[PyromanFO]

I play Ogg pretty exclusively on my Karma and I get 15 hours of battery life.

Re:But does it matter?

[ErroneousBee]

This says 10-11 hours at -q7 versus 15-17 hours of (admittidly lower bitrate) MP3.

I guess its the higher bitrates that are the killer. iRiver force high Ogg bitrates by having a 96kbps floor, so many -q3 and even some -q4 encodings wont play.

Re:But does it matter?

[D. Book]

There are devices however such as from iRiver which can play Ogg Vorbis

I was one of those nincompoops who rushed out and bought one the moment I read the words "iRiver" and "Ogg" in the same sentence, but when I updated its firmware the latest version with Ogg Vorbis support, I found that many of my files wouldn't play.

It turns out that most of the iRiver players with Ogg support added have a half-baked implementation and support only a limited range of bit-rates and frequencies. The iFP-300 series, to which my player belongs, only supports 96Kbps - 360Kbps (if it's a VBR file and the bitrate drops above or below that, distortion occurs), and also has trouble with files encoded in less commonly used frequences (i.e. lower than 44.1KHz).

In case anyone like me thought iRiver was committed to improving their Ogg support, their latest iFP series players are even more limited, supporting only 96Kbps - 225Kbps at 44.1KHz. Their new iDP series doesn't support Ogg at all. And owners of the iMP have been waiting months for Ogg support which still has not materialised.

Only the H series supports a decent range of Ogg Vorbis bitrates, but even it only officially supports one frequency (44.1KHz).

Re:But does it matter?

[Darren Winsper]

I beg your pardon?

Re:But does it matter?

[D. Book]

My mistake. But checking the forums and firmware pages, it appears that the iMP's Ogg support is even more limited than my iFP -- 96-256Kbps only (presumably with problems playing non-44.1KHz files, if the other players are any indication).

Re:But does it matter?

[Anonymous Coward]

all where in there prime

"all were in their prime".

where ... where ... where

"were" ... "were" ... "were".

I think I speak for everyone...

[Anonymous Coward]

... when I ask, "Technology is Sexy? WTF?"

Re:I think I speak for everyone...

[Jesselovesscripts]

no, you don't. your new to slashdot huh?

Re:I think I speak for everyone...

[RumpRoast]

"you're" you twit.

mp3 still defacto standard

[Random Web Developer]

No matter what researchers find the best format, the best format for users is what they can doubleclick to play, use on their el-cheapo portable mp3 player or whatever music device they own.

This might be of interest to musicians but the proverbial "jane doe" will keep using mp3 for quite a while

Re:mp3 still defacto standard

[Roland Piquepaille]

This might be of interest to musicians but the proverbial "jane doe" will keep using mp3 for quite a while

Actually it's not that simple. Jane and Joe Doe will start using Ogg, AoTuV or other TLA and ETLA compression schemes when their favorite music players feature them. In the case of Ogg, it's not going to happen anytime soom because:

1 - There's an entrenched MP3 market, as you said

2 - It's an open-source format, i.e. it reeks of piracy and hackers in the minds of music player manufacturers and of the public

3 - It doesn't have the backing of major industry players, being seen as a "maverick" effort to undermine other potentially money-making closed-source formats

4 - It certainly doesn't have the backing of the RIAA, because it doesn't have DRM and other in-the-customer's-face copyright protection schemes

In short, people using Ogg will be opensource-aware and advocates for a long time to come. As for other Apple customer-unfriendly sort of schemes, I'm not convinced the general populace has bought into the idea of paying for music tracks that can become unplayable at the next Apple format-change-du-jour, because they're copyright-protected and therefore impossible to convert to another standard (in theory).

So yes, you're right, MP3 will stay around for a long time. I certainly won't convert my collection anytime soon...

Re:mp3 still defacto standard

[minus_273]

are you kidding? i can click a button on itunes and remove all DRM from the music. It is called burning to CD...

Re:mp3 still defacto standard

[Phekko]

Winamp already plays Ogg out of the box. So do the Open Source counterparts (Zinf and XMMS do at least) but still MP3 seems to be the thing.

For me MP3 is the preferred format because my car stereo plays it but not Ogg. It is a couple years old and I don't intend to replace it because a) it works and b) I don't have the money. Had there been an Ogg-enabled car stereo around at the time I bought it things would be different.

The whole MP3 vs. Ogg conversation reminds me of VHS vs. Beta vs. V-2000. Which one wins has little to do with which one is best.

Re:mp3 still defacto standard

[Anonymous Coward]

Why would you "convert" your collection in the first place? All that would result in is a loss of audio quality as it's recompressed in another lossy format.

Unless you're suggesting that you actually own the CD's from which your "collection" is made, which I somehow doubt.

Re:mp3 still defacto standard

[RAMMS EIN]

``2 - It's an open-source format, i.e. it reeks of piracy and hackers in the minds of music player manufacturers and of the public''

And you think they care? These are the people who get all upset when someone sues them for stealing music, and the people who write the software used to play stolen music. Believe me, the public does not have an aversion of piracy, nor of open source. They just don't know it, and it's the fear of the unknown that keeps them away.

Re:mp3 still defacto standard

[Zilch]

2 - It's an open-source format, i.e. it reeks of piracy and hackers in the minds of music player manufacturers and of the public

Yeah - because piracy and hackers didn't have a hand in making MP3 popular.

Zilch

Re:mp3 still defacto standard

[alex_tibbles]

Grandparent noted the existence of 'cheapo' MP3 players. There is a good reason why OGG could gain mainstream acceptance: royalty-free use. The MPEG license will eventually be a significant part of the cost of the chips in a portable music player (if not a significant part of the overall cost of the player), but still a saving is a saving...

Re:mp3 still defacto standard

[molarmass192]

What you're describing isn't "click a button on itunes and remove all DRM from the music". It's more like:

1) Make a playlist
2) Burn playlist to CD
3) Rip playlist from CD
4) Reorganize your ripped files

You also pay a price in quality by re-encoding a low bitrate file on ripping. That said, there's always PlayFair to get rid of that pesky DRM in one nice step, not a click though.

Re:mp3 still defacto standard

[fantastic max]

It really irritates me whenever there's an article about audio codecs and everyone ends up comparing mp3: vorbis as vhs:beta and citing that the better technology is not necessarily the winner. But just think about why the better technology wasn't the winner with beta. It had to do with a monopolistic control over beta and high prices. Think about the economics of the late 70s and its clear that the cheaper technology would win. And that is why you can't compare Beta to Vorbis.... Because vorbis is free! Vorbis is the better technology and its the most economical. Most people don't realize the added cost associated with having an mp3 capable player along with software that can encode. Royalties go somewhere, you know. Anyway, mp3 has the edge because it is the long standing de facto standard and no one gives a crap about vorbis because they just don't know about it. I am a Rio Karma owner because I do know about vorbis and FLAC. And my FLAC and vorbis library runs fantastically on my Linux and Windows boxes without additional whoring of myself to another corporate cartel in the mdeia industry aside from the RIAA that provided me with most of my CDs.

Re:mp3 still defacto standard

[Anonymous Coward]

You forgot the part about Ogg Vorbis being an asinine name.

Re:mp3 still defacto standard

[timeOday]

...closely followed by Apple's AAC implementation and LAME MP3, which improved markably since last year thanks to further tunings of its VBR model done by Gabriel Bouvigne.

You see? The benchmarking has already resulted in better mp3 encoding, which is still compatible with all those devices people own. So even if mp3 can't be dethroned, these tests benefit everybody.

Re:mp3 still defacto standard

[Phekko]

I'd buy your argumentation if Linux was dominating over Windows. After all, it is more cost effective and also technically superior.

If you prefer the comparison of Microsoft Office vs. OpenOffice, fine, I'll go along with that. The point is still the same.

Re:mp3 still defacto standard

[damiam]

Compatibility. I could bring out a new format infinitely superior to VHS/DVD, and it would fail miserably if it didn't have the support of the content producers. If no movies were released in my format, then no one would want it regardless of how good it was.

The same applies to OSs. No matter how good Linux is, if it doesn't have the apps, then people don't want it.

Re:mp3 still defacto standard

[Phekko]

Ah yes. But how, then, is not OpenOffice more succesful? It is free and it is Microsoft Office compatible, right?

Re:mp3 still defacto standard

[damiam]

It's not fully compatible, and it doesn't supply the same featureset. While OO.org is great, it's generally speaking not as good as MS Office. Sure, it could meet the needs of 95% of people, but it's still glitchy opening some Office documents - pretty much a show-stopper.

I know, there are times when it will open corrupted documents that Office won't, but the fact is that it doesn't open everything that Office does 100% perfectly - and that's what's needed for it to work in a heterogenous environment.

Re:mp3 still defacto standard

[glitchvern]

3 - It doesn't have the backing of major industry players, being seen as a "maverick" effort to undermine other potentially money-making closed-source formats

It is being used in games due to its royalty-free status. Always remember one person's money-making is another person's cost.

Re:mp3 still defacto standard

[benwaggoner]

And if I might add another

5 - the compression efficiency difference isn't THAT big.

If Ogg was 25% the size of a MP3 of the equivalent quality, it'd be a slam dunk. But if it's 80% of the difference, that's not really enough of a motivation to switch. MP3 is widely compatible, and "good enough" for a wide variety of applications.

Now, if we ever get widely available parametric stereo versions of HE AAC that you can dance to at 24 Kbps, things might change...

Re:mp3 still defacto standard

[hc00jw]

Yeah - because piracy and hackers didn't have a hand in making MP3 popular.

The rules have changed now. The playing field in no longer in the hackers back yard, it's in the RIAA's.

Re:mp3 still defacto standard

[Anonymous Coward]

No matter what researchers find the best format, the best format for users is what they can doubleclick to play, use on their el-cheapo portable mp3 player or whatever music device they own.

The best format for me is what I can use in my QUALITY AND VERY EXPENSIVE portable mp3 player, thank you very much. I suppose a little tact is too much to ask.

Re:mp3 still defacto standard

[mattyrobinson69]

1) write to isofs (stored in tmpfs partition for speed)
2) mount isofs
3) rip songs from mounted iso
4) listen without drm
5) (profit????)

you also wont be wasting time waiting for the cd to burn/rip. the burning will only take as long as writing to memory/swap

Re:mp3 still defacto standard

[mattyrobinson69]

the mp3 license only requires royalty to be paid for encoders, not decoders.

it is lame that is a bit iffy, legally, mpg321 is fine

Re:mp3 still defacto standard

[mattyrobinson69]

ive never had open office fuck up a file that it saved itself

MS Office on the other hand (office XP) fucks up all the assignments at my college (the boxes are always in the wrong place). this is probably something to do with the templates that the tutors use. This was with MS Office XP installed on every machine on campus

Re:mp3 still defacto standard

[alex_tibbles]

good point.
so my point is only relevant where portable music player makers want to ship software that can take audio from any source (eg. CD) and put it on the player. this will involve an encoder.

moo

[Anonymous Coward]

moo

Re:moo

[Anonymous Coward]

Offtopic? If you understood cow, you would realise that parent is both on topic and informative.

If you don't understand a post, don't moderate it.

The Big Marketing Push

[millahtime]

The open source ones don't have the big push amungst the general population. So, number 3 on the list Apple (ACC) can say in independent tests ACC scored higher than WMA or MP3. The top 2 don't have the marketing push to get out and be popular in the general population.

This does give more fuel to Apple. Although I'm not complaining about them having fuel over Microsoft.

Re:The Big Marketing Push

[Anonymous Coward]

"The open source ones don't have the big push amungst the general population."

I don't see why not, I hardly notice what the extention is when I play music files.

Since Ogg is open source though I encode only in that format. If more people were to do this it would catch on.

Re:The Big Marketing Push

[millahtime]

Yes, but WMA and ACC have the push of Apple and Microsoft. Marketing push out weighs how good it is to most people since most people can't tell the difference.

Re:The Big Marketing Push

[jafuser]

Well, to be fair, both WMA and AAC were slightly ripped off on bitrate since they stuck excactly to the required bitrate.

Average Bitrates
128 AAC
136 MPC
135 Vorbis
134 Lame
128 WMA
132 Atrac3

It may not be by much, but also the rating differences between AAC, Vorbis, Lame, and MPC were not all that much either.

FLAC?

[hot_Karls_bad_cavern]

er, did i miss flac in there somewhere? It's lossless, fast and compresses quite well....why leave it out?

Re:FLAC?

[runderwo]

WTF? Why would you even need a listening test if codecs are lossless? By definition, lossless compression produces final sound identical to the source. If it didn't, it would be lossy.

Re:FLAC?

[Electroly]

Because this was designed to test various codecs at 128Kbps. You can't make flac do 128Kbps. Besides, flac, being lossless, sounds exactly the same as the source media, so what's the point of testing how it sounds?

Re:FLAC?

[hot_Karls_bad_cavern]

Ah. My mistake. WTF? Why are you even listening to lossy codecs then?

Re:FLAC?

[Anonymous Coward]

We need (Score: -1, Incompetent) for Postings like yours.

Re:FLAC?

[Slowtreme]

Assuming FLAC is truly LOSSLESS:

1) Is it really a codec? Seems to me it is a compression method for media, like .zip .tar etc., not an encoder... technically.
2) It should sound exactly like the original. LOSSLESS = no loss. No point in comparing it to lossy codecs, unless it's not truly lossless.
3) The stored file sizes although smaller than the raw music are still way to big to be portable IMO.

Re:FLAC?

[Anonymous Coward]

1) Is it really a codec? Seems to me it is a compression method for media, like .zip .tar etc., not an encoder... technically.

Umm... technically, by definition, yes it is a codec. enCOder/DECoder. It encodes a WAV file into something else, which happens to shrink the file size, and can then decode the something else back into a WAV file, restoring the WAV file losslessly.

Re:FLAC?

[Malc]

Er, come on son: engage your brain.

Re:FLAC?

[Roland Piquepaille]

1) Is it really a codec? Seems to me it is a compression method for media, like .zip .tar etc., not an encoder... technically.

"Codec" means "coder-decoder". FLAC sounds encoded to me, if you need a FLAC library to enable a piece of music-playing software to read it, then I'd say the FLAC library is a codec.

2) It should sound exactly like the original. LOSSLESS = no loss. No point in comparing it to lossy codecs, unless it's not truly lossless.

Actually, it's interesting to compare lossless and lossy compressions because, these days, there's a fair chance that very good lossy compression sound so good it's almost impossible to tell the difference with the lossless compression.

3) The stored file sizes although smaller than the raw music are still way to big to be portable IMO.

Depends how much smaller. I'd say anything that doesn't produce at least 5x compression is worthless in any music player. You can zip a wav file and despite being much smaller than the original, it will still feel worthless to you in a compactflash card in terms of size.

Re:FLAC?

[swv3752]

Because I want to fit more than two songs on my MP3 player. If the encoding is good enough, then it is indistiguishable from the original are close enough. I don't want to dedicate an entire 80GB drive just to house my music collection.

Re:FLAC?

[Fweeky]

3) The stored file sizes although smaller than the raw music are still way to big to be portable IMO.

Well, sure; I don't want to devote 90% of my 512M CF card to a single album, but for my desktop it's perfect. I just transcoded an album from it to Vorbis (this is where it's nice that FLAC is so effecient to decode) -q-1 (that's right, quality minus one) and it sounds fine on my iPAQ. FLAC reduced the filesize from 683M to 466M, which is fine for my desktop; Vorbis reduced the filesize from that to 23M. It's not the sort of thing I want anywhere near my desktop with it's 90UKP soundcard and Sennheiser headphones, but for a portable device it's fine.

In future I hope portable storage scales up to the point at which I don't need to worry about encoding to lossy formats though. In the mean time, having my source as FLAC means I can transcode to whatever format best suits the available technology; from burning an exact CDDA copy for my CD player to a bunch of MP3's for my DVD player to a bunch of ultra low bitrate Ogg Vorbis files for my iPAQ. That'll do me fine until I can get a 256G CF card :)

FLAC has an extensive testing suite if you're not convinced it's lossless, btw.

Re:FLAC?

[hot_Karls_bad_cavern]

"I don't want to dedicate an entire 80GB drive just to house my music collection."

Glad to hear it. But i sure don't want to have saved a small (think future with me) amount of space at the cost of crappy audio. Your call.

Re:FLAC?

[Kukling]

Testing FLAC is useless, as most of you say.
But I think it was convenient testing any kind of lossless (FLAC or WAV).
The audience may have giving slanted scores because they know the clips suffered lossy codecs, they were hearing artifacts the way people hear "voices".
It would have been interesting to know if lossless scores 5.
And a codec wich really improves music would help many artists today XD

Re:FLAC?

[Jugalator]

First -- it's only crappy if the difference is noticeable. Icode my music in a lossy format, but of course enough so it isn't noticeable when using headphones. That's good enough for me, and as a bonus I've saved the disk space. In the future 80 GB might be little, but it isn't today and that's what matters as you buy hard drives.

Re:FLAC?

[Jugalator]

what's the point of testing how it sounds?

To see how much worse the lossy codecs are in comparison?

Re:FLAC?

[0x0d0a]

1) Is it really a codec? Seems to me it is a compression method for media, like .zip .tar etc., not an encoder... technically.

Compressers are encoders of a particular variety. They just choose a different data representation as an encoder does, but make an effort to take advantage of specific known characteristics of the data they are compressing to get a smaller, reasonable representation..

ZIP and gzip (tar does not do compression, just file joining) do very poorly at compressing audio. They do things like look for patterns of repeating (or at least commonly seen) sequences of data, and simply say something like "every time you see "z1", I really mean ";lt&a href="". This approach often works very well in computer-generated files.

However, it's very unlikely that you will get exactly the same sequence of bits in an audio recording, so .zip/.gzip are very poor at compressing audio recordings. FLAC and similar lossless audio compression look for things like (I would imagine) relatively small deltas from each sample point to the next, since this is a common characteristic of audio data.

FLAC is indeed lossless.

Re:FLAC?

[runderwo]

Are you done with your holier-than-thou charade yet? Imagine this -- other people just might have different needs and desires than yourself when it comes to compressing audio. That's why it's great to have choices. Comprende?

Re:FLAC?

[13Echo]

An 80 GB drive should hold nearly 200 (or more) FLAC encoded CDs. That's CDs, not tracks. This is assuming that the CD is nearly full, and the FLAC compression averages out at its usual 60% of the original.

Lossless is a more viable solution as we get larger hard drives and faster Internet connections.

I guess that if you are like many people and you have a ridiculous collection of pirated songs though, FLAC may not be a good solution.

Re:FLAC?

[13Echo]

I certainly cannot tell a difference between FLAC encodings and the original WAVS. The raw output looks identical if you examine the waveform.

1) It is a codec. You encode/decode from raw PCM/WAV to another format.

2) Sounds lossless to my ears. This is because it uses a compression format that is similar to ZIP/TAR.GZ/ETC., as you suggest. Nothing is cut out in the process. MP3 and other lossy formats make it a point to strip the highs and lows that the human ear cannot normally hear.

3) An average of 40-50% savings on a fully compressed FLAC file are very much worth it, in my opinion. Most people these days have many (hundreds?) of gigs of storage space. As I noted above, 200 (or more) full albums in FLAC format should fit on an 80 GB drive. I can normally fit at least two whole FLAC albums on a single 700 MB CDR for archival and backups. It's really hard to tell though. Certain types of music, like classical, compress better in FLAC format than noisy rock n' roll. You never really know what you are going to get, but on average - it is about 50%-60% of the original size for most of my music collection.

Re:FLAC?

[lovemayo]

according to this, 54 results was discarded because they ranked the reference file, instead of the encoded file. If flac was to be included in such a test, I'm sure it would have won, but im also sure it wouldn't have scored a perfect 5, even if it should have.

Re:FLAC?

[Anonymous Coward]

I guess that if you are like many people and you have a ridiculous collection of pirated songs though, FLAC may not be a good solution.

or how about: you don't have a dedicated hard drive for your songs, like most people?

Re:FLAC?

[mcg1969]

The way I understand the testing methodology (and I could be wrong):

--- The files that the testers received were all uncompressed (or compressed with a lossless codec). In other words, they each had been run through their respective lossy encoders and then decoded back to .wav format. There are several reasons for this: 1) it prevents the testers from deducing anything from the file sizes; 2) it eliminates the need for the testers to have access to all the codecs; and 3) it simplifies the design of the ABC/HR+ testing tool.

--- The tests weren't pitting the codecs against each other per se, but rather each against the uncompressed original. So there is no sense in including a lossless codec in the test because, well, it's already there in the form of the control file.

Again I could be wrong about this, so please correct me if I am wrong. In any case, there is no sense in testing FLAC here.

Re:FLAC?

[Anonymous Coward]

Actually, it's interesting to compare lossless and lossy compressions

But you can just as well compare lossy compression against the original file...same thing.

Re:FLAC?

[benwaggoner]

That seems like a questionable methadology to me. If the use couldn't tell the difference, it seems like that should be an automatic 5. Dropping cases where there isn't a perceptible difference woud tend to underrate the quality of the best encoders.

Re:FLAC?

[ff123]

That seems like a questionable methadology to me. If the use couldn't tell the difference, it seems like that should be an automatic 5. Dropping cases where there isn't a perceptible difference woud tend to underrate the quality of the best encoders.

The problem is that it's difficult to develop a consistent and fair method of determining what to do with results where the reference was ranked. Maybe just assigning 5 to these cases is one way of dealing with it. I can see doing this for someone who marks 4.9 for one of the references, but what about somebody who scores 3.5 on multiple references (I'm exagerrating, but not too much!)? Is it wise to even keep results from such a listener? I think that as long as there are enough results, which seems to be the case here, the fairest and most consistent way to deal with these cases is to simply discard them.

ff123

Re:FLAC?

[swv3752]

I own about 250 CDs. To be able to conviently store that amount, I need better compression. As I also don't want to have to do some on the fly conversion to MP3 so I can load songs onto my RIO, Ineed them to be in MP3.

Re:FLAC?

[Electroly]

It would be a control codec, then. That being the case, you might as well just use the actual source media.

Good.

[Cyno01]

Good to know MP3 is still improving. Yes vorbis and others are great, but i know every software and hardware player out there plays MP3. I'll be ripping all my cds to high quality MP3 befor i go to college, not because its the absolute best, but because its a standard. Standards aren't always the most efficient, but their strength lies that you cant change them on a week to week basis. Whatever hologrphic storage based finger sized half terabyte 24th generation iPod i buy ten years from now will probably still play my 128 and 256 MP3s.

Re:Good.

[Malc]

If you can play back Vorbis, then why would you care if your collection doesn't meet the most common standard? I bought an iHP-120 and I don't care if anybody else can listen to my tracks or not because I can hear them fine. If I'm listening on my computer, then I have the choice of any number of players that support it. In fact, I used EAC + Mario and I've been ripping my CDs to FLAC at the same time and I still have no problems with playback on the PC. If I even need my CDs in a different format, I can re-encode from my FLAC archive. So why is it that you really want MP3? So you can swap them with somebody else?

Re:Good.

[m0rbidini]

OK, you have a point. But check VorbisHardware for hardware with Ogg Vorbis support. Also, though Lame did well, MP3 is known to have some limitations. But if you have to use MP3, experiment --alt-preset standard in Lame. It was made to offer very good sound quality in bitrates that average below 200 kbps in most cases.

Regarding the results... It's a bit surprising that this third party tuning/tweak of Vorbis did so well. Which is great and I think Xiph should think about incorporating this work on their official encoder as soon as possible, in order to take advantage of its potential. You may be surprised about the relative low performance of AAC. This is partially due to the fact that the chosen AAC encoder was a CBR only encoder (because it was the best AAC encoder at this bitrate on a previous test - Nero encoder is also a good one and offers VBR encoder). With a good implementation of VBR AAC, it should be possible to get a better performance.

While most of the tested codecs/formats showed good performance at 128 kbps, this test alone shows that none can give transparency ( transparency == unability to distinct from the original source for most people and under good conditions) at this bitrate, contrary to what many think. People who think this is important should demand higher quality files from famous online music services (like iTunes Music Store).

People interested in lossy audio encoding should also try Musepack (file extension .mpc). It is considered by many of the hydrogenaudio enthusiasts as the best format at medium/high bitrates, offering transparency with bitrates normally lower (with standard preset ~170 kbps, typical 142 ... 184 kbps) than what is possible with other formats/codecs. It's now open source (LGPL, iirc). Its biggest disadvantage is the lack of support in portable players (though decoding musepack is faster than decoding the other formats in this test). There are plugins for almost every software player and foobar2000 (which I consider the best one) has native support for it

Re:Good.

[Ploum]

IMHO, you are wrong.

If the Fraunhoffer institute decides that every MP3 decoder must buy a very expensive lisense, you will not be able to listen legally to your music for free.
Dont say "it will never happen !". It CAN happen.. Look at the Movable Type story.

It's why MP3 codec is not available in the default Fedora system.

If you convert all your music to Ogg Vorbis (like I do and there's more and more Ogg Vorbis compatible player), you can be sure that in 100 years you will be able to play your music legally for free !

It seems that a lot of people aren't aware of the difference between a "Standard" and a "De facto standard"... There's a huge difference...

Re:Good.

[DrEldarion]

I think he was getting at the fact that MP3 is pretty future-proof. Sure, maybe a couple portable players support OGG and FLAC right now, but people who look for that feature are RARE. If the product doesn't do well and the company sees that that feature isn't something people want, they won't use it in their products in the future. MP3, however, is pretty much guaranteed to be around for a loooooooooong time.

Would you want to be stuck using a 10-year-old OGG player in the future when the awesome 300GB new-tech MP3 players built into your watch are out?

Re:Good.

[Malc]

I can't say that I'm that worried about it. It's just a matter writing a short script to iterate over my FLAC archive and re-encode. I anticipate doing that anyway as encoders improve at the same bit-rate. In fact, I'm already thinking of doing that with anyway to change some of my options...

Re:Good.

[Anonymous Coward]

I've been using Lame for many years now, ever since I purchased one of the original Diamond Rio's.
I use the HQ VBR 0, Joint Stereo, setting most of the time.
The Lame codec has definately improved over the years.
If your using a Windows box Audiograbber v1.83 is now freeware.

Re:Good.

[13Echo]

Yeah. I think that there will always be some sort of player that supports the most common lossless format out there. If FLAC gets overthrown by some other format (which is unlikely), it's just a matter of running a script to convert the files to another lossless format.

Meanwhile, I'm probably going to buy a RIO Karma to play my FLAC library on the road.

Re:Good.

[Malc]

Think about that a bit more. I have about a quarter of my CDs ripped so far (80 of them) and that requires nearly 27GB. I also looked at the Karma, but was worried about it only coming with a 90 day warranty - not long for a portable with a hard drive. Then again, I may be over-blowing it and being unduly paranoid. I'm still weighing up Vorbis vs MP3 on my iHP-120. It's coming down to battery life vs. storage requirements. The .OGGs take less space but they seem to burn through the battery in 2/3 - 3/4 the time.

Re:Good.

[13Echo]

Trust me. If I purchased a Karma, I'd be damn sure to buy a Best Buy replacement plan. Those Karmas have bad karma, and conk out pretty easily from the reviews I've read. They are really cool devices though, when they last.

Re:Good.

[ratamacue]

I'll be ripping all my cds to high quality MP3 befor i go to college, not because its the absolute best, but because its a standard.

Archive them with the FLAC lossless format, and you can later re-encode them to any lossy format you want. Archive them with a lossy format (MP3), and be stuck forever in that format.

I've archived my entire CD collection in FLAC format (almost 200 discs, about 60GB), and I normally play them right off the hard disk. But, I also have scripts to automaticlly convert to MP3 or OGG whenever the need arises (for portable CD player, etc).

Re:Good.

[Cyno01]

Its not an archive, its a portability thing, i have over 300 cds and i'll also be ripping some of my dad and sisters cds all in 256kb MP3. I'd like to keep the entire thing below 80GB, since i have a lot of video (archived TV shows on about 300 cds...) that i need room for too. I'm not an audiophile, although my system has 7.1+ surround, but i dont need lossless, this is mostly just so i dont have to take my CDs with me.

Re:Good.

[evilviper]

(though decoding musepack is faster than decoding the other formats in this test).

No, Atrac was faster last time I checked. Plenty of people also consider Atrac to be the best codec at high bitrates; the primary reason it was used for MiniDiscs.

Re:Good.

[evilviper]

MP3, however, is pretty much guaranteed to be around for a loooooooooong time.

I'm not so sure. People aren't all that happy about having to pay to license it right now, so if some other format really takes off in a short time, I would expect player manufacturers to start dropping MP3 if they can, or at least make it an extra-cost add-on somehow.

MP2 and RealAudio files were pretty popular before MP3 really took off, and there's no remant of them left, despite their widespread use about 3 years before portable hardware came along. RealAudio is still around, yet you don't see it in any players at all.

Casettes are another good example. They owned the world just a few years ago, and now it's getting a bit difficult to find players for them.

Also, one reason Vorbis isn't too popular is it's high CPU-usage. In a short time, embedded processing power is going to go through the roof, and it's not going to be a concern anymore.

Re:Good.

[ratamacue]

That's the main drawback with lossless -- you need a lot of space to store it all. If it's not going to be a formal archive, then MP3 will be fine, but going with 256 is overkill. If you're going lossy, you might as well go with a lower bitrate and reap all the benefits of lossy compression. Even the highest possible quality of lossy compression is still lossy, after all. Going with 256 is like saving a JPEG at quality level "10" -- you may have slightly better quality, but the main benefit of lossy compression (space saving) is negated.

BTW, lossless compression has nothing to do with being an audiophile. It's not really about quality. It's about having an exact duplicate of your master copy -- this is why lossless is suitable for archiving and lossy is not. A lossless copy is as good as having the original CD and always will be; a lossy copy is not and never will be.

Re:Good.

[afidel]

No, it can't happen. There are too many source code implementations of MP3 for Fraunhoffer to squash them all. Besides in much of the world they would have a hard time stopping any player which doesn't rely on their source because software patents aren't enforcable universally.

Re:Good.

[badasscat]

If you can play back Vorbis, then why would you care if your collection doesn't meet the most common standard? I bought an iHP-120 and I don't care if anybody else can listen to my tracks or not because I can hear them fine.

And what happens when 4 years from now your screen dies and it either a) costs more to fix it than to buy a new player, or b) there are no parts available anymore? Or what if you drop it and it breaks? So you have to buy a new player, and who knows what formats players will play by that time?

The only format that's guaranteed to be around, and will be the "default" format for all hardware players (except those made by Sony) - simply because so many people already have such a large collection built up - is MP3. I think this test is basically affirmation that even if all you care about is sound quality, it's still ok to use MP3 and get the best of all possible worlds - sound quality and compatibility.

MP3 was going to be around for many, many years anyway... but if it finally overcomes its undeserved reputation for being a "last-generation" codec with poor sound quality (an idea being promoted by the RIAA through their DRM partners at Apple and MS), it'll probably be around forever. This test proves MP3 is a fine codec for high fidelity and it also happens to be the most compatible and future-proof in the bargain.

Re:Good.

[MojoStan]

While most of the tested codecs/formats showed good performance at 128 kbps, this test alone shows that none can give transparency ( transparency == unability to distinct from the original source for most people and under good conditions) at this bitrate, contrary to what many think. People who think this is important should demand higher quality files from famous online music services (like iTunes Music Store).

But the online music stores might not be using the same free encoders that were used in this 128kbps listening test. Maybe I'm wrong, but I'm assuming the iTunes Music Store uses a pro-level AAC encoder rather than the free encoder distributed with iTunes. Also, I know at least one famous music store (Musicmatch Downloads) is providing higher quality files than 128kbps (160kbps WMA).

This listening test gives very useful information for those that rip and encode their own compressed music files, but it might not give an accurate representation of the quality of online music store files.

Re:Good.

[m0rbidini]

These are the encoders used by online stores. iMTS uses the AAC encoder used in this test. No other MP3 encoder can do better at 128 kbps than the setting used in the test with Lame. WMA9 Std was used in this test rather than WMA9 Pro. Why? Because all online stores use this version, and not Pro version, which is better. And the Pro version also lacks hardware support and MS isn't pushing it also because of that (thru WMP9 you can only encode to WMA9 Std.). I don't care, I don't use WMA...

cya

They will never

[millahtime]

So given Microsofts stated goal to bring us innovative technology, they should throw in the towel and ship OggVorbis and derivatives with Windows, right?

Microsoft will never. They will take the code from the #1, put a DRM to it and ship it as the next version of WMA. If they can't make the best they buy or take the best and make it their own. (with some tweaking of course)

markably?

[Alan Partridge]

a) Markedly
b) Remakably

Choose ONE only.

best vs popular

[trs9000]

i realize the geeks of the populace want the highest quality encoding to win. naturally. and it helps when something such as vorbis is rated so highly; it gives it even more geek cred.

however: as someone who studied music and audio, i am constantly surprised at what people will listen to. my friends (well some of them) have no problem cranking low quality mp3s of 50 cent, while i drop my jaw at the poor audio quality as a result of lost information. one time i even remarked to my dad "oh its an mp3" when he was playing something i had given to him which had been apparently later encoded. he wasnt sure (he didnt do the encoding) but doublechecked and yes it was mp3 (probably 160 kbps). he was impressed, when to me the timbral change in the cymbals was a dead giveaway. another time i asked a friend of mine if he was using aac to import all his cds in to itunes when he had been recently doing so. he looked at me blankly and said "whats aac?". which meant, yes he was.

i apologize for rambling, this is what im arriving at:
despite early adoption influence etc that geeks hold, how much does all of this really matter. most people dont care what format its in as long as they can listen to it. and often they cant discern loss of quality unless its extreme. so while i applaud these efforts, im simply wondering if -- aside from research -- they arent futile.

Re:best vs popular

[Anonymous Coward]

Is there such a thing as lossless audio recording? No. You can get better results with expensive equipment I agree with your comments about the futility of these studies, what if people liked the 128kbs mp3 version better than the cd recording! You can critise the 'timbral change in the cymbals' but why do we worry so much about this when, in many cases came off a synthesizer anyway.

Re:best vs popular

[trs9000]

You can critise the 'timbral change in the cymbals' but why do we worry so much about this when, in many cases came off a synthesizer anyway.

no, im afraid you may have misunderstood. what i was trying to convey is that while i may have a personal preference for a more 'accurate' representation, that doesnt really matter because most people dont seem to care too much. it has nothing to do with the synthesizing of it. i make electronic music so i tend to be quite fond of that, personally. (^_^)

it can be considered personal preference and if someone would rather listen to an downsampled, 8bit version of my music, well by all means, let them.

but in the pursuit of standards and codecs, etc, i think we should strive for accuracy -- but keeping in mind that it may not be adopted: whatever means you give the majority that is readily available and working, i believe *that* is what they will use.

Re:best vs popular

[Vellmont]

I listened to many of the blind tests. I have a decent sound card (Nvidia n-force), a high quality receiver, and decent, but not excellent quality speakers (i.e. $200 bookshelf speakers I bought a few years ago). I had a very hard time telling most of the samples from the reference implementation. Even the ones I thought I could tell a difference I wasn't sure.

I do remember a few years ago listening to really crappy implementations of mp3 codecs and hearing seriously awful artifacts. Considering that most samples scored far above 4, with 5 being imperceptible and 4 being perceptible but not annoying I think the results of this test mean that your choice of codec doesn't make much difference. Don't choose WMA or Altrac3 and you'll likely never notice a difference, or the slight differences aren't annoying. The worst score among the decent codecs was lame mp3 for Kraftwerk, and even that scored a 3.32 where 3 is slightly annoying.

Re:best vs popular

[trs9000]

to beat anyone else to the punch:

i am constantly surprised at what people will listen to. my friends (well some of them) have no problem cranking low quality mp3s of 50 cent, while i drop my jaw

yes i would drop my jaw if my friends cranked 50 cent too!

oh come on 50, i kid, i kid.
please dont shoot me.

Re:best vs popular

[Anonymous Coward]

Your shift key seems to be broken.

Re:best vs popular

[tomstdenis]

it's not 50 cent it's "fitty cent".

Say it like the "rapstors" say it.

Tom

Re:best vs popular

[evilviper]

another time i asked a friend of mine if he was using aac to import all his cds in to itunes when he had been recently doing so. he looked at me blankly and said "whats aac?". which meant, yes he was.
[...]
despite early adoption influence etc that geeks hold, how much does all of this really matter. most people dont care what format its in as long as they can listen to it. and often they cant discern loss of quality unless its extreme.

You more or less answered your own question (before you even asked it). While the public may not know the difference between Vorbis and MP3, the people writing the programs certainly do. CDex (popular CD ripper/encoder) supports many different codecs, but it defaults to Vorbis. WinAmp supports a few codecs, but it defaults to AAC. Anybody who doesn't care what they use will be using what the author decided was best. Right now, vorbis has two very strong reasons to be chosen as the default by applications (quality and license) so expect to see more and more programs outputting to Vorbis by default.

That leads to more players supporting Vorbis by default, and more hardware players supporting Vorbis as well.

As for quality loss, that's an interesting issue... If somebody can't tell the difference between a 128K MP3 and the source, then odds are they wont be able to tell the difference between a 48K Vorbis file and the source, so they can have the same (rather low-) quality, in half the storage space. Everyone wins.

Re:best vs popular

[Anonymous Coward]

Your brain appears to be broken/stupid. We/you are not rappers. Rapstors is not a word, and rappers do not deserve to have their name changed ala hax0rs. Hax0rs are elite computer genious', rappers are morons who cant sing

Re:best vs popular

[thakadu]

I had the same experience as you. I really tried hard to hear a difference, and when listening to music, one should be relaxed and not trying to find flaws! I concluded that from now on I will encode at 128kbps MP3 whereas before I was encoding at 320kbps. I can now fit so much more on a CD-R! I even set up my own test where I took a piece of music I know really well and did a test at 128, 256 and 320 against the wav file. I personally cannot tell the difference. Maybe I am just lucky!

Academically interesting, but pointless

[Anonymous Coward]

MP3 is as much a brand as it is a technology these days. Someone stop my mom, she is about to buy that new vorbis forked, hoooptee super player which by the way is not compatible with any of her software or her friends players or her in dash player or her DVD players.. yeah right.

iTunes AAC encoding problems

[lotsofno]

It is interesting that the note that they used the AAC encoder in iTunes 4.2 instead of the newer 4.5 because of "quality" concerns.

Apparently there's some "high frequency ringing" going on.

Better stick to something else for now, if planning to rip to AAC.

How much of this is just OGG fans voting?

[DrewBeavis]

I read some of the results, and I'm not a Vorbis hater or anything, but how much of this is open source fans voting for their favorite codec? I looked at the test just now, but can't tell if it was blind or not.

Re:How much of this is just OGG fans voting?

[technology is sexy]

It was a double blind test (ABC/HR) adhering to ITU-R BS.1116-1. Read more about the methodology in the initial announcement.
In addition to being double blind results were also encrypted so manipulation is very unlikely.

Re:How much of this is just OGG fans voting?

[Anonymous Coward]

It was double blind!

Re:How much of this is just OGG fans voting?

[0x0d0a]

I conducted a double-blind test on my own, unrelated to this test, between MP3 (VBR, LAME) and Vorbis. This was about a year ago.

In general, using the particular (percussion-heavy) piece I was listening to, I could consistently distinguish between the ogg and the original wav file at a higher bitrate. Drums just sound slightly different when compressed with Vorbis.

However, the Vorbis artifacts didn't make the music sound unpleasant to listen to. MP3 artifacts sound *awful*, turning cymbals into swooshing and wooshing sounds. As a result, I use vorbis rather than mp3.

Keep in mind that this test is a year old, and as the study indicated, Vorbis has undergone significant improvements, so my results may be less useful than they once were -- I would not be surprised if LAME's psychoacoustic model has been changed as well.

Also, I only worked with a single clip of music, thirty seconds long. It's likely that some codecs do significantly better or worse on particular songs.

Re:How much of this is just OGG fans voting?

[13Echo]

I was always fond of LAME encodings in a high quality VBR mode. It was always my favorite method of storing my music on my hard drive, since the quality was quite good. Over time, I decided that I would really start comparing it to some other formats for long-term archiving. I wanted to settle with one format, once and for all. I had originally been a BladeEnc user, but LAME seemed far superior to me.

When I first enconded some of my music in the Vorbis format, I was a bit underwhelmed when comparing it to LAME. It didn't really sound the same. Then, I compared the Vorbis files to the raw WAV rips. Surprisingly, the Vorbis files sounded more true to the original WAV rips. I was very surprised. All this time, my ears had tuned to the LAME acoustic model, which wasn't as accurate as I had once thought. After comparing a large portion of my CD collection in both LAME and Vorbis encodings, I made a decision...

I decided to start using FLAC. That way, I could listen to al of my music without any concern for quality. Sure, each CD takes up about 300 MB of space (50%-60% average compression), but it sounds so sweet.

If quality is a concern, maybe LAME MP3/AAC/Ogg Vorbis aren't the the right choices. Hard drive limitations aren't so much of an issue anymore. I guess that I cna see a point in having lower quality files for easy web transmission and low storage capacity, but the quality difference is just too noticable for me to ignore, when comparing any of these formats to a lossless format like FLAC. That's also one of the reasons that I like Magnatune so much, since I can buy music online that is already compressed in lossless FLAC format.

Re:How much of this is just OGG fans voting?

[jafuser]

Did they include a reference (uncompressed) entry? I didn't see it in the data. Or was it was a factor in calculating "rating".

Re:How much of this is just OGG fans voting?

[quake74]

Yeah, they did. And 52 tests results were thrown out for ranking the ref sample, i.e., if you're deaf enough to not recognize the ref we shouldn't trust your other rankings. See the discussion at HydrogenAudio.

Re:How much of this is just OGG fans voting?

[technology is sexy]

For fuck's sake read my reply. The ITU standard implies a reference and a hidden reference (=> double blind): Take a look at this to get an idea about what the test looked like.

Yes, but how did Ogg do?

[Anonymous Coward]

*ducks*

Open source wins: really cool, but ...

[eatmadust]

it isn't everything. Microsoft still has enough cash to fight this. Our local radio (Switzerland) still broadcasts in .asx. I sent them an e-mail asking them why. They said because their server is sponsored by Microsoft. Now I listen to virgin radio, they broadcast in broadband ogg

Re:Open source wins: really cool, but ...

[SYRanger]

The Norwegian Public Broadcasting company (nrk) now offers radio streaming in ogg vorbis in addition to mp3 and wma.

Re:Open source wins: really cool, but ...

[jafuser]

You get geek points for choosing the station you listen to by it data format instead of it's content =)

Re:Open source wins: really cool, but ...

[Anonymous Coward]

You're assinine enough to stop listening to a radio station cause of its codec. Bravo, you've just proven how stupid open source advocates really are.

Inaccurate test, big bitrate differences

[Echnin]

Quoting post on second page of discussion:

This particular test should be called, "The 128 kbps test for iTunes/WMA, and the low-130 test for AC3 and LAME, and the close-to-160 test for MPC/Vorbious.
Leahy iTunes MPC Vorbis Lame WMA Atrac3
bitrate 128 155 149 133 128 132
Score 4.34 4.41 4.68 4.11 4.37 3.76

That really doesn't look very fair to me! MPC and Vorbis using about 20% more bits than Lame and iTunes AAC.

Re:Inaccurate test, big bitrate differences

[technology is sexy]

Those numbers are just plain wrong. Take a look at the detailed result page to see the correct ones.
Additionally the encoder settings were tested on gigabytes of material to ensure they averaged at about 128kbps.
That's the beauty of using a VBR codec: It can allocate bits where they are needed. (That's also the reason for LAME's phenomenal performance)

Re:Inaccurate test, big bitrate differences

[Anonymous Coward]

The average bitrate for all files was:

iTunes: 128
MPC: 136
Vorbis: 135
Lame: 134
WMA: 128
Atrac3: 132

Re:Inaccurate test, big bitrate differences

[trs9000]

That really doesn't look very fair to me! MPC and Vorbis using about 20% more bits than Lame and iTunes AAC.

thats a feature, not a bug:

Vorbis! Now with 20% more bits!

Re:Inaccurate test, big bitrate differences

[technology is sexy]

Those numbers are wrong. The real average bitrates are:

iTunes MPC Vorbis Lame WMA Atrac3
128 136 135 134 128 132

Take your time to look at the detailed results yourself next time.

Re:Inaccurate test, big bitrate differences

[Anonymous Coward]

If I look at the detailed result page, the bitrates still differ.. They state the average bitrate on the page, so it really doesn't matter how many GBs of music they encode.

How can you compare 128 kbps to 136 kbps and be surprised that the 136 kbps encoding sounds better?

Re:Inaccurate test, big bitrate differences

[13Echo]

128 / 149 = 86%

Vorbis is not a CBR codec like WMA. It's almost impossible to get it directly on the nose. The encoder doesn't easily allow that kind of control without seriously damaging the quality of the finished file. I'm not sure that the 14% difference really matters as much as you insist.

To be fair though, WMA does perform reasonably well for a CBR format. However, that's not what the test is about. It's about getting the best sound out of a similar amount of space.

I don't doubt that Vorbis would still beat WMA if the bitrates were 100% even, to be honest with you. It's just not that simple to get it directly on the nose. It would have been interesting to see the results of Vorbis on a quality level that is a notch lower, so that we could see how much variance there is between each level.

Re:Inaccurate test, big bitrate differences

[Anonymous Coward]

Eeh, WMA can be both CBR and VBR (just like Vorbis). In this test it seems 1-pass VBR WMA was used.

Re:Inaccurate test, big bitrate differences

[RalphBNumbers]

The numbers you quote are the listed average bitrates for all of the songs togeather, the grandparrent seemed to be talking about the bitrates for one particular song/test. He wasn't wrong, but the average numbers you give are certainly more interesting to most of us.

The problem is your average numbers don't add up. If you look at the WMA column in the chart you got those average bitrates from, you'll see only 2 numbers under 128k. That fits better with an average bitrate of 129k for WMA unless you just truncate instead of rounding down..

In any case, iTunes certainly was at a disadvantage in these tests because of it's lower average bitrate than any of the other top 4 codecs.

Re:Inaccurate test, big bitrate differences

[13Echo]

Well, I suppose that would make sense. I've just never seen a WMA file in anything other than 128 bit CBR. I've actually not used Windows in several years, so I don't really have much experience with the format.

It does, however, seem kinda supid that they would perform the test that way, but I guess they were just trying to go by each format's standard method of encoding.

Re:Inaccurate test, big bitrate differences

[BillyBlaze]

First, you only looked at the first set of results - scroll down to the bottom and see the average scores and average bitrates. Now, as a quick-n-dirty compensation, here's the average scores times 128 over the average bitrate.

________Vorbis__MPC___Lame__iTunes__Atrac3__WMA
Q uality__4.59___4.47__4.18____4.26____3.73__3.98
B itrate___135____136___134_____128_____132___128
= Q*128/B_4.35___4.21__3.99____4.26____3.62__3.98

The ranking is Vorbis, MPC, iTunes, Lame, WMA, Atrac3.

Slashcode sucks.

# Please try to keep posts on topic. # Try to reply to other people's comments instead of starting new threads. # Read other people's messages before posting your own to avoid simply duplicating what has already been said. # Use a clear subject that describes what your message is about. # Offtopic, Inflammatory, Inappropriate, Illegal, or Offensive comments might be moderated. (You can read everything, even moderated posts, by adjusting your threshold on the User Preferences Page)

Re:Inaccurate test, big bitrate differences

[BillyBlaze]

Lame reply to self - actual ranking is Vorbis, iTunes, MPC, Lame, WMA, Atrac3

Re:Inaccurate test, big bitrate differences

[m0rbidini]

If you had read the results page you'd see:

"The unusual quality settings for MPC and Vorbis were chosen after testing several qualities over a wide range of albums and styles, and picking the setting that generated results closer to 128kbps."

Also, as already been said, those numbers are not the average birates of all the samples.

Re:Inaccurate test, big bitrate differences

[benwaggoner]

Actually, it was 2-pass VBR, so a bitrate could be specified. 1-pass VBR is quality limited for WM codecs.

The WMA9 codecs are somewhat unique in that they have modes like 2-pass CBR, peak constrained 2-pass VBR, and an unconstrained 2-pass VBR.

Compared with radio

[danormsby]

Compare this with radio. There are a lot of popular AM and LW radio stations here in the UK even though FM is a superior format. MP3 will be around for almost ever due to the popularity and level of takeup.

Re:Compared with radio

[MrIrwin]

Well that is a nice quaint british thing, aint it!

But I wonder if there is anywhere else in the developed world where music stations target FM, if nit for licence/economic reasons?

BTW, Radio 4 is the **only** UK station on LW, and is also available on FM, the LW 198KHz band is mainly kept active for the marine weather forcasts as so a low band is recievable quite a way offshore. Nor does it have music.

Radio's 1, 2 & 3 are maintained on AM, but they are also available on FM, digital, and satelite.

Similar story with commercial radio, but more bias to FM.

So what do people **actually** use to listen to music in the UK?

Re:Compared with radio

[Polkyb]

So what do people **actually** use to listen to music in the UK?

Headphones...?

:-)

Re:Compared with radio

[myc_lykaon]

So what do people **actually** use to listen to music in the UK?

I'm afraid I'm an old fogey so I listen to R4 (mostly commentary/news and current affairs) and ClassicFM. Both in my car on the way to work. FM holds up better in terms of reception quality in a car AFAICT. I also suffer from cloth ears - I used to shoot .22 rifle in competition so I'm a bit deaf in the 6kHz range and as a result when listening to digital streams, anything above 128kbps is a total loss on me.

Re:Compared with radio

[Malc]

I thought SW had a better range. Didn't they used bounce it off one of the upper levels of the atmosphere to get it in to eastern bloc countries?

Re:Compared with radio

[MrIrwin]

SW is bounced of the ionosphere. The 'angle' of bounce depends on the frequency, but the height of the ionosphere changes throughout the day (depends on sunlight). This means to broadcast via 'bounce' you must be using the right frequency at the right time. SW is **still** widely used for "state broadcast stations", if you listen to them you will hear that the "program guide" is mostly concerned with which frequencies they will be using to transmit to which areas at which times.

LW, by contrast, hugs the ground. You cannot go round the world, but you can get quite a way from the transmitter (several hundred miles at sea) on a single frequency and simple equipment. This makes it ideal for the weather bulitins that serve people like fishermen and sailboats that may not be fitted with sophisticated equipment.

Try tuning onto LW and see what stations you can pick up!

Re:Compared with radio

[Anonymous Coward]

the UK wasn't known for being on the forefront of technology the last time i checked. the number of slashdot readers should be indicative.

Re:Compared with radio

[karnal]

That's interesting.

When I was 13, a firecracker blew up near my left ear, leaving a never ending ringing in it.

Now, over twice that age, I get laughed at when I put in earplugs to shoot a .22 rifle. Of course, all I do is smile back, and tell them they should be wearing protection as well. It may not be as loud as a .44 revolver (wore protection for that one too), but my ears still hurt at the .22 level, unguarded.

I also take earplugs to concerts. As an amateur drummer/guitar/bass player, I can't believe people actually play drums (or any "rock" instrument) without hearing protection....

Re:Compared with radio

[evilviper]

There are a lot of popular AM and LW radio stations here in the UK even though FM is a superior format.

FM is not superior. FM has advantages, but it has major drawbacks compared to AM and LW.

Codecs aren't even remotely similar... MP3 isn't broadcasted longer distances with less power, so why are you trying to make the comparison?

Re:Compared with radio

[Anonymous Coward]

the UK wasn't known for being on the forefront of technology the last time i checked.

True. I believe that the culinary arts and brewing fine beers are their specialties. Or is it making great motor cars? (I cherish my Rolls Can'ardly).

Seriously though, they are the source of some of the best comedy on the planet.

Re:Compared with radio

[Anonymous Coward]

(the UK) are the source of some of the best comedy on the planet.

My favorite is one featuring Tony Blair as a Greek sex maniac. It's called "I Always Bend Over For Some Bush".

Open source marketing

[millahtime]

What we need is open source marketing. The open source community may be great at producing programs but at marketing them, well that's another story. We need open source marketing.

Umm because it's a 128k test?

[Kjella]

FLAC compresses to what, 5-600k? The only way you could make it fit the competition would be to mutilate the original WAV, and trust me, you wouldn't want to do that.

Certainly, it's a good reference if you want to compare across bitrates (what's 128k vs 256k vs lossless like in quality/size) but it has no place here.

Kjella

Aotuv vorbis enconder for Debian

[rsilva]

There is a Debian repository at rarewares where you can get the aotuv version of vorbis enconder. Grab yours today!

More vorbis content is needed

[mojo17]

One way I see Vorbis making it into the mainstream is if there were high availability of Vorbis content on the net. This includes P2P channels as well. If music releasers in the underworld start adopting vorbis, then Joe 'I own the original CD' Downloader will get a far wider familiarity with the codec, same as to what happened IMHO with xvid. More content will eventually lead consumers to start demanding vorbis compatibility in their hardware.

Re:More vorbis content is needed

[Petronius]

archive.org has lots of *free* content in OGG format.

Musepack

[Anonymous Coward]

What is it? I followed the link, but there was no info about it. It appears to be open source though!

No

[RAMMS EIN]

No, they won't. Their definition of innovation is making the same thing in an incompatible way.

Re:No

[Anonymous Coward]

Funny, seeing as how WMA was there first.

Control

[vossman77]

It would have been nice to have an original unencoded piece and rate it against the masses. That way we'd be sure the listeners weren't picking up on a mastering problem that is muffled by an encoder.

Re:Control

[Anonymous Coward]

How do you mean? In the ABC/HR test setup each codec sample is compared to the original.

Blind testing? And "Best sound" or "Accuracy"?

[turnstyle]

Of course it matters! I took a quick look, but I didn't seem to find anything describing how the tests were performed.

IMHO, the best way to test is to provide an uncompressed source and a variety of compressed files, and ask "which most closely matches the uncompressed source" -- and NOT "which sounds best."

Years ago, I did an a/b switch test with a high-end audio engineer between a CD and a 128kbit/s MP3. Though we could both clearly hear a difference, he actually guessed wrong.

My point is: the test needs to be blind, and the test should be looking for compressed files that most closely sound like the uncompressed original -- and not the ones that "sound best."

Re:Blind testing? And "Best sound" or "Accuracy"?

[Short Circuit]

ask "which most closely matches the uncompressed source"

Subjectively, ala an untrained consumer, or objectively using a computer?

(Not critisizing...just curious.)

I'm not an audio specialist by any means, or even all that well versed in audio compression, but here are some thoughts:

If you wanted to do it with a computer, the best means (or so it would seem to me) would be run a continuous frequency analysis of the original digital sample and of the digital outputs of each of the competing codecs.

You can compare the running frequency analysis and find out considerably more than which one comes closest to the original sample. For example, you could find out the overall frequency response, and how much it changes at points in a sample where there's a lot of different sounds or signals to contend with.

Re:Blind testing? And "Best sound" or "Accuracy"?

[Bertie]

No, because having an audio spectrum which is actually close to the original is much less important than having one which sounds the same to a human being. We don't understand exactly how all that masking of sounds etc. works yet, and so just because codec A replicates the frequency spread of an original piece more accurately than codec B doesn't necessarily mean that it'll sound better to you or me. This is still, to some degree, a black art. The only device which can properly measure the relative performance of the various codecs is the good old human ear. It can't be automated effectively.

Re:Blind testing? And "Best sound" or "Accuracy"?

[turnstyle]

Hey Short Circuit, does this mean I'm not your "foe" anymore? ;)

I'm not saying that it needs to be a computer-based accuracy check (that's fairly easy -- you just: 1) compress, 2) uncompress, 3) compare that to the uncompressed source).

People are fine, if not even better -- but, I do think that the question "which of these sounds most like the original" is a better question than "which of these sound best" when it comes to deciding what codec works best.

Re:Blind testing? And "Best sound" or "Accuracy"?

[Anonymous Coward]

Hey Short Circuit, does this mean I'm not your "foe" anymore? ;)

Probably not till I see your argument style changes. See my journal for that bit.

Besides, my freind/foe arrangement only serves to draw my attention to posts by such users. I don't color my discussion based on friend/foe lists. It just doesn't make sense. For example, I respect you as a professional, but I strongly disagree with some of your views.

That doesn't mean I don't want dialog. A lot of smart people have had horrible views. The term "evil genius" is a semantic manifestation of that fact. I don't normally want to ignore what could be valid points just because the person saying them has different views on some or all catagories. That's essentially mentality racism. In fact, there's a link in my info page for people to email me to discuss why they marked me as a foe.

I included the clarification that I wasn't criticizing so that you hopefully wouldn't think I was attacking you, since that wasn't the impression I wanted to get across.

-SC

Re:Blind testing? And "Best sound" or "Accuracy"?

[turnstyle]

"I respect you as a professional, but I strongly disagree with some of your views."

Just curious: what are some of my views that you strongly disagree with?

Re:Blind testing? And "Best sound" or "Accuracy"?

[TechniMyoko]

That makes no sense. If the frequency data matches the original, then it would sound like the original. The human ear isn't reliable, as human judgement isn't either. For instance, those tests where they give a person tap water in 2 differently labelled bottles (ie: evian) and they claim to be able to tell the difference.

Human judgement can't be trusted when compared to a computer

Re:Blind testing? And "Best sound" or "Accuracy"?

[Anonymous Coward]

Here's my journal entry on the subject. Too keep from going farther OT here, why don't we continue discussion there?

Re:Blind testing? And "Best sound" or "Accuracy"?

[damiam]

If the frequency data matches the original, then it would sound like the original.

Only if it exactly matches, which it doesn't in this case (these are lossy codecs). It's possible for codec A to be match almost perfectly, but in such a way that the difference is easily audible. It's also possible for codec B to produce a markedly different spectrum that still sounds very close to the human ear.

People, not computers, listen to these compressed files. So the only sensible way to judge compression is by using people.

Re:Blind testing? And "Best sound" or "Accuracy"?

[turnstyle]

"It's possible for codec A to be match almost perfectly, but in such a way that the difference is easily audible."

Correct!

A psycho-acoustic difference is not necessarily the same as numeric data difference.

Some minor numeric differences can be perceived as substantial psycho-acoustical differences -- and conversely, some substantial numerical differences be perceived as minor psycho-acoustical differences

Re:Blind testing? And "Best sound" or "Accuracy"?

[turnstyle]

Here's my journal entry on the subject. Too keep from going farther OT here, why don't we continue discussion there?

Just tried, but it's locked.

fwiw, have you seen my personal page? That page links off to stuff that I've written, and stuff that took time to prepare (as opposed to a simple post), and so it's really a better way to get a sense for where I stand...

Re:Blind testing? And "Best sound" or "Accuracy"?

[Have Blue]

some substantial numerical differences be perceived as minor psycho-acoustical differences

Well, duh. How do you think lossy compression works?

Re:Blind testing? And "Best sound" or "Accuracy"?

[Anonymous Coward]

The test was blind, and it wass looking for compressed files.

Re:Blind testing? And "Best sound" or "Accuracy"?

[thakadu]

I took the test and it was exactly as your indicated best way. In other words, you had to identify which of the encodings most closely matches the original (blinded of course). Maybe my ears are bad but I could not tell a difference in any of the samples which was really an eye opener for me. All along I have been encoding at 320kbps. Ever since taking the test I saved a huge amount on CD-Rs by encoding at 128.

Re:Blind testing? And "Best sound" or "Accuracy"?

[Short Circuit]

Read it now. Personally, I favor having voices like you and voices like the EFF out there. It's the only way to keep a middle road available.

...and so it's really a better way to get a sense for where I stand...

Not sure about that. You can learn plenty about who a person wants you to think he is by reading carefully prepared and spellchecked documents. But that sort of personality is usually hollow, finite and static. The only way to really get to know a person is to catch them off-gaurd.

And if you're going to debate with someone, you need to know them. Not their public persona.

Oh, and check that journal link again. ;)

Re:Blind testing? And "Best sound" or "Accuracy"?

[iminplaya]

Yay...We finally got something where how I made/make my living is relevant. I, too, "work the board". Nice high end ones when it's somebody else's. Don't ask for names, just put me in front of one, and see if I can produce satisfactory results. Live, studio, mostly TV. While I get to play with the occasional 16 channel Mackie(not the best, but the help can drop it off the van, and it still works perfectly, for a while.), usually, the "best" mixer I see down here came with Windows. Maybe now, you might understand why I get so pissed at the "copyright whores" when they cripple my equipment just to keep the price out of the reach of the consumer and the low(no) budget bands that just want to produce the best sound possible.
Another "yay". I actually agree (OMG) with your post. While I personally thought that 128k sounded a bit muddy(192k is my personal minimum), It wasn't a blind test by any means. I sure would like to try it sometime. I always wore ear protection at the airport(A&P mech, also) and the shooting range, so I'd like to think my ears are in pretty good shape. However, I STILL think digital audio is garbage, but convenient and necessary for multi-generation work(difficult to do with consumer DAT because of you know who. Might require "illegal" hacking to accomplish). In another non-blind test, I compared a DAT machine with an analogue multi track(12 or 24 don't remember.) moving at 15 in/sec. The DAT sounded like "Mississippi Mud" compared to the other. While the "depth"(16 bit) might be sufficient, digital just does not take enough samples. For me it completely destroys anything above 8-10khz. How can one expect an accurate representation of a complex waveform when you barely get 6 samples at 8khz?. A-D conversion and back simply "rounds off" too much. I'll need to see it on a scope to verify that. I'm not sure how familiar you are with TV broadcast signals, but the signal coming out of the exciter(modulator) looks nothing like what comes out of the antenna. We have to do all that pre/de emphasis, phase distortion, differential gain stuff also. I can only imagine that the same thing is happening in A-D D-A conversion. I never claimed to be a designer, so how can I know for sure? Come on down and build a studio, so I can take the test, ok?
As a side note, mp3 or the other compressors do nothing foe me except allow me to put my collection on the computer so I can let it play for a week or so without having to get up to change the damn disc every hour. I sure hope I don't have to hack DRM to keep doing that in the future. Discuss?

Re:Blind testing? And "Best sound" or "Accuracy"?

[Anonymous Coward]

You should have replied to the poster up one from where you did -- he was the one who didn't understand, not the one you replied to.

Re:Blind testing? And "Best sound" or "Accuracy"?

[Anonymous Coward]

No. No you did not. You are lying. Any "high end" sound engineer can tell the difference drunk and blindfolded between CD and 128kbits MP3. Even with LAME blah blah blah. Minidisk too.

He is not a audio engineer or you are a lying sack. One or the other.

Re:Blind testing? And "Best sound" or "Accuracy"?

[Anonymous Coward]

Any "high end" sound engineer can tell the difference drunk and blindfolded between CD and 128kbits MP3.

Have you never met an MCSE who couldn't find his way around a command prompt? People with little to no clue seem to find their way into jobs without difficulty. It doesn't seem much of a stretch to imagine that the audio engineer field is as susceptible to this phenomenon as every other line of work.

Expensive earbuds and MP3 players

[0x0d0a]

It could also be marketing.

MP3 players got *heavily* marketed after Napster and friends got press and serious college use. "MP3" became associated with "free music". They took off.

The iPod, a decent but not earth-shattering MP3 player, sold *much* better than other MP3 players out there. Why? Marketing. Lots of ads -- the only significant difference to cause such a change.

Vorbis doesn't have a lot of ad money behind it pushing it.

I'd also like to point out that:

* People still use CBR MP3s. CBR was designed for exactly one reason -- allowing constant-rate streaming. It's *stupid* to use CBR for locally stored files -- it gets significantly worse quality for the size -- I've generally found that on the music I listen to, using VBR is equivalent to at least a 30% increase in bitrate in terms of my ability to distinguish between a master an an MP3. If people cared about quality, CBR MP3s would not exist. They wouldn't even have to switch their hardware/software around, since it's the same format, but they won't even go that far.

I *really* get a kick out of it when people buy an MP3 player and a pair of high-end earbuds. It's just plain inane. They just purchased a low-quality audio playback device and then spent a huge amount of money on an expensive pair of earbuds that don't let them hear the now missing nuances of the audio. It's the ultimate in trendiness -- like buying Nike or Banana Republic clothing. iPod + expensive earbuds is not "the ultimate in sound reproduction" even if you really, honestly gave a lot of retailers a whole lot of money for the combo.

Re:Expensive earbuds and MP3 players

[trs9000]

i think you have some excellent points -- and another poster here even said something like "what we need is open source marketing"....
but unfortunately what we are dealing with is something larger than the marketing of an audio format or even -- perhaps more importantly -- an audio playback device. the riaa probably wouldnt be too keen on an open format being used for the encoding of music that they want to make money off of.

when -- and i say when cause i believe it will have to change -- the model of payment changes for recording artists to something such as subscription or etc, then we may see a flourish of superior technologies being used. especially those that are open source which may allow companies to save money the r&d end of things.

Re:Expensive earbuds and MP3 players

[kryptkpr]

I *really* get a kick out of it when people buy an MP3 player and a pair of high-end earbuds. It's just plain inane

Ever heard of --alt-preset-extreme?

Sure.. stuff I download will continue to sound crappy (I don't even keep anything below 192kbit anymore).. but stuff I encode myself sounds quite good. I'm not audiophile, but I cannot tell the difference between an --alt-preset-extreme'd recording and the original.

Re:Expensive earbuds and MP3 players

[Unknown Lamer]

The DAC in the iPod is fairly high quality. It is not unreasonable for someone to simply encode their CDs using Apple's lossless codec and put them on the iPod. With a 40G model around 60 albums (assuming an average size of 650M) could be stored losslessy in WAV; a few more using Apple's lossless encoder. It would be like turning your 40G iPod into a 5G iPod and swapping music around but such is life.

It becomes more realistic when you have 80G and 100G drives in your player; in a few months the Neuros is supposed to have 80G backpacks available (right now up to 40G are available and a few online stores are advertising the availability of the 80G model early) and you can order an 80G backpack right now from Cool4u2View. The Neuros doesn't support any lossless codecs except for WAV right now (although there is support for WMA I have never used it and do not know if it supports WMA lossless or even if WMA lossless is anything more than tagged WAV). 80G is still around 110 albums. The Neuros IIRC uses the same DAC as the iPod so the quality of the sound would be excellent.

For me -b 160kbps Vorbis files are good enough; I plan to re-encode my collection to FLAC when I get a larger HD for music (right now it is a poor little 20G that only has 4G free) as well as Vorbis (abcde makes it easy to encode to more than one format and put them in different directories) -q5 (for my Neuros).

So your last comment still applies to most people. Not everyone though.

Re:Expensive earbuds and MP3 players

[brysnot]

It's the ultimate in trendiness -- like buying Nike or Banana Republic clothing.

Yeah.... hey the 90's called, they want to tell you to get a fashion clue.

Re:Expensive earbuds and MP3 players

[damiam]

I *really* get a kick out of it when people buy an MP3 player and a pair of high-end earbuds. It's just plain inane. They just purchased a low-quality audio playback device and then spent a huge amount of money on an expensive pair of earbuds that don't let them hear the now missing nuances of the audio.

Low-quality audio playback device? The iPod (and most other players can play lossless audio, if you really want to be snobby. 99% of people can't distinguish between the original and 192kbps AAC or VBR MP3, on any equipment. However, just about anyone can hear the difference between the iPod's $10 earbuds (not what they're sold for, but about what they're worth) and a decent $40-$50 pair.

Re:Expensive earbuds and MP3 players

[foolip]

I don't know if it's of any interrest to you, but since vorbis-tools-1.0.1 oggenc can take flac input, so you really only need to encode to flac and then you can very easily reencode these to vorbis (tags preserved!) when you need it.

Re:Expensive earbuds and MP3 players

[AdamWill]

Agreed. Loss from compression and the problems of crappy earphones aren't the same; even badly compressed music will still sound BETTER on good earphones. Not great...but better.

Re:Expensive earbuds and MP3 players

[Anonymous Coward]

They just purchased a low-quality audio playback device and then spent a huge amount of money on an expensive pair of earbuds that don't let them hear the now missing nuances of the audio.

I don't mean to seem nit-picky, but wouldn't that be *let* them hear...or maybe let them not hear? High quality playback devices (nice earbuds) and a low-quality source (presumably from the iPod's DAC or from the MP3s) usually make my experience worse for the reason that they make it very apparent that there is important information either missing or mutilated. When I've got crappy source, a lot of times I'll play back on crappy devices (e.g. I usually prefer a 320x240 movie played back on a TV because it masks some of the artifacts). It makes the poor quality less noticeable by reducing the entire movie to a lower quality representation. That being said, I think that most MP3 playback mechanisms have good enough source/reproduction to substantially benefit from a good pair of headphones/earbuds.

Re:Expensive earbuds and MP3 players

[Tiro]

CBR is for one thing: streaming

If you have a CD with lots of tracks that run into each other, CBR is required [in LAME at least] if you want to do gapless encoding.

Re:Expensive earbuds and MP3 players

[EllisDees]

>I *really* get a kick out of it when people buy an MP3 player and a pair of high-end earbuds. It's just plain inane.

Quoting someone on the HydrogenAudio forums:

"I think a lot of people thought that the test was going to be very easy (me included), "Come on, it's 128kbit! That sounds like crap, everybody knows that.". ...only to find out that there couldn't be found any major imperfections in the couple of samples tried. Sample 1 looks like it was one of the hardest ones to abx; very tough start, especially for someone who had set his mind on the assumtion above."

I'm betting that in any real test, you wouldn't fare any better.

Re:Expensive earbuds and MP3 players

[Anonymous Coward]

People still use CBR MP3s. CBR was designed for exactly one reason -- allowing constant-rate streaming. It's *stupid* to use CBR for locally stored files -- it gets significantly worse quality for the size -- I've generally found that on the music I listen to, using VBR is equivalent to at least a 30% increase in bitrate in terms of my ability to distinguish between a master an an MP3. If people cared about quality, CBR MP3s would not exist. They wouldn't even have to switch their hardware/software around, since it's the same format, but they won't even go that far.

Sadly, not true.

Not all players/devices properly support VBR (mostly 1st and 2nd generation devices). CBR always works.

Even WinXP has issues with VBR encoded MP3s. (Pull up a folder containing your VBRs, switch to detail view and add the extra duration/bitrate columns and watch the fun.)

Re:Expensive earbuds and MP3 players

[Anonymous Coward]

And i use a pair of sennheiser HD 600's with an external dac and headphone amp to play back my LAME alt-preset extreme MP3 collection....

There are no missing "nuances" from the encoding. On maybe 5-10% of the samples there is a tiny reduction in soundstage or "openess", but certainly nothing distracting or something i could pick out unless i was doing an A-B comparison. On the other 90% + of the music, there is simply no difference even in a direct comparison with the original cd.

Only the craziest audiophile will have a set of "high-end earbuds" that could exceed the resolution of my setup. And i'm sure they would use losseless encoding anyways along with an iriver ihp-140 connected via optical output an external dac. For most people, a set of "high-end earbuds" would probably cost from $50-150, and that certainly isn't overkill for the mp3 format or the average mp3 player. And it would certainly be a good way of getting good music on the go, even if it doesn't quite sound as good as the home system of a snobby audiophile like yourself.

In fact, i'd say its a waste of money to spend $300 + on an mp3 player and not have a good set of headphones/earbuds to go with them.

Re:Expensive earbuds and MP3 players

[Anonymous Coward]

Well I dunno about expensive earbuds, but some people replace the trademark white ones because few other fashion accessories say "mug me" quite so loudly.

Re:Expensive earbuds and MP3 players

[Patik]

The iPod, a decent but not earth-shattering MP3 player, sold *much* better than other MP3 players out there. Why? Marketing. Lots of ads -- the only significant difference to cause such a change.

No, it won because it extremely small, very easy to use, and held a ton of music compared to the popular solid state storage players. I hardly saw any ads for the iPod at first; it was seeing other people's players that made me want (and get) one.

It's *stupid* to use CBR for locally stored files -- it gets significantly worse quality for the size

I agree with you, but unfortunately a huge drawback is that players can't seem to tell how long a track is. iTunes and Winamp, in particular, display grossly inaccurate song lengths, which makes the slide bar useless and annoys me since I like to keep an eye on that sort of info. It seems like the players read the bitrate of the first frame of the file and calculate the song length based on filesize.

Re:Expensive earbuds and MP3 players

[dozer]

If you have a CD with lots of tracks that run into each other, CBR is required [in LAME at least] if you want to do gapless encoding.

That's not true anymore. I do gapless VBR all the time using LAME.

lame --alt-preset standard --nogap track1.mp3 track2.mp3 track3.mp3 etc.

Note that your player must also support gapless playback. XMMS doesn't.

It's called a "control"...

[gumpish]

What if a lossless codec were included in the test - and it came in dead last?

That would provide useful information: either the listeners weren't up to the job or the lossy codecs at ~128 kbps were truly indistinguishable from the source material.

Re:It's called a "control"...

[runderwo]

What if a lossless codec were included in the test - and it came in dead last?

I still don't understand why a lossless codec must even be considered in a survey that is comparing lossy codecs. A file compressed with a lossless codec will be indistinguishable from the uncompressed original audio files (which I assume he included for comparison, else it's a rotten test).

What I can't seem to find out is if the original file was both supplied for comparison as well as made part of the double blind test. That would be the best methodology in my opinion.

Re:It's called a "control"...

[gordyf]

The tests are performed blind - you hear two bits of the same song - one's the original, the other the encoded version, but you don't know which is which. You then choose the one you think is the encoded version, and you rate it. If you choose correctly, the encoder's rating goes down. If you can't tell the difference, you'll be choosing randomly, which will give the encoder a high rating. Testing a lossy codec would give a high rating, since people would be choosing randomly (they'd have to be, since they'd be hearing the same thing on each sample).

The fact that encoders were getting low scores in this test shows that they can be differentiated from the original, and that they do sound bad, since people chose correctly.

Re:It's called a "control"...

[ratamacue]

What if a lossless codec were included in the test - and it came in dead last?

Why would you go to the trouble of "testing" lossless when you could just play the original disc?

No

[Anonymous Coward]

"there's a fair chance that very good lossy compression sound so good it's almost impossible to tell the difference with the lossless compression."

Simply untrue.

What you might be saying is "using my earbuds, my ears can't distinguish the difference between the CD and 128kb AAC".

Really, that's pretty thin justification for cheerleading for iTMS (which sounds pretty thin).

Vorbis and MP3 formats from a techie view point

[tiger_omega]

Having designed and written a mp3 decoder and now working on a vorbis decoder I can't say I'm that suprised by vorbis coming out on top.

From a technological standpoint the Vorbis codec has 10 years of audio compression R&D in it since MP3 was invented.

MP3 is a subband DCT based codec using fixed window length. Vorbis is also DCT based but encodes an approximation to the orginal frame's spectral curve and also uses variable length window length.

In using the source from the vorbis library and the decoder specification to help guide its development I have to say it is a real joy to code. The people at xiph.org have really done a first class job and have approached some of the problems of audio codec design with some of the best lateral thinking that I have ever seen.

Believe me! Coming from me that is very rare praise.

Re:Vorbis and MP3 formats from a techie view point

[Hast]

I have never done any Ogg development but I have been part of a project where we did a hardware implementation of MP3. Does Ogg correct some of the /really/ stupid errors in MP3 such as 18 frequency lines? I remember when we were reading up on the theory and basically all of the clever and fast algorithms for DCT (or any D*T for that matter) requires the frequency lines are a power of two. There were many issues in that same area, particularly for the for the two really heavy parts of the decoder (filter bank and iDCT).

So is that better in Ogg?

Re:Vorbis and MP3 formats from a techie view point

[Anonymous Coward]

Can Vorbis do low bit rate files like MP3 can? Something that could stream over a 28.8 modem?

I was kinda disappointed that xiph.org felt the need to come up with a separate codec for this (Speex). MP3 is "one-stop shopping" in this regard.

Re:Vorbis and MP3 formats from a techie view point

[Anonymous Coward]

Speex is a voice codec. MP3 is not really good as a voice encoder. Vorbis is better at lower bitrates than mp3 for music. Speex is better than both for low-bitrate voice encoding.

trocLlkore

[Anonymous Coward]

fear the reap3r were c0mpounded

Why would you care about accuracy?

[cgenman]

If a sound was perfectly accurate except for an instantaneous annoying pop every few seconds, it would probably average as the best codec, but it would be useless as a consumer standard. I remember a codec shootout years ago where Mp3Pro sounded "tinny," WMA sounded "flat," and MP3 sounded "fuzzy." Was being objectively closer to the source material more important than the type of distortion introduced? Not at all.

When dealing with sound equipment, from pre-amps to encoders, the tone of the introduced distortion is very important. Everything introduces distortion, in some way or another. You just want it to make the sound better, not worse.

What it means for Vorbis

[QuantumKnot]

Well, I can only comment from a technical point of view, but firstly it is very good news that we are progressing in the right direction in terms of quality. Secondly, compared with the other codecs (esp. the proprietary ones), Vorbis is quite simple and minimalistic and lacks a lot of advanced tools and profiles, yet we've been able to extract quite competitive performance from some adjustments here and there. There is more to do in Vorbis and Monty has some new ideas that he wants to implement in the next major version like a better stereo model, noise normalization (which in its current form is mostly experimental), and support for 5.1 stereo. Given the success of aoTuV and the fact that Monty is fully aware of these third-party tunings, I think Vorbis development is looking ever-more exciting. :) (Note I don't work for Xiph.Org but just one of those third party Vorbis tuners)

Re:What it means for Vorbis

[Anonymous Coward]

Uh ok, I guess that's how a ogg freak would view reality.

Lame CBR better than VBR?

[pfavr]

Recently I decided to use lame with cbr 192kbps after comparing to the preset vbr settings (including extreme). I use the settings: --cbr -b 192 -h -q0

Using vbr I can hear the noise floor being modulated e.g. by a large amplitude low pass filtered bass sound. I contribute this to vbr changing bitrate. Maybe the psychoacoustic model just doesn't fit my ears:-)

The vbr files average around 200kbps anyway, so they're not smaller than 192kbps cbr.

It would have been nice if the test included cbr as well.

I use good headphones: Sennheiser HD-25, and my mp3 player: archos jukebox recorder running the open source firmware Rockbox

Re:Lame CBR better than VBR?

[technology is sexy]

LAME developers themselfves discourage the use of CBR and I doubt you have done any ABX tests to back up your claim that 192kbps CBR sounds better than one of the VBR presets?!

Re:Lame CBR better than VBR?

[pfavr]

LAME developers themselfves discourage the use of CBR and I doubt you have done any ABX tests to back up your claim that 192kbps CBR sounds better than one of the VBR presets?!

Well, I compared the original wav and the two mp3s... (using xmms) but it wasn't a blind test (I was aware of which one was playing).

As I stated in my original post it has to do with the noise floor changing - (and this applies to faint echoes and reflections as well when they're almost buried in the noise). IMHO the benefit from having some frames encoded at >192kbps is not balancing out the disaster of having some frames encoded <192kbps. Or you could say that to me overall quality is more important than occasionally saving a few bits.

Re:Lame CBR better than VBR?

[technology is sexy]

Before you don't provide ABX results your findings are pretty worthless (AFAIK lame itself provides are rather minimal ABX tool - look at the source archive). You're welcome to discuss them at Hydrogenaudio.org though.

Re:Lame CBR better than VBR?

[pfavr]

Before you don't provide ABX results your findings are pretty worthless<snip>

My findings are quite valuable - because I like my mp3s better now :-)

Just encode your stuff as you find best. I'll do the same. Everybody's happy :-)

Re:Lame CBR better than VBR?

[m0rbidini]

Recently I decided to use lame with cbr 192kbps after comparing to the preset vbr settings (including extreme). I use the settings: --cbr -b 192 -h -q0 Using vbr I can hear the noise floor being modulated e.g. by a large amplitude low pass filtered bass sound. I contribute this to vbr changing bitrate. Maybe the psychoacoustic model just doesn't fit my ears:-) The vbr files average around 200kbps anyway, so they're not smaller than 192kbps cbr.

YUO, sir, are completely clueless about this. *ALL* tests made up to now show that what you wrote is completely false, specially if you are using 192 kbps. Check HydrogenAudio, use the search. Countless tests showed that the CBR approach is completely stupid (unless to comply to some system requisites) if you want to have a constant quality level. That's the way all modern lossy encoders work (be it audio or video). The only exceptions are very low bitrates and, of course, LAME's -alt-preset insane, which uses 320 kbps CBR.

Even most "pirate" release groups have finally adopted VBR for their releases (LAME's --alt-preset standard).

I'll save you some effort:

Recommended Lame settings
LAME 192 CBR vs LAME aps
why 192kbps?
CBR vs VBR

Please read'em all. Of course, you can use whatever you like. Just don't spread BS around.

Re:Lame CBR better than VBR?

[pfavr]

I don't think I'm clueless (but how would one know;-). I don't know a lot about encoder internals but I do have ears. The purpose of vbr is to save bits where the vbr algorithm sees fit. But what if the vbr algorithm is not up to the job? I would expect vbr with no frames <192kbps to sound superior to cbr 192kbps. But in order to encode some frames with >192kbps it needs to encode some with lower rate (for the same average bitrate). As I stated in my original post, I could hear the noise floor being modulated on some music with large amplitude low pass filtered bass (excellent candidates for low bitrate encoding). I contributed this effect to the vbr algorithm changing bitrate dynamically, and decided to use cbr 192kbps instead which I'm satisfied with (lame is a great encoder btw). Just wanted to report my experience because everybody seems so keen on vbr and after that experience - I'm not.

Thanks for the links (damn, now I need to use the rest of the afternoon on comparing and reading about encoders - again;-)

Re:Lame CBR better than VBR?

[m0rbidini]

The purpose of vbr is to save bits where the vbr algorithm sees fit.

No, the purpose of VBR is to maintain a constant quality level. And LAME's VBR model is quite well tuned, be sure of that. Try the --alt-preset standard/extreme presets. Please do some ABX tests with some decent headphones (you don't need extra expensive "audiophile" stuff).

About your LAME command line. Up until recently (v3.95 or v3.96, dunno for sure) -q0 could be in fact bad for quality. Once again, check HA and use the search. According to HA, the recommended LAME version is still 3.90.3. You can find it at www.rarewares.org (maintained by rjamorim, who conducted this test). But v3.96 seems quite good too (it has been tested in HA, and the results have been mixed). But please, once again, try the presets.

cya

Re:Lame CBR better than VBR?

[pfavr]

No, the purpose of VBR is to maintain a constant quality level

So the sound quality of cbr320 varies too much and that's why vbr was invented? Always thought it was to save bits... ;-)

Re:Lame CBR better than VBR?

[m0rbidini]

That's why I wrote "The only exceptions are very low bitrates and, of course, LAME's -alt-preset insane, which uses 320 kbps CBR."

Happy now? :-)

DOH, I shouldn't reply to trolls. ;-)

Re:Lame CBR better than VBR?

[pfavr]

DOH, I shouldn't reply to trolls. ;-)

I wasn't intending to troll - you just cited me out of context and so did I.

Happy now? :-)

original post is now moderated flamebait and you call me a troll and clueless :-(

I did have a track which produced artifacts (at the noisefloor) using lame with vbr. Using cbr solved the problem.

If I could find the original track I could post some evidence. But maybe lame vbr improved in the meantime and thus wouldn't show this problem now.

Re:Lame CBR better than VBR?

[menders]

I recommend getting the recommended LAME compile from Hydrogenaudio and using the recommended LAME settings. So that's LAME 3.90.3 using --alt-preset standard. You will not be disappointed. If you still think you hear something, e-mail me.

What if?

[Anonymous Coward]

"what if people liked the 128kbs mp3 version better than the cd recording!"

People like to pierce all parts of their body too. All we can learn from this practice is that 50% of the population is below average.

And people listening to 128kb lossy music and saying "they sound just like the CD" are in the lower part of that bell curve.

That's hardly insulting, its just reflects the statistical model here.

Here's the thing

[tkrotchko]

"You can critise the 'timbral change in the cymbals' but why do we worry so much about this when, in many cases came off a synthesizer anyway."

Not everybody listens to pop music.

Some people like orchestral works, or baroque music or choral music, or bop, or dixieland, or a whole host of genres where synthesizers are never (or rarely) used.

If a lossy codec sounds fine, then you should be glad; you've found something inexpensive that works well for you. For other types of music, even CD's struggle to maintain good sonic quality and so my frustration is that we're currently undercutting quality for more convenience.

For Debian Users

[Anonymous Coward]

libvorbis-aotuv-dev_0.b2-1_i386.deb
libvorbis-aotuv-0_0.b2-1_i386.deb
libvorbisenc-aotuv-2_0.b2-1_i386.deb
libvorbis-aotuv-0_0.b2-1_i386.deb
libvorbisfile-aotuv-3_0.b2-1_i386.deb
oggenc-aotuv_1.0.1+aotuv-2_i386.deb

You don't even need to uninstall your existing vorbis packages.

If it's a fork, it isn't Ogg Vorbis

[Anonymous Coward]

Calling it "Vorbis" in the title sort of makes it seem like you're pimping a certain format, rather than offering an unbiased pointer to some information.

If the creators of AoTuV hadn't considered their work important enough to require distinguishing from Ogg, they probably wouldn't have gone to the trouble of forking a new project from it. If you're going to put their work on a pedestal as exemplar, at least afford them the respect they deserve by acknowledging the real name rather than referring to its catchword roots for whatever reason.

Re:If it's a fork, it isn't Ogg Vorbis

[Anonymous Coward]

Calling it "Vorbis" in the title sort of makes it seem like you're pimping a certain format, rather than offering an unbiased pointer to some information.

It's a fork of the encoder. It still a real Ogg Vorbis bitstream, it'll play back in the same players, it's just a better represntation of the audio within the same framework than the reference encoder produces.

These patches may one day end up in the official Vorbis code, but xiph are dragging their heels accepting them. Let's hope they do soon.

We are banwidth wasters :)

[hkfczrqj]

Quoting a couple of posts in Hydrogenaudio:

a post:

What about all the /.ers?
Seems they were just interested in wasting bandwidth after all

the reply:

More than 500 people downloaded the samples through bittorrent only - not counting HTTP downloads! :B

I won't ever understand these people.

Disclaimer: I am NOT new here :)

Reason 2 is bogus.

[anti-NAT]

2 - It's an open-source format, i.e. it reeks of piracy and hackers in the minds of music player manufacturers and of the public

I think you are way off here.

Firstly, a number of portable players support Ogg Vorbis. There is a list of four here, I'm sure the number will increase.

Secondly, I'd doubt that many of the public know about Ogg Vorbis, let alone consider it to "reek[s] of piracy and hackers".

Furthermore, the "success" of P2P music sharing indicates that the public are the last group of people to have morals about the source or the format of the music they listen to.

Ogg isn't as widely used by the public, because it is not known by the public, it is as simple as that. That will change, as more and more players support it, and the public find out that it is a DRM free alternative to the flexibility restricted formats such as AAC.

why 16 bits /44100 Hz ?

[makapuf]

I'd wonder why songs are encoded from 16 bits / 44100 hz, while our ear can hear a difference with 96 khz/32bits and consumer sound cards can often do 48khz ? (if you can do better than rip from a CD, that is)

I mean, would 160 kbps sound better from 44100 or 48 / 96 khz ?

Re:why 16 bits /44100 Hz ?

[m0rbidini]

I'd wonder why songs are encoded from 16 bits / 44100 hz, while our ear can hear a difference with 96 khz/32bits and consumer sound cards can often do 48khz ? (if you can do better than rip from a CD, that is)

I mean, would 160 kbps sound better from 44100 or 48 / 96 khz ?

It's not really proved that we need a sampling rate higher than 44.1 kHz with 16 bits per sample resolution. And don't confuse kbps with kHz. Here's some threads quoted from the Hydrogen Audio FAQ.

High definition digital audio

Do CD sound good enough ?
Theoretical discussion : 44 KHz (CD) not enough !? (Nyquist etc.), plethora of distortion frequencies?
Listening test : 96 vs. 48 or 44.1 kHz sampling --> scientific test, perhaps here is the 1. listening test !
Another discussion : Sound, the human ear, and the digital world

Why was 44.1 kHz chosen ?
48, 44.1 khz, why no and/or not 40 khz, Only for 2 channel audio...........

SACD/DVDA
Help! Sacd Good Or Bad?, Does SACD ought to sound like crud?

96 kHz sampling rate
George, Watch this!!!....(96k) @George Massenburg, abstract in page 33

There is a lot of corporate FUD about this and sometimes it's hard to distinct it from good tests. Also, we have a lot of bad sounding CDs because they are badly mastered (search for Loudness Race). Some hybrid SACDs even have a badly mastered copy of the album on the Audio CD compatible layer... I wonder why.

Re:why 16 bits /44100 Hz ?

[EllisDees]

> I'd wonder why songs are encoded from 16 bits / 44100 hz, while our ear can hear a difference with 96 khz/32bits

No, they can't. I challenge you to find a single double-blind test (as this one was) that shows anyone who can hear the difference.

No WMA9 double pass encoding

[Anonymous Coward]

It seems this WMA9 encoding type has been omitted, so it's not a complete research to me.

Interesting MPC outlier

[mcg1969]

The MPC codec was neck-and-neck with Vorbis most of the time, except for one song by Debussy. What is interesting though is that it only encoded at 91kbps for that song---suggesting that perhaps if it were forced to use more bits it might have scored higher. It seems the heuristics it uses to determine how many bits it needs didn't quite work for that song.

Here's the thing

[TechniMyoko]

Ive listened to mmp3, ogg and wma versions of the same song and can't tell the difference. They all sound the same, so I just encode into the smallest format which is wma

ABR vs VBR isn't exactly fair

[Durandal64]

Apple's AAC implementation uses ABR rather than a purely VBR implementation. Look at the bitrate distributions. Ogg's bitrate was significantly higher than iTunes' in the majority of listening tests.

Re:ABR vs VBR isn't exactly fair

[CaptainPhong]

On an average body of music though, they will average out to be the same. Many of the samples chosen were picked because they are difficult to encode and a good VBR codec will increase bitrates on those samples. On easy to encode samples or quiet passages a VBR codec will have _lower_ bitrates. This isn't unfair - it's just that VBR has an advantage because it can be more efficient (which makes it inherently better unless you are doing something like streaming that can't handle the fluxuations.)

Re:ABR vs VBR isn't exactly fair

[Durandal64]

Yes, but the problem is that the average bitrates for the Ogg files were almost always higher than iTunes AAC, for example. iTunes AAC was always at 128 kbps average. One of the Ogg files was at 148 kbps average, meaning that it had a higher average bitrate and thus a larger file size. All we can tell is that Ogg sounds better than AAC when Ogg has a higher average datarate. That's not very useful. And considering how close Ogg and AAC were throughout the comparison, I'd be willing to bet that, bitrates being equal, AAC would be just as good as. if not better than, Ogg.

don't fear the ripper

[Doc Ruby]

All these codecs work on audio data that has to come from somewhere, typically CDs. Xiph offers the standard CDDA->WAV converter that feeds LAME and the other codecs. Sure, CDParanoia III v9.8 works very well, but it's slow: typically the best error recovery cuts performance by at least 65%. The Paranoia news page says

"March 27, 2001

Things on hold for now: No, that doesn't mean the project is dead, just that active development is on hold while we throw all the time we have available to get OggVorbis to 1.0 in a reasonable amount of time. Once Vorbis hits 1.0, we'll get back to Paranoia."

Vorbis hit 1.0 22 months ago:

"README 1.14 22 months xiphmont That's it. Full 1.0 libVorbis code handoff to release engineering."

Of course, this is OSS, and free, to boot (pun(s) intended ;). So it's hard to demand a new version, even after 2 years. But a lot has happened, especially in performance opportunities, since that release. So where's Xiph's committment to the expectations they created with that announcement? Where's the community effort to advance this essential tech?

Hacking the paranoia lib for device performance tweaks isn't my bag, baby. I would help another, lead, developer, by testing, or higher level hacking, or project organization support, etc. This library is the bottleneck for media format freedom - everyone on unix/linux has to use it, in one form or another. That widespread use usually drives progress in OSS. Where's the love?

Re:don't fear the ripper

[Dr.Dubious DDQ]

So where's Xiph's committment to the expectations they created with that announcement?

In the same place the committment to Theora (Now nearly a year behind the original "final release" date, and not even in Beta yet, and only being very sporadically worked on - though there's more going on with it now than there was for the 'dead' time that went on for so long after alpha1...the "Monty's the only developer and he's busy with Tremor [Integer-only vorbis for hardware players, if you didn't already know] for the forseeable future" period)

I'm beginning to think that Xiph, collectively, just doesn't have an attention span. NOW their big project is apparently connecting things to the Helix player. They're apparently "committed" to incorporating Theora support into Helix, too, which will be nice if they can finalize it (and Theora itself) before they get distracted by the NEXT project, whatever it may be...

As far as I can tell (as a "lurker" rather than a directly involved developer - YMMV) they are still debating what the specification for the OGG file format should look like in its final form, or so the only recent activity in the Theora mailing list seems to imply (revolving - as I amateurishly interpret it - around complaints that Ogg is hard to map to Microsoft(r) DirectShow(r)...)

I agree, though - it's FREE software, so even the incomplete stuff we get from them is still better than the nothing we'd get without them, so consider all of this a "lament" rather than a "complaint" from me...

Re:don't fear the ripper

[Doc Ruby]

OSS projects can suffer from the "poison placebo" effect, where too little is worse than nothing, because it detracts from the critical demand mass that would otherwise generate a project that delivered. This "poison placebo" is just the corollary to "the perfect is the enemy of the merely good", inhibiting development at the high end of project performance.

Bitrates!!!

[Stigmata669]

Did anyone actually read the whole results section? Vorbis, MPC and LAME were using VBR that resulted in average bitrates higher (and in some cases much higher) than 128kbps. Go back to the results page and scroll to the third from bottom table and have a look for yourself. On the samples where there was a clear or runaway winner, the bitrate for that codec is generally higher than the average. Only AAC and WMP actually had 128kbps samples.

For example, the final sample "Waiting" the winning codec MPC was actually 153kbps, followed by LAME and Vorbis at 144 and 148kbps. Unless I don't understand his algorithm, it doesn't seem as though he is taking these bitrate differences into account

The report claims to be a 128kbps Multiformat Test, and it's really a "128kbps standard preset on our encoder" Multiformat Test which may result in higher bitrates than 128kbps. I am sure that if you gave the static bitrate codecs the same bitrate as the resulting VBR average bitrates, they would be much more competitive.

Re:Bitrates!!!

[CaptainPhong]

If you had actually read some of the background info, you would know that the settings chosen for each encoder were ones that averaged 128kbps over a wide range of music (bitrate tests were performed beforehand to determine these settings). If you were to encode a large, varied collection of music with all of these encoders at these settings, the average bitrate would be quite close to 128kbps.

It would be profoundly moronic to tune each encoder's settings so that they got 128kbps over only this specific set of 18 30-second samples, or worse, to tune it so that it got exactly 128kbps on each sample. Likewise, there is no logical or mathematical basis to adjust the scores based on the average bitrate over that set of samples.

Re:Bitrates!!!

[Stigmata669]

No, I understand that the point of the test is to pick an encoder/codec that is good for a whole library of music, but I think that the benchmark appears to compare music quality/bitrate, which it does not. It is a guide for encoding your whole collection, not a benchmark to show the best codec quality:size.

The logical basis for tuning the VBR and CBR encoders to the same bitrate is simple: If you want to compare quality/kbps you need to do this. The whole point of audio compression (IMHO) is to get the best quality:size ratio. It is equally moronic to compare a 160kbps MP3 with a 128kbps AAC and announce that MP3/LAME is superior sound quality at ~128kbps settings.

Re:Bitrates!!!

[Anonymous Coward]

Did you actually read the whole results section?

"The unusual quality settings for MPC and Vorbis were chosen after testing several qualities over a wide range of albums and styles, and picking the setting that generated results closer to 128kbps."

Re:Bitrates!!!

[be-fan]

You seem to still miss the point. If I want to encode 100CDs to put on my iPod, it doesn't matter if Vorbis at a certain setting will give me 150kbps for certain songs and 100kbps for other songs, as long as they average out to about 128kbps per song. My metric is quality/kbps, but its quality/kbps averaged over the whole collection of music. Its simply a weakness of CBR encoders that they can't vary their encoding rate depending on song complexity, to give you the best overall quality/size ratio over your entire collection.

The tests are blind.

[nbanman]

The testing is ABX. I am not a scientist, but the methodology looks pretty good. The only thing that might be suspect is that the subjects could send in false reports after the testing was done.

The way it works is, you listen to a given music clip. You have three streams to choose from. One is the uncompressed .wav, and is labeled as such. The other two are not identified, and consist of the compressed source and the original source. You then rate the two unidentified sources based on how closely they approximate the original. Then you repeat the process five times for each of the codecs. When you're performing the experiment, you don't even know which codec you're testing at any given time.

Re:The tests are blind.

[ff123]

The only thing that might be suspect is that the subjects could send in false reports after the testing was done.

It is possible, but unlikely. Both the configuration file and the results file are encrypted, so the listener can't tell how he rated things until the public key is distributed after the test is completed.

ff123

Re:The tests are blind.

[nbanman]

Thanks for the info. I confess that I did not finish this current test, and that the last one I did was a long time ago (64kbps codecs, Ogg won, though I think I preferred AAC). With that one, the submitted results were in plaintext.

No HE AAC?

[benwaggoner]

I'm a little surprised that they didn't including HE AAC, a more recent MPEG-4 audio codec. It's best known in its AAC+ implementation from Coding Technologies. It's definitely entertainment quality at 48 Kbps at 44.1 stereo.

Anyone know why it got left off?

We also have parametric stereo AAC coming as well, which should be able to do entertainment quality at 24 Kbps.

Re:No HE AAC?

[ff123]

The AAC competitor was chosen on the basis of the winner of Roberto's AAC test, conducted a few months prior to this one. An HE-AAC contender was not available at this bitrate at the time. Moreover, I'm not sure that HE-AAC is actually competitive at this bitrate. Its goal is to improve low-bitrate encodings, and it may not produce transparency at higher bitrates.

ff123

Innovating with other codecs

[benwaggoner]

Bear in mind that WMA9 "Standard" is a legacy codec for Microsoft these days. They haven't revved the bitstream in years, in order to keep backwards compatibility.

There are new WMA9 codecs, though. WMA9 Professional, which goes up to 96 KHz 24-bit 7.1. It does have a 2-pass VBR 128 Kbps 44.1 stereo mode, and it'd be interesting to see that included in a future version of this test.

There is also WMA9 voice, which is better thought of as "WMA9 narrowband" since it includes a music mode as well. Only up to 22 KHz mono @ 20 Kbps, though, so it's really meant for modem streaming kinds of applications.

Re:Innovating with other codecs

[ff123]

There are new WMA9 codecs, though. WMA9 Professional, which goes up to 96 KHz 24-bit 7.1. It does have a 2-pass VBR 128 Kbps 44.1 stereo mode, and it'd be interesting to see that included in a future version of this test.

WMA9 Pro, bitrate VBR at 128 was tested in the previous multi-format test:

First 128 multiformat test

Although results aren't strictly comparable between tests, in that test WMA9Pro was essentially tied with iTunes AAC 4.2, so it probably would have ended up near iTunes and Lame in this one.

ff123

~2:1 compression with Apple lossless

[benwaggoner]

Actually, you'd get about 2:1 compression with Apple's lossles codec (for whatever fundamental reason, almost all lossless codecs, video and audio, work out to around 2:1 compression on average with natural content).

And since the average audio CD isn't full, I imagine you could get nearly 150 CD's on an iPod in that mode.

Still, I happily use 320 Kbps MP3 on mine, same as I use on my server for SqueezeBox access. I used to convert to 128 Kbps AAC for the iPod, but that was too much management trouble. Now I can just copy the files.

160 kbps

[azpcox]

So now at at the 128K level, the difficulty in perceiving differences is showing up on the graphs. Since portable players are coming out with more and more space these days (40 Gig iPod anyone), why not test at 160 kbps or 192kbps and then see if there is even a perceived difference. The file size from 128kbps to 160kbps is only going to increase by around 25-30% but you'll get a nearly indistinguishable file from the original.

I'd wager that at 192 kbps on any codec (except ATRAC by the looks of the numbers), only the real Golden Ears (TM) can hear the difference.

unfair comparison

[dtfinch]

The two winners had the highest average bitrates, 135 and 136 kbit, giving them an advantage over the encoders that outputted the correct bitrate.

Re:unfair comparison

[Anonymous Coward]

If you can tell the difference between 128k and 135k on even 2/3 samples I'll give you a job in my studio. Read the entire testing thread at hydrogenaudio to see that dozens of people took the time to encode a few thousand samples at the settings used in the test. It ensures that the average bit rate in a large CD collection will be 128k no matter the style or codec used.

Meaningless?

[wan-fu]

Though I think that this is a great idea for a study, aren't the conclusions found by this research ultimately meaningless? In his overall ratings graph, there are a total of eighteen samples and the other track-based analysis have fewer samples than that (N=17, N=15, etc.). This isn't even close to what is required for a good sampling of the US population! (I believe you need about 1,500 to get even close for most statistical comparison of US population.)

Add onto this the fact that no random sampling was done and this test is even more meaningless because of the self-selectivity of the participants. People who were sampled were most likely those with active Internet connections and those of the audiophile and/or Slashdot-type "Nerd" communities online. Naturally, we are more accustomed to hearing vorbis as a file format. Perhaps "Joe Blow" down the street hears a very different tune, but this method of sampling doesn't account for that, because Joe Blow never even heard about participating in this study.

Granted, that might be okay since I think perhaps the self-selectivity of the participants yield results that will match up with those who actually care, but I still can't see how the insignificant sample sizes used for this research can have any real meaning.

Re:Meaningless?

[ff123]

Though I think that this is a great idea for a study, aren't the conclusions found by this research ultimately meaningless?

Not meaningless; just not representative of the general population. As you said, the self-selection helps to yield results that represent the population of people who actually care.

For everybody else, probably all of the codecs are "good enough," even wma9 standard and atrac3.

ff123

Grammar police

[Anonymous Coward]

You mean, "Other people just might have different needs and desires from yours when it comes to compressing audio."

Well, you know what to do!

[Anonymous Coward]

Get ripping!

What's wrong with Ogg Vorbis?

[Anonymous Coward]

Vorbis doesn't have a lot of ad money behind it pushing it.

But this is the day and age of the Internet, where WinAmp took off like a rocket and made its author very rich. "Ad money" shouldn't be a factor.

There is something inherently wrong with Ogg Vorbis. Think about it: Despite being free, none of the major manufacturers (Panasonic, Sony, Phillips) are embedding it in their electronics, so you get also-rans -- "iRiver"? "Rio"? Not exactly household names.

Possible problems:

1. Firmware instability. The list at http://wiki.xiph.org/VorbisHardware says stuff like "firmware 1.41 out, fixes some problems with Vorbis files", "iRiver flash players support Vorbis with firmware update. ", "Note that firmware versions prior to 1.25 cause stability problems for some people", and "There are reportedly problems with some versions of the firmware...". Is that a result of the algorithm? Why are they having so many problems writing stable firmware? I read a review of a Rio player where the reviewer complained about lockups and warned readers not to buy it.

2. Floating point format , ie Ogg Vorbis mainpulates everthing in floating point format. While this is no problem for a standalone computer, this is an additional expense for a portable battery operated device, because you have to have an additional floating-point coprocessor. This extra chip

eats up the battery more quickly

adds to the expense of the device.
Alternatively, you convert everything to fixed point, and thereby lose the fidelity of the original Ogg Vorbis result. Indeed, is it possible that the internal floating point is the reason that Ogg Vorbis has a better sound fidelity than MP3 simply because the error quantization is less in floating point numbers than integer discretization?

Re:What's wrong with Ogg Vorbis?

[QuantumKnot]

Xiph.Org released a fixed point Vorbis library quite a while ago called Tremor.

atrac3...

[Anonymous Coward]

wow.. atrac3 sucks badly.. but they didnt even bother to rate atrac3plus It is like 48Kbps and sounds better than atrac3 132Kbps.

of course, its very heavy into the DRM (like all sony stuff), which kills it, but the compression is great.

Why keep calling AoTuV a fork?

[xiphmont]

It's odd to keep hearing this code referred to as a 'fork'. Yes, it's based on our reference code while doing further tuning just like all the free MP3 encoders are based off of the original dist8 or dist10.

Fork seems to imply that they're trying to make something incompatible or doing it without our blessing. Neither is true! We never wanted to have *the* only encoder. Nor did we want to be the only people trying to improve Vorbis's encoding.

AoTuV is a 100% real Vorbis encoder and the results of the test speak for themselves. Aoyumi and crew deserve kudos, and I'm glad to see them working on improving Vorbis encoding.

Monty

Re:Why keep calling AoTuV a fork?

[sholden]

Because what you described is the very definition of a fork.

A fork doesn't have to be a "bad" thing.

Firefox is a fork (an internal one mind you) of Mozilla without any "badness".

JACK, Jack, jack & the ripper

[xiphmont]

The speed of CDparanoia is not limited by the cdparanoia code. Regardless of future improvements, speed gain would never be one of them. If anything, additional error correction would only ever make it slower. Your limit is Linux forcing programmed I/O because the IDE subsystem doesn't know how to use DMA on non-multiple-of-512-byte sector sizes. CDDA is 2532 bytes per sector. Linux 2.6 partially fixes this.

Also, cdparanoia (III) was finished long ago. It has not bitrotted. As new kernels came out, we+others kept it up to date. The distribution maintainers have added whatever fixes have been necessary for their distros. Nothing that worked in 1999 is broken today.

In summary... paranoia does 100% of what *I* need it to. I write software that I need. I don't have to keep releasing 'improved' versions of software that already works as an ego-trip or to placate a marketing department desperate to sell you the same thing in a new box every six months.
Others have expressed interest in doing new things with paranoia, but no one has followed through... at least not yet. Paranoia isn't all that complicated to use or hack. That speaks to a pretty damned low demand for new versions.

The website: yah, OK, I'm lousy at writing HTML updates. My diary hasn't been updated in three years. There is certainly a website attention span problem ;-)

Theora: I'm not one of the primary coders today, I only did the initial code import. Also, the Helix project has required relatively little time; Real has done nearly all the heavy lifting on integration there.But, if 'Theora is dead', why does CIA show 500 commits in the past two months?

DirectShow issues can be summed up as 'ugh, what an awful system'. But we'll make it work. The discussion about mux was proposed changes to spec. Voluminous discussion reveals what we have now is still the best option, as designed five years ago.

Monty

What about cross-polination?

[kjoonlee]

It's nice to hear that, but is cross-polination going to happen?

How much are you willing to delegate to the non-xiph tuning crew?

cdparanoia updates?

[kjoonlee]

Monty, I think I remember you mentioning that cdparanoia was designed to work with CD-ROM drives that had 1 meg or less of cache. Most CD-ROM drives have more than 1 meg now.

Will an update for modern drives be released, if cdparanoia is not so difficult to hack..?

Re:cdparanoia updates?

[xiphmont]

I'll probably get to it eventually actually; ATAPI finally added packet commands for dealing with cache management on ATAPI drives, just like SCSI always had (but most drives just ignored). ..Or you can increase the cache thrashing constant and rebuild. These days computers have more than 8 meg, it's probably worth doing ;-)

Monty

Or perhaps, call it a "tuning"

[QuantumKnot]

Since that is what it essentially is...a highly tuned version of the reference encoder. That is probably why Garf called his encoder GTune (Garf Tuned) and I've followed suit with QKTune. Also, aoTuV stands for "Aoyumi's Tuned Vorbis" IIRC. :)

Vorbis needs more working binaries!

[Anonymous Coward]

Vorbis needs more working binaries!

On vorbis.com, the only Windows players listed are Winamp 2, Winamp3, foobar2000, and zinf. THAT'S IT?! Only four (three?) Windows players support Vorbis??

There should be a WORKING .ACM codec for Vorbis (and for Speex). Let people try out these free codecs using Windows Media Player or Sound Recorder, WITHOUT making them download and install more bloatware or broken freeware crap. (And note that some non-Microsoft media players also support .ACM files.)

I told you it was RIGGED

[Anonymous Coward]

.

I told you it was RIGGED.

.

HEY, MODS!!!!!

[Anonymous Coward]

The parent, GP, GGP, GGGP, etc., are all off-topic. Get off you're asses and do something about it!