Domain: ff123.net
Stories and comments across the archive that link to ff123.net.
Comments · 48
-
Re:And the point of this is?
So some people will say Codec A sounds best. Some will say Codec B sounds best. Some will say that Codecs A and B suck donkey shit and Codec C sounds best. What exactly does this prove?
You're commenting on something you have no knowledge about.
This is a blind ABX listening test. You can go read about what that is here.
This is the program being used in this test.
It proves whether or not a statistically significant difference in perceived audio quality can be found between the various codecs by the group of testers. It is valid and scientific. If it wasn't, you wouldn't see the Hydrogen Audio forum associated with it, because they're actually interested in the real deal, and it's against their forum rules to make claims without evidence. (See here. Rule #8.) That would also be why many encoder and codec designers/developers are active in the forum, because it's where they get real empirical feedback.
-
Re:Like the phonograph.... The what?
If you dare, check out this page in order to train your ears to be more sensitive to lossy compression artifacts:
http://ff123.net/training/training.htmlInteresting. The worst for me are the ones that sound like little skips in the rip, as in Blackbird and Your Latest Trick. A lot of the other artifacts, though, even when I could kinda-sorta hear them (usually they either sounded "underwater" or "less bright"), weren't all that distracting. I'd give up some compression to get rid of them if I cared about the songs, but I wouldn't reject the encodings if they were all I had. That is to say, I've heard way worse (like most YouTube audio).
-
Re:Like the phonograph.... The what?
Low quality mp3s sound more like you're listening to music with cotton in your ears.
That is the case for music which has been decimated with a low-pass filter (i.e. the high frequencies are not "passed" through). But there are other artifacts like pre-echo (before a sharp attack like from a cymbal or castanet there is kind of a echo or "smear" added to to the music).
If you dare, check out this page in order to train your ears to be more sensitive to lossy compression artifacts:
http://ff123.net/training/training.html -
Re:Lets see... where to start...
BTW, altering the overall volume down by 2dB will introduce significant undesirable quantization noise. I see no good reason why you should do so
Not being a signals and systems engineer and able to describe the problem, this is the best link I could find. Excerpt from http://ff123.net/norm.html:
"How is this relevant to audio coding? If you take a signal that is heavily compressed to full scale, and try to filter it (lots of filtering is involved in mp3 encoding and decoding) the resulting signal can easily go over full scale. Here's an example... Download this file redacted (it's only 80k) Note the peak amplitude. Now filter it - try something like a low pass filter at 18kHz. So you're removing energy, right? Now look at the new peak amplitude - it's higher than before, even though you've removed part of the signal. A simlar (sic) thing, to a lesser extent, is causing clipping to occur in mp3 files." (when they are being decoded)
My understanding is that imperfect D/A converters on your sound card can also clip when fed heavily compressed near-full-scale data. -
Re:Loudness War
The constant re-allocation of the available bandwidth among the sub bands, and the way the quantization noise follows it, is part of the problem. It sounds like subtle amplitude modulation of the signal to me, like with a badly set up multiband compressor, and you can hear parts of the ambient noise floor vanishing and re-appearing. Other distortions are caused by the ringing of the FFT filters.
Another sound containing short high energy impulses that MP3 encoders really don't like are clavs.
http://ff123.net/preecho.html
I bet I could make a sound combining simultaneous swept sines, clav like impulses and jangling keys that would obviously fox any mp3 encoder.
It's not really fair as it's not really music and is designed to reveal artifacts, but it would be a good educational tool. -
Probability that 4 out of 5 is random guessing
..is 18.7%
http://ff123.net/abx/abx.php -
Re:How much of this is just OGG fans voting?
For fuck's sake read my reply. The ITU standard implies a reference and a hidden reference (=> double blind): Take a look at this to get an idea about what the test looked like.
-
Artifact training
hmmm. I'm surprised no one mentioned this. You migh want to head on over to ff123's artifact training page and listen to the examples there if you aren't familiar with the kinds of artifacts that can occur with lossy compression of audio.
-
Re:A consideration.
The truth is 90% of the people can't tell 256k mp3 from the original cd track.
Everything on MP3.com had to be encoded at 128 kbps. Some people put a lot of work into tweaking their encoder settings to get decent-sounding audio, but most didn't bother.
I agree that 256 kbps would be fine for downloads (with a good encoder like LAME or FhG). But if I was paying for the music, I'd want the option of getting a losslessly compressed file. -
Re:Error bars
Fisher LSD is better than I thought, but you are still going about it the wrong way. The only way the Fisher LSD is protected is if you do the ANOVA first and it shows a significant difference between the groups, then you do the post hoc tests accordingly.
That is exactly what is happening here. A blocked ANOVA (each listener is a block) is used to determine if there is a significant difference anywhere. Fisher LSD is used if the ANOVA is significant (this is what it means to be protected).
As I mentioned in another message, if people really want to nitpick, they should take the raw data and run it through my bootstrap resampling program, which does not assume a normal distribution and corrects for multiple comparisons.
http://ff123.net/bootstrap/
However, you can't get the nice graphs that Roberto showed (tabular pvalues are the best you can get), and the difference in the conclusions will be something that is more conservative than the Fisher LSD. But again, who cares? So there is a bigger chance that you'll get a type I error. Get over it. It's not a black and white thing, where if you do it one way the results are totally wrong, like you're making it out to be. Are the conclusions reasonable? Yes. Are they significant to 95% confidence? Maybe if you use ANOVA/Fisher LSD. Maybe not (90% confidence?) if you use another, more conservative method. It's not the end of the world.
BTW, Blocked ANOVA with Fisher LSD is what the book, "Sensory Evaluation Techniques," by Meilgaard, Civille, and Carr recommends for subjective tests like these.
Also BTW, there really isn't one or two results which are strikingly different from the others. There was no low anchor included in this test, which would have produced such a result.
ff123 -
Re:Don't use ANOVA here
The intervals in the rating scale are 0.1 steps, which is close enough for argument's sake.
And ANOVA is a robust method as you've commented, so it's probably reasonable to assume normality. In any case, the raw data is available for any stats weenies to play with, and there are a couple of more conservative methods besides the Fisher LSD readily available to try, if anyone has an uncontrollable urge.
I point you to:
http://ff123.net/friedman/stats.html
where you can run a non-parametric analysis of the raw data using a web form if you like. Or download the program to correct the pvals for multiple comparisons using Tukey's Honestly Significant Difference instead of the Fisher LSD (either paramteric or non-parametric).
And if you want to really geek out, you can do a bootstap resampling method with different methods of correcting the pvals for multiple comparisons:
http://ff123.net/bootstrap/
But let's not lose the forest for the trees here. The blocked ANOVA/Protected Fisher LSD used for the test provides a reasonable (if not the most conservative) summary of the results.
A couple of more important weaknesses of the test are (and they're related):
1. Only 12 samples were used. Although this is probably close to the practical limit, more samples always provides a more comprehensive picture. The results are suggestive, but not definitive.
2. The selection of the samples makes a difference. If you listen mainly to classical for example, this test may not be representative for you. -
Re:Don't use ANOVA here
The intervals in the rating scale are 0.1 steps, which is close enough for argument's sake.
And ANOVA is a robust method as you've commented, so it's probably reasonable to assume normality. In any case, the raw data is available for any stats weenies to play with, and there are a couple of more conservative methods besides the Fisher LSD readily available to try, if anyone has an uncontrollable urge.
I point you to:
http://ff123.net/friedman/stats.html
where you can run a non-parametric analysis of the raw data using a web form if you like. Or download the program to correct the pvals for multiple comparisons using Tukey's Honestly Significant Difference instead of the Fisher LSD (either paramteric or non-parametric).
And if you want to really geek out, you can do a bootstap resampling method with different methods of correcting the pvals for multiple comparisons:
http://ff123.net/bootstrap/
But let's not lose the forest for the trees here. The blocked ANOVA/Protected Fisher LSD used for the test provides a reasonable (if not the most conservative) summary of the results.
A couple of more important weaknesses of the test are (and they're related):
1. Only 12 samples were used. Although this is probably close to the practical limit, more samples always provides a more comprehensive picture. The results are suggestive, but not definitive.
2. The selection of the samples makes a difference. If you listen mainly to classical for example, this test may not be representative for you. -
Re:If I were to buy a new mp3 player...Good points re: licensing. Not sure they really affect me, or most people, since I'm not commercially distributing music.
Anyways, here's listening tests at 64kbps. HE AAC beats Vorbis handily, QT AAC not so well... but that's at 64 kbps. 128kbps tests are here.
Overall, Vorbis seems to be great at low bitrates - 64 kbps, for instance. At higher bitrates, though, it doesn't keep up compared to AAC.
-T
-
You suspect correctly about the audio quality
All things being equal (source quality, etc.), which they probably aren't, AAC should beat out WMA handily at bitrates like what the iTMS and Wal-Mart are using. The only chance WMA would have of approaching AAC in quality at that bitrate would have been if Wal-Mart had used WMA Pro, but because of the lack of hardware player support for WMA Pro, that probably won't happen soon.
I haven't seen tests directly comparing AAC to WMA (non-Pro), but Roberto Amorim's testing at 128kbps with AAC and WMA Pro and ff123's testing of a different AAC codec against WMA non-Pro probably say enough.
Also, Apple has actually spoken about the quality of the sources that they encode from (the original masters rather than CDs themselves), and Wal-Mart hasn't.
I do hope that whoever elects to actually directly compare the quality of Wal-Mart's music to Apple's doesn't just look at frequency analysis to do it. Apple's AAC lowpasses at 16 KHz, but to use this as some sort of indication of quality is ludicrous.
-
Re:rjamorim conducted the test, not HA
Did you forget to credit ff123, without whom none of the tests would have been possible?
-
Re:why lossless for live?
So what i'm asking you, BUSTER, is: has an "scientifically valid" "are differences audiable" experiment been done or not?
Yes. Individuals have performed such tests and established bit rates at which they became unable to distinguish the difference. Here is one such test. I have never seen any test results in which the listener could, with any statistical significance, identify a 320k MP3 vs. a CD of the same music.
But the important thing is whether you can hear the difference. If you are downloading music for your own enjoyment, it makes no difference whether I, some other random Internet user, or a panel of listeners could hear the difference or not. What matters is whether you can hear the difference. And the only way to be sure when the differences are subtle is with a double-blind test.
Your "power of suggestion" theory needs this experiment and yet you claim it has never been done.
It's not my "theory." It's accepted as scientific fact. Ever heard of "the placebo effect"? That's an example of the power of suggestion. Why do you think that clinical drug trials have a control group who receive placebos? It's so that the the power of suggestion does not cause a patient to imagine drug effects that are not there. -
Ratings are nice, but...
Why not show spectrum analasys of different songs encoded into the given formats too?
Perhaps I'm just an audio freak, but I would find that a lot more interesting than just ratings. -
Spectum analysis in invalid
Learn why you shouldn't use spectral analysis to determine lossy codecs' quality.
The most respected technique is double-blind testing using an ABX tool such as PC ABX, WinABX or ABC/HR.
More info on conducting blind tests can be found at the PC ABX site. -
Re:Nifty, but...
mp4 is video.
Cmon now. You can do MP4 audio-only files, just like an MP3 file is really an MPEG-1 (or is it 2? whatever), audio-only file. People have even started to compare them since QuickTime 6.0 came out. -
Re:Killer App?
-
BladeEnc and Fastenc
I have only used bladeenc. Is there much of a difference?
Yes, as much difference exists between a Hyundai Excel and a Porsche :-)
Bladeenc is the worst MP3 encoder out there. Not only do its MP3s sound terrible, it is very slow at encoding. On the other hand, Fastenc has been the best offering from Fraunhofer [co-inventors of the MP3 format] so far. IMO, its 128 kbps MP3s remains unbeaten by any other encoder's, not only quality-wise but also speed - on a Pentium 233, I was getting 3.2x realtime; on a Celeron 400, about 4.5x.
Note that Fraunhofer's codecs [a la l3enc and mp3enc] usually go for about $300-$400. Then consider Fastenc is free. Amazing, if not incomprehensible. I believe the Win32 standalone build was a fluke which was soon pulled off [hence the Geocities mirror]. Now it's only available as an inextricable part of other programs [CoolEdit 2000, MusicMatch etc.]
You will notice the difference between encoders if you know what to look out for: a warbling, swishy, underwater-like sound distortion is the most prominent artifact. Once you encode a few files with Fastenc and do a careful comparison with your old MP3s on a good set of headphones, you will never be able to tolerate BladeEnc again. No wonder Tord [the project maintainer] recently abandoned development.
If you're encoding at higher bitrates, I would recommend LAME, another GPL'd encoder which should be transparent at 170 kbps and above. The recommended setting is "--alt-preset standard", which should average out at 200 kbps.
But if space is important and you prefer 128 kbps, then Fastenc is the way to go. Note that it's Win32 only, but it should run fine under Wine.
For more information on audio encoding, quality comparisons and a lot more, visit Hydrogen Audio and ff123.net.
Finally, if you intend to rip music only for use on your computer, I would recommend Ogg Vorbis instead of MP3. Not only is it free in every sense of the term, it is possibly the highest quality audio encoder out there - even 100 kbps sounds transparent to most people. I switched a long time ago and have no regrets. Its only Achilles' Heel is hardware support [car players, portables and so on] but this should be addressed soon - Ogg users have been quite vocal about it :-)
Have fun. -
Re:Ogg Vorbis support?
Panos' audio test has been ridiculed all over the audio compression world. Look here for a review of some reviews, including the Panos' test (ekei.com)
Better tests to refer to would be ff123's test or the new C'T test (which IMHO isn't as good as ff123's). There were some complaints about C'T not using the optimal settings for Ogg Vorbis but it didn't method since Ogg Vorbis clearly won the test anyway. -
"Germany's respected computer magazine c't"
c't is a good computer magazine, but it's not an professional audio magazine.
If you want to learn about audio encoding, listening tests and so on, visit audio-illumination.org and ff123.net
A discussion about heise's listening test can be found here.
-
look around the site
-
Re:Quality
It's difficult to do fully double-blind tests, because many people who are interested in this kind of structured testing can identify a particular format by the type of artifacts it produces. Having said that here is a pretty rigorous test of a number of formats. Mp3pro comes out top, ogg a very close second, with aac and wma trailing. This only covered 64kpbs samples.
Note that mp3 wasn't even included - pretty much everyone interested in the topic already knows that mp3 doesn't match up to the best formats such as ogg or mpc (probably not even wma), so there's not a lot of point testing it. It's only use is its wide-ranging hardware compatibility (a good use, mind you :) )
Oh, and mp3 isn't free, the cost is just hidden.
Cheers, Paul -
Re:Thank god for ogg!
Well, for low bitrates: ff123's low bitrate test might be what you are looking for.
gnoshi -
Re:Who cares about 64 kbps tests?Take a look around his site. He's done other listening tests, including two separate tests at 128 kbps, with MPC and AAC highest in the first and MPC and Ogg Vorbis highest in the second.
Note the tests at 128 kbps seem much more difficult to discern a clear winner without resorting to some statistical work.
-
Re:Who cares about 64 kbps tests?Take a look around his site. He's done other listening tests, including two separate tests at 128 kbps, with MPC and AAC highest in the first and MPC and Ogg Vorbis highest in the second.
Note the tests at 128 kbps seem much more difficult to discern a clear winner without resorting to some statistical work.
-
Re:Who cares about 64 kbps tests?Take a look around his site. He's done other listening tests, including two separate tests at 128 kbps, with MPC and AAC highest in the first and MPC and Ogg Vorbis highest in the second.
Note the tests at 128 kbps seem much more difficult to discern a clear winner without resorting to some statistical work.
-
Re:Stats note: 95% confidence levelPlug the results into the linked non-parametric stats analysis page, and you find:
mp3pro is better than wma8 (99.9% confidence)
mp3pro is better than aac (99.9% confidence)
oggq0 is better than aac (99.3% confidence)
oggq0 is better than wma8 (99% confidence)
ogg64 is better than aac (97.2% confidence)
ogg64 is better than wma8 (96.1% confidence)
Is 99% good enough for you? Or perhaps you should just take the two at 99.9%?
Dammit, the lameness filter is kicking in. No, these are *not* junk characters - I'm trying to show the peon some useful statistical information, you worthless piece of software. I've already removed all the hyphens, what the hell more do you want me to do? Is a percentage sign 'junk'? Is a question mark? Is a space? What the *fuck* use is this, when it doesn't stop all the crapflooders in any way whatsoever - they just flood with random gay/incest/beastiality sex stories instead... I've been posting on this site for years, and for my sins haven't crap-flooded once - give me a LITTLE FUCKING LICENCE TO POST MATERIAL.
-
ALL YOUR BASE IS OGG TO US!
Vorbis at 96kbps is usually somewhere between the quality of a good 112 or 128 cbr mp3, and I and quite a few other are already in the belief, after early testing from 1.0-ish CVS-code, that it is better than wma8 at 64kbps.
ff123 will be conducting a 64 kbits/s blind listening test where people will send in their results, and that will show how vorbis stacks up against the likes of wma8, mp3pro and quicktime-AAC.
IMO it doesn't really matter if it is better.. if it is at least comparable, than that should be enough for us to make the switch. Because besides being a flexible codec of high quality, it is open source AND completely free of patents (amazing!).. oh yeah, plus it has that really cool bitrate-peeling feature. Anyway, this is one of the few chances we have to get something right in the computer world (for a change!), so let's not blow it! Spread the word and take your hats off for xiph and vorbis!
The waiting is over people, at last we can start ogging for real! ^_^ -
OGG: Already supports 24bit, 96+KHz, 255 channelsVorbis beats your perfered codec on all your own points:
- Unbiased listening tests prove OGG sounds better then WMA8 at 128Kbit/sec.
- Current software (not just the format) supports 255 channels, 24bit sample depth, 96KHz+ (though it's not tuned for >48KHz, it works fine and gives acceptable results)
- The Vorbis format is far more extensiable the the WMA8 format, there should be room for at least another 20% quality vs bitrate increase with no decoder change.
- Most modern mp3 players (the same ones which can play WMA) are technically able to be firmware upgraded; all thats missing is your demand.
-
Re:WMA 8 is the way
* Much better than OGG and MP3
* Picture perfect at 128 kbit/s
And what is this comment based on? These results have been pointed out in comments for previous articles, but I'd like to mention them again. ff123 has been conducting double blind tests comparing various audio codecs, and the results are here.
The following is from the page:
Comparisons in red below are true as a group with 95% confidence.
ogg is better than wma8
mpc is better than wma8
ogg is better than xing
mpc is better than xing
aac is better than wma8
aac is better than xing
lame is better than wma8
lame is better than xing
Looks to me like WMA8 got beat by pretty much everything... But hey, what good is statistical analysis anyways... -
Re:Double Blind Listening Tests... Where ???We're getting there. RC3 has only been released for about an hour
:)
The best tests we have at the moment were conducted by
ff123 at 128kpbs. There have been two so far (the second is technically still underway, although it's now based on outdated encoders, so I imagine a third will start fairly soon). The
first listening test compared RC2 Ogg Vorbis, LAME MP3, Xing MP3, Liquifier AAC, MPC, and WMA8. The formal analysis showed that, on the file compared, the encoders could be divided with 95% confidence into three groups (from best to worst):
- MPC and AAC
- WMA8 and LAME and OGG
- XING
The second test used a CVS version of OGG from about a month and a half ago. This time there are three test samples which participants can choose to evaluate. While technically still underway,
the interim results can be found here. Of the three test samples, the first can't discriminate between the encoders, the second looks like it will but needs more listeners (and the results so far look interesting), and the third discriminates well, to the extent that it shows that Xing and WMA8 are statistically much worse on that clip than all the others.
Now all we need is a third test with the latest updates of all the encoders - since we now have a new stable version both of Ogg Vorbis (RC3) and LAME (3.91). - MPC and AAC
-
Re:Double Blind Listening Tests... Where ???We're getting there. RC3 has only been released for about an hour
:)
The best tests we have at the moment were conducted by
ff123 at 128kpbs. There have been two so far (the second is technically still underway, although it's now based on outdated encoders, so I imagine a third will start fairly soon). The
first listening test compared RC2 Ogg Vorbis, LAME MP3, Xing MP3, Liquifier AAC, MPC, and WMA8. The formal analysis showed that, on the file compared, the encoders could be divided with 95% confidence into three groups (from best to worst):
- MPC and AAC
- WMA8 and LAME and OGG
- XING
The second test used a CVS version of OGG from about a month and a half ago. This time there are three test samples which participants can choose to evaluate. While technically still underway,
the interim results can be found here. Of the three test samples, the first can't discriminate between the encoders, the second looks like it will but needs more listeners (and the results so far look interesting), and the third discriminates well, to the extent that it shows that Xing and WMA8 are statistically much worse on that clip than all the others.
Now all we need is a third test with the latest updates of all the encoders - since we now have a new stable version both of Ogg Vorbis (RC3) and LAME (3.91). - MPC and AAC
-
Re:Double Blind Listening Tests... Where ???We're getting there. RC3 has only been released for about an hour
:)
The best tests we have at the moment were conducted by
ff123 at 128kpbs. There have been two so far (the second is technically still underway, although it's now based on outdated encoders, so I imagine a third will start fairly soon). The
first listening test compared RC2 Ogg Vorbis, LAME MP3, Xing MP3, Liquifier AAC, MPC, and WMA8. The formal analysis showed that, on the file compared, the encoders could be divided with 95% confidence into three groups (from best to worst):
- MPC and AAC
- WMA8 and LAME and OGG
- XING
The second test used a CVS version of OGG from about a month and a half ago. This time there are three test samples which participants can choose to evaluate. While technically still underway,
the interim results can be found here. Of the three test samples, the first can't discriminate between the encoders, the second looks like it will but needs more listeners (and the results so far look interesting), and the third discriminates well, to the extent that it shows that Xing and WMA8 are statistically much worse on that clip than all the others.
Now all we need is a third test with the latest updates of all the encoders - since we now have a new stable version both of Ogg Vorbis (RC3) and LAME (3.91). - MPC and AAC
-
Re:Double Blind Listening Tests... Where ???We're getting there. RC3 has only been released for about an hour
:)
The best tests we have at the moment were conducted by
ff123 at 128kpbs. There have been two so far (the second is technically still underway, although it's now based on outdated encoders, so I imagine a third will start fairly soon). The
first listening test compared RC2 Ogg Vorbis, LAME MP3, Xing MP3, Liquifier AAC, MPC, and WMA8. The formal analysis showed that, on the file compared, the encoders could be divided with 95% confidence into three groups (from best to worst):
- MPC and AAC
- WMA8 and LAME and OGG
- XING
The second test used a CVS version of OGG from about a month and a half ago. This time there are three test samples which participants can choose to evaluate. While technically still underway,
the interim results can be found here. Of the three test samples, the first can't discriminate between the encoders, the second looks like it will but needs more listeners (and the results so far look interesting), and the third discriminates well, to the extent that it shows that Xing and WMA8 are statistically much worse on that clip than all the others.
Now all we need is a third test with the latest updates of all the encoders - since we now have a new stable version both of Ogg Vorbis (RC3) and LAME (3.91). - MPC and AAC
-
Re:Double Blind Listening Tests... Here!ff123 from the r3mix.net/hydrogenaudio.org forums is conducting automated ABX double blind tests comparing Vorbis, mp3 (several encoders), AAC, WMA and MPC. The best part of this is... everyone can participate.
If you want to take the listening test yourself, read the instructions and jump in. For now, there's also a page of interim results, but to quote ff123, "Major conclusion: I need more listeners!"
Monty
-
Re:Double Blind Listening Tests... Here!ff123 from the r3mix.net/hydrogenaudio.org forums is conducting automated ABX double blind tests comparing Vorbis, mp3 (several encoders), AAC, WMA and MPC. The best part of this is... everyone can participate.
If you want to take the listening test yourself, read the instructions and jump in. For now, there's also a page of interim results, but to quote ff123, "Major conclusion: I need more listeners!"
Monty
-
Re:Double Blind Listening Tests... Where ???How about ff123's 128kbit double blind test. It's using RC2 as a basis, and the quality improvements since then have been quite substantial.
ff123 is planning more tests at such bitrates - it's difficult to organise a test, gather results and publish findings on the same day as an RC.
-
Re:I'm pleased...There was one recent newspaper test, done very badly, comparing files encoded at different bitrates, non-blind, which said WMA was best. This was about as far from scientific as it is possible for an 'objective' test to be. Luckily, there is a properly organised test underway to compare WMA, MPC, AAC, MP3 and Ogg at 128kpbs (which is still a very popular bitrate). To participate, go to
http://ff123.net/128test/instruct.htmlThis test is actually being analysed properly, using the insights gained from his previous 128kbps test, whose results you can find here). You can see the preliminary results here: http://ff123.net/128test/interim.html. In particular, look at the results of the last of the three test files. With overall 95% confidence we can say that on this test clip:
mpc is better than xing
ogg is better than xing
lame is better than xing
aac is better than xing
mpc is better than wma8
ogg is better than wma8
lame is better than wma8
aac is better than wma8
Aside from this, listening tests have previously shown that WMA8 is better than WMA7 at the low end (sub 64kpbs), at the expense of being worse at the middle to high end (112kpbs upwards). For listening to music on anything that costs more than $10, you would do best to stay away from WMA (and, looking at the other results on the page I previously mentioned, from Xing encoded MP3s as well...).
-
Re:I'm pleased...There was one recent newspaper test, done very badly, comparing files encoded at different bitrates, non-blind, which said WMA was best. This was about as far from scientific as it is possible for an 'objective' test to be. Luckily, there is a properly organised test underway to compare WMA, MPC, AAC, MP3 and Ogg at 128kpbs (which is still a very popular bitrate). To participate, go to
http://ff123.net/128test/instruct.htmlThis test is actually being analysed properly, using the insights gained from his previous 128kbps test, whose results you can find here). You can see the preliminary results here: http://ff123.net/128test/interim.html. In particular, look at the results of the last of the three test files. With overall 95% confidence we can say that on this test clip:
mpc is better than xing
ogg is better than xing
lame is better than xing
aac is better than xing
mpc is better than wma8
ogg is better than wma8
lame is better than wma8
aac is better than wma8
Aside from this, listening tests have previously shown that WMA8 is better than WMA7 at the low end (sub 64kpbs), at the expense of being worse at the middle to high end (112kpbs upwards). For listening to music on anything that costs more than $10, you would do best to stay away from WMA (and, looking at the other results on the page I previously mentioned, from Xing encoded MP3s as well...).
-
Re:I'm pleased...There was one recent newspaper test, done very badly, comparing files encoded at different bitrates, non-blind, which said WMA was best. This was about as far from scientific as it is possible for an 'objective' test to be. Luckily, there is a properly organised test underway to compare WMA, MPC, AAC, MP3 and Ogg at 128kpbs (which is still a very popular bitrate). To participate, go to
http://ff123.net/128test/instruct.htmlThis test is actually being analysed properly, using the insights gained from his previous 128kbps test, whose results you can find here). You can see the preliminary results here: http://ff123.net/128test/interim.html. In particular, look at the results of the last of the three test files. With overall 95% confidence we can say that on this test clip:
mpc is better than xing
ogg is better than xing
lame is better than xing
aac is better than xing
mpc is better than wma8
ogg is better than wma8
lame is better than wma8
aac is better than wma8
Aside from this, listening tests have previously shown that WMA8 is better than WMA7 at the low end (sub 64kpbs), at the expense of being worse at the middle to high end (112kpbs upwards). For listening to music on anything that costs more than $10, you would do best to stay away from WMA (and, looking at the other results on the page I previously mentioned, from Xing encoded MP3s as well...).
-
For some of us......it sure ain't worth it.
There recently was a post in for an audio test. Let's see... http://slashdot.org/article.pl?sid=01/10/31/21272
3 1&mode=thread, which leads to this: http://ff123.net/128test/instruct.htmlI participated in this test. There were 3 samples of music, each encoded 6 different ways. I could not tell the difference between any of the 18 encoded files and the originals. No way, no how. As a caveat, I should mention that I was in the US Navy for 10 years and worked in an engine room for many of those years, so I was exposed to high decibels for long periods of time. I found a tone sweep somewhere and played it. I can't hear anything over about 13 kHz.
So, for me, this is overkill. I've been ripping my CD's to MP3's pretty steadily. I've found a few that sound bad. I'm marking them and when I get through with all of them, I'll re-encode those at a higher bit-rate. Total cost to me: 40GB hard disk - $200; Winamp - free; Lineout cables to preamp - $10. Already had the computer and sound card. At this point, I'm not sure 40GB is enough, but if it comes down to it, there's a lot of crap in amongst what I've ripped so far. (Not every song on every album is a good one.)
So, a word of advice from an old (38) geezer: Buy expensive stereo stuff when you're young. Also, don't blow out your ears too early - wear hearing protection at concerts, while shooting guns, perhaps at work if it's noisy there. You'll thank yourself later.
-
Re:I am reminded...
The tests that you link to were done incredibly badly, and should just be ignored.
Here is a test that, although not perfect, was at least semi-blind. The conclusions: at that bitrate, MPC ('MPegPlus' not 'MP3Pro') and AAC were the best, followed by LAME MP3, OGG & WMA8 all together, and finally the very worst was XING encoded MP3.
This test was run with Vorbis RC2. RC3 will be out in a week, with much improved noise masking. For a taste of RC3, you can check out the Vorbis CVS, which includes most of the RC3 fixes but encodes at a fixed rate of 128 kp/ps. This raises the low pass, improves the noise masking, and the stereo channel coupling code. -
Re:MPEGplusYou must be using Windows. Monkey's Audio is a Windows only format... and you should not trust your data to a single-OS file format (yes, I would say the same for Linux-only file formats). Use FLAC instead.
MPC has better licensing than Monkey's Audio: the *decoder* is open source (GPL even), so you will always be able to decode your music. *encoding* is only possible on Windows however (although there is an older binary version available for i386 Linux systems), and the encoder will be made shareware in the near future. This is a real pity, because tests have shown that even at 128kp/s MPC is up there with AAC (MPEG-4 audio).
-
Re:How to do listening testsIf you don't have a spare friend, you can use the ABX testing method to see if you can distinguish between two files. Take a file, compress it, save to a WAV, and then give the files to the ABX program, which acts as your spare friend
:)If you're running Windows, you can get ABX from http://www.pcabx.com/. On UNIX systems, the LAME source code comes with an ABX program (in the misc/ directory, I think).
Here is an example of a test that took place using a slightly different testing methodology, more akin to MUSHRA (which is used to evaluate lots of encoders at the same time): http://www.ff123.net/128tests.html.
-
MP3 at 128 is fine if you use good encoder
even though i also use 192kbps on my mixed song, i found that 128kbps on mp3 is pretty good enough for everyday listening.
check out http://ff123.net/cbr128.html