Slashdot Mirror
2nd Multi-Format 128kbps Public Listening Test
technology is sexy writes "Roberto Amorim has launched his latest public listening test evaluating the performance of different audio codecs at 128kbps, among them Apple's AAC implementation (used in iTunes), LAME, Ogg Vorbis fork auTuV, WMA, Musepack and even Sony's Atrac3 format, which is soon to be used in their own music store. Read more on Hydrogenaudio and check out the results of prior tests. As opposed to most evaluations of audio codecs, this is a scientific test adhering to ITU-R BS.1116-1 as much as possible while still allowing everybody to participate."
Never heard of it.
Ogg, ogg ogg. Ogg oggity ogg ogg!
Now that that's out of the way, let the insightful comments begin.
Do we trust participants who say they are running "good gear"? How will they discount the votes of those with "codec agendas" so to speak?
I know you can do frequency analysis on the output of these various codecs. Just compare that to the average human auditory capacity and you can get an objective measurement of the merits of these various compression methods.
So uh, why is this necessary, exactly?
128kbps doesn't cut it. It's an absolute lossy, disgusting bitrate, no matter what it's in. They should test similar file sizes instead of by bitrate, to determine whether something is good or not- this gives a better impression of quality vs size, instead of a purely comparison based test.
What about bitrates people actually use? I know every codec should be transparent for casual listeners if the bitrate is high enough, but I'd like to know what people with good equipment and golden ears think.
Great, now all the ____ fanboys are going to forge results to make their codec look good. Talk about useless tests.
Not possible. All you will get is a bunch of WAV-files, you have no way to tell which file belong to which codec.
That said, I don't care which codec wins the test because Vorbis is still the only one free from patents and the margins are so incredibly small.
Vorbis will win for me even in the unlikely scenario that it comes out last.
My other account has a 3-digit UID.
How do you bas a listening test on the web? People with crappy speakers are going to say that all of them sound bad yet the people that have the better speakers are going to have the better responses. This should be something that is done in a controled environment so that the hardware that is playing back the audio is standard.
Yes... certainly this kind of listening test is important to access the capabilities of each codec.
But in the real world other factors may be more important to chose a coded, like for example general acceptance, freely available code and specs, and a large content base available.
You see: performance will increase allways in all codecs with time... so this kind of testing is only a minute factor amongst others.
You cannot proceed from the informal to formal by formal means
http://pessoal.onda.com.br/rjamorim/abc-hr_bin.zip
http://audio.ciara.us/test/abc-hr_bin.zip
is .wav
...
wav files just don't last as long as mp3s
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
But most people encode at 128Kbps, and most stores sell them encoded at 128Kbps, it's basically been decided by public opinion that it's an acceptable size for a song.
Unless you can get a 64Kbps recording to have the same quality as a 128Kbps recording, there's no market justifiable reason to really care.
Of course, if that turns out to be inferior to any of the other formats, it would prove that something's wrong with the tests.
Why does anyone still use 128kbps? I hate it when I download music (legal ;) and the only bitrate available for the song i want is 128. With 200GB+ hard disks being so affordable these days and everyone having high speed, I think everyone should encode their (mp3||ogg||aac) at 192 or 256.
MOD PARENT DOWN!
BTW, I think the difference between MP3 and Vorbis at 128 kb/s is perfectly noticeable. MP3 sounds rather bad, vorbis sounds pretty good. And the point is precisely to tell which format sounds best, so you don't want to do 512 kb/s bitrate where all formats sound close to CD quality.
A moron with too much on his hand..
There used to be a great site called r3mix.net, which, IIRC, did some spectral analysis on some of the assorted compression algorithms (trying various different options for them). It was focused on the LAME mp3 encoder, but also looked at a few others.
They also had some great forums for info on music ripping/preferred encoding methods/CD burning/etc.
Now, that URL goes to some lame "sponsored mp3 links" site.
Anyone know why r3mix.net died and if there's any new site that makes a good replacement?
Wow, some people must really have nothing do do. (And yes, I am new here. Don't start with that.)
I've started mousing over all these links before I click on them - you never know when somebody will pull something like this.
Karma: Segmentation fault (tried to dereference a null post)
here
"If you are going through hell, keep going." - Winston Churchill
When you listen to compressed audio over inexpensive speakers / headphones, you can't hear the difference. With my Sony Studio Monitor headphones, I lost the difference at about 250k with mp3, so I started using 320K as that was the best at the time. Then I bought $2000 Martin Logan Mosaic Speakers, and the original CD was clearly better than even the 320K bitrate. So now I only do lossless compression. That's fine at home, but in any other environment, there's usually so much noise and distractions that even if you had excellent headphones or speakers, you wouldn't appreciate that little difference lossless brings over 256K or even 128K.
DON'T CLICK THE LINK!
The sad thing is that somebody went to the trouble of putting together a perfectly reasonable, logical post just to throw in a porn link. *sigh*
Karma: Segmentation fault (tried to dereference a null post)
Just because you don't have a use for 64k audio, doesn't mean the results are meaningless. Lots of people have small-capacity players, and some codecs can tolerate that bitrate for very casual listening (such as in the car). Lots of streaming audio sources are at this bitrate or lower. Satellite radio is at 64k or lower. Also, it's not a good idea to try to extend these results to other bitrates. MPC for example, isn't even worth considering at 64kbps, but at bitrates over about 140kbps, it will beat the pants off of anything else.
Of course, proving the patent-freeness of Vorbis requires searching every single patent with a fine-toothed comb, further indicating how messed-up the whole patent system is at this point.
I just have to wonder how many companies are waiting to pounce on the first major commercial user of Vorbis with a patent suit. (Yes, I know there are commercial users of Vorbis, but none are really big enough to attract patent litigation, especially since none of them are wedded enough to vorbis that they wouldn't be able to just drop it for mp3 support with its well-known and, IMO fairly-reasonable, license fees.)
There are 10 kinds of people: ones who understand ternary, ones who don't, and ones who think this joke is about binary
I'd read the thread when they were discussing which version of Apple's ACC codec to use for the test, and concluded based on a few samples that the new version was subpar.
If they'd included both versions of iTunes/QuickTime in this test, perhaps they could have helped shame Apple into fixing what they broke.
Could you supply the link so I don't click on it?
Check the link, its a redirect:m ?site=http://www.peoplesprimary.com/?n=lutz
http://pediatrics.about.com/gi/dynamic/offsite.ht
You dumbass...
The chosen Vorbis fork is aoTuV, not auTuV.
Check the link before you click, it is a redirect.
However, there is a difference in file size. Apparently, it takes ~200kbps (I may be off) for Ogg to get a better quality-to-size ratio compared to mp3 (Lame encoding, I believe).
You can lead a horse to water, but you can't make it dissolve.
That is not lamer-proof.
One could just send in forms with the same ratings to manipulate the test arbitrary.
If you mod this up, your slashdot background will turn into a beautiful sunset!
The best replacement for r3mix.net in my opinion is HydrogenAudio . The forums are frequented by a lot of professionals, as well as developers of LAME, FLAC, Nero AAC, Musepack, Wavpack, and other codecs.
There may be some merit in attempting to quantify codec properties. Don't think these tests will provide any useful answers. Whatever value the results have will be squandered by the incessant blather of countless audiophiles. They will prattle on with their subjective claims and anecdotes until any credible results fade into the noise. Left with nothing credible on which to base decisions, those who use codecs to publish sound will consider cost/profit and little else.
The purpose of being an audiophile is to discover/invent/imagine minute differences in noise. There is no "best" speaker, codec, bit rate, circuit or recording method. Live performances are flawed because the auditorium will not have been built 2 days prior based on the latest theories of sound propagation and is, therefore, obsolete. No combination of equipment or technology will ever satisfy even the most tolerant audiophile.
I clearly remember the awe we once had for the phenomenal quality of 128Kbps CD quality recordings. It was called "near perfect" and "almost live." Today, 128Kbps is considered laughable crap suitable only for fools that don't know better.
The whole thing is a pseudoscience wannabe filled to the brim with vendors lauding useless gear before gullible fools. I will not participate.
Maw! Fire up the karma burner!
Not possible. All you will get is a bunch of WAV-files, you have no way to tell which file belong to which codec.
.ogg vorbis, an mp4 and 3 flacs. If you want to be biased either for or against mp3/oggvorbis/quicktime itunes AAC, you can.
Check the contents of the sampleXX.zip files; you actually get an mp3, an
SCO employee? Check out the bounty
You are 100% clueless, pardon my french.
.wav file is about 1.5Mbps.
The bit rate of
You are clueless as well. The wav file rate of ~1.5Mbps, more specifically 1411 or 1408 or something like that is only for stereo sound sampled at 44.1khz. Change the sampling rate or change it to mono and your wav file bitrate changes...
Yes, but had you actually taken the test you would have seen that you have no clue which one you're listening to at the time of taking the test. I would wager that 99.9% of the people that take the test will find no perceptible difference in quality. I consider myself to have a fairly good ear and I was rather shocked at how little I was able to identify identical samples in the ABX test by doing more than just guessing.
I am a leaf on the wind. Watch how I soar.
Sure, we'd never want what's subjectively best but should accept what's generally available. I opt that you listen to music through the telephone for the rest of your life.
I'll set up the juke box in the sky you seem to crave. I'll rig a little server up that will answer the phone with voice recognition. Any song you ask for will be searched for, downloaded and played to you for nothing more than the cost of your long distance bill. Because you don't care about quality but only want quantity, I'll use all the codecs even WMF to insure you get multiple crappy coppies of what you want. You can listen anywhere with your cell phone and the RIAA won't have to worry about piracy because no one else in the world would ever want to listen to such junk. I promise the service will always improve, honestly I do, so trust me.
Friends don't help friends install M$ junk.
Vorbis does variable bit rate and you set the quality you want. That way you don't waste lots of bits where they are not needed. My 4MB ogg file sounds as good or better than my little brother's 6MB mp3. The difference is more songs on my 256MB compact flash card. Yes, it's easy to play that music on my Zaurus, which cost about as much or less than DRM gimped portable music players.
I hate it when I download music (legal ;) and the only bitrate available for the song i want is 128.
Cry me a river.
Friends don't help friends install M$ junk.
o/~ Ring, ring, ring, ... bananaphone! o/~
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
And yet, there were times when it was not only acceptable, it was the hot new thing, which much better quality than AM.
The Raven
The r3mix tuning (--r3mix), while a small step forward, was inherently flawed because of his insistance on tuning based on pictures instead of acual listening tests. As a result, the --dm-presets were invented and improved by Dibrom (the HydrogenAudio founder) along with a multitude of testers. eventually those were included in LAME as the --alt-presets (and in the latest version they just replace the normal --presets). In short, Hydrogen Audio is THE place to go for this stuff now.
Jeremy
After a while, once you have weeded out bad ways, one is going to reach the following situation. Each algorithm will perform very well for a large set of music and poorly for some small set of music. Barring pathologies, The poor set will be assymtotically fixable by increacing the bit rate. By the way this is not just my opinion. Theres theorems that say this is true of any compression scheme when applied to all problems.
what does this mean? it means that the end user is never going to work at the truly low end of the bit rate specrrum because they want something that virtually always works. Plus they want a wee bit more just in case they have to transcode it. So if the recommended rate is 128 people will encode at 160.
So these comparisons need to be done not at the bitter edge where music flaws are easy to spot because NO ONE WILL ACTUALLY MAKE THAT THE OPERATING POINT THEY USE. That is to say everyone knows vorbis sounds so-so at 64KB while MP3 sound much worse. But no one wants So-So they want darn good. So they are going to recors their Mp3 at 160 and at 160 Ogg and Mp3 sound so close that the size of the test you'd have to do to pick up the difference is silly.
the proper way to do this is the following. Pick the gold standard format, say MP3 and its standard excellent operating point, say 160. now test all the others at lower bit rates than 160, and see which one has the lowest bit rate that scores as good as the Mp3 at 160.
comparing all methods at a constant bit rate, esepciall a low one, is stupid
Some drink at the fountain of knowledge. Others just gargle.
Why would anyone bother to encode at that rate? Not for storage purposes I hope.
Storage space is cheap, so why not just encode at 192-320kbps and get the good quality audio for listening.
Most of the time when you are listening at home by the computer, which probably is hooked to your $5000 high-end stereo set you can enjoy the crystal clear sound of the high-bitrate files.
For the road where storage space is limited, you won't be able to carry those 7.1 speaker systems with you naturally. You can easily re-encode those files with lower bitrate so it'll take less space.
I use simple perl script I found with google and with little tuning it now creates perfect 31Mb or 63Mb copy of the album I choose from my music library to fit in my portable players internal memory or the smartmedia card.
Perfect for home, perfect for the road.
Works for me atleast.
There are no atheists when recovering from tape backup.
Are these the same professionals who claim that one form of digital connection is superior to another?
When working at Sony I discovered that Audio professionals were still caught in the analog days, and would for instance insist on say, fiber optic, over another purely digital data link between connections claiming something was lost in the sound and they could "hear" it.
Of course that's ridiculous, once converted to digital either all the ones and zero's get from one piece of equiptment to the next or not.
Using visual graphing and statistics makes a HELL of alot more sense than having people listen, people imagine what they here, they don't imagine digital analysis. The Audiophile who argues with the computer is wrong everytime.
heh, i think the guy was writing a valid post and just had a porn site in his clipboard.. not sure why he'd go AC then though..
I did a listening test for the danish Mac news source macnyt.dk a week ago.
My conclusion is that for (iTunes encoded) MP3 and AAC, you dont need anything above 128 kbps. Casual users may even make do with 96 kbps AAC.
I simply couldn't tell the difference between 128 and 160 in any of the formats.
With apologies to ogg fans =)
And, "You atonal ass, you're not immune, write a song with a fucking tune!"
ask these guys. ?
mpc is opensource and is presumably patentfree too(the old patents covering its technology has expired). MPC also won the last 128kbs test (or was one of the winners)
128kbps stereo = 2 64kbps mono channels
Why was Vorbis forked?
And more importantly, why didn't they take advantage of the chance to give it a better name than Vorbis? "aoTuV"? WTF?
Actually, no, these aren't those people.
These are people who do double-blind testing and who recognize that other so-called audiophiles are being silly when they buy ridiculously expensive power cables.
(On an unrelated side note, if any HA regulars are reading this, it was pretty much my fault that the previous test wasn't attributed to Roberto. I apologized to him as soon as I realized my error, but I'll apologize here once more just to be sure.)
Here's what i have them: PeoplesPrimary.com
Buncha stupid cunts..
You may or may not know, but that site is being hosted on one of those (originally) free web hosting packages offered by 1and1.net. The server itself is actually a user-mode linux box hosting a ton of other sites using virtual hosts. If you attack that box, you will be breaking into a node in 1and1's web cluster. And undoing the damage is as simple as restarting the image (web data is hosted on a SAN)
They'll probably send the FBI after you. They're not idiots.
Let's not abuse what is otherwise a nice service (I have an account, it's great)
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Come on, why not?
Sigh, that's what I get for showing up late, all the other directions were taken.
it's probably a copy/paste from another post, with the link changed. that's not a porn site anyone would be willingly looking at (except for the person writing the script that handles it).
It's subjectively better sound. The idea is that you want it to sound, to a human, as close to the source as possible. Now I said sound close, not be close. If the resulting wave looks nothing like the source, it doesn't matter so long as it sounds like the source. These are end compression schemes, intended to be done only on the final product. If you need to do more processing, you should pick another format that is built for objective accuracy (like a lossless format).
Objective tests of lossy compression is interesting, and can help in optimising, but it ultimately doesn't matter. Listeners don't care that something is or is not the same as the source, they care that it sounds the same, and does so in a small amount of space.
As someone that works with tons of analog gear, I know that the second you have to dump your 24 tracks of 2" tape to that 16 or 20 bit DAT deck, that's when the tears come rolling down. (Ask Steve Albini!) THAT is when you start to hear "artifacts" the most. If you do some tests with these files against the original CD's, you would definitely hear a lot of the same artifacts... the compression just makes it a little more noticeable. Bring back the cassette!
--Nick
http://cassettefetish.com
dear roberto,
i'm quite impressed by the tool you created! when i saw the slashdot post, i thought that's gonna be lots of fun.
i tried it out immediately, figured out what codecs, thought - hmm creating ogg files with the default ogg-lib instead of the optimized one you mentioned in the readme - that's a pitty... is that same preconditions for all...? well, in fact, different preconditions are okay, since the sound reality is also different everywhere. having started a bit with bartok, debussy and chan chan remix, which i all found quite annoying for the middle of the night, i soon realized that it's not easy to hear a difference between the original and encoded files. thinking how to increase the quality, i remember that i use oss on my new debian box since my soundcard is broken in alsa in unstable. shit, where do i get an old package without building from source... or is even jack worse a try for a listening test, or doesn't that affect the playback...?
listening tests are really fun (and interesting by the way), but it would be much cooler to hear good, chilly music while doing that. therefore i propose to create an enduring test out of it! my idea is to create a database which is permanently fed with inputs from users, sometimes new samples, mostly listening reports. a web application collects this inputs, creates the ecf etc. files out of the uploaded flac-file and creates current results in a cron job. your application could be bound in a java-applet (though, do we have permissions from there to write or read on the hard drive...?), or even written as a mozilla application (say an extension...) using xul & co. that would be really cool. the web service on server side could be done in zope, perfectly.
hm... it would be really cool to do a listening test with sigur ros, radiohead idioteque, tryo or pink floyd - with music i really like to spend my time with!
what do you think?
greetings from sweden,
nico
p.s. is copyright a problem if we always only take a part (half / 10 % / 30 s) of the song? if so, we can always ask the band (or the music company...) - that's also sort of promoting for them. they could even submit us the album cover, so we can display it in the mozilla application... let's try to change the existing music business structure - promoting is done via listening tests today...
On low end speakers and headphones, which many people have, 128k from a new format like OGG or WMA is more or less CD quality. They honestly can't hear the difference between the compressed and uncompressed file. So, why not save some space and just go with it? That and it has become the "gold standard". It's what people have been told they should use, so they do.
Thanks to schnofler, any person using an Operational System with Java can participate.
:)
'Operational System'... that's cute.
People are constantly comparing audio coding standards, but realize that most of the stuff you hear is marketing speak. Many companies have lots of IP in this area and they obviously want to make their solution the standard.
:) Those books can provide a good basis of how all the coders tested works.
What makes one codec sound different than others is the psychoacoustic measures implemented, quantization method, and the windowing scheme implemented before MDCT is performed. Note that all of the coders tested there do not use the same windowing method, but all of them use MDCT in a way.
MP3 is a subband coding, it slices the audio into sub bands before transforming them. AAC, OTOH, is not. AAC uses straight MDCT and does the filtering there. The criteria for filtering is still the same old, tho. That is part of the reason why AAC at 128 kbps way outperform MP3.
Psychoacoustic is not new, it's been described extensively in a book "Psychoacoustic" by Fastl. The catch is, audio coders have to take into account the complexity of performing the full model. MP3 uses a very simplified version of it, and it taxes the highest spec of its day. That is also the reason why AAC-LC (low complexity) is more popular than AAC-Main profile nowadays.
Vorbis can sound better because with new hardware, a more mathematical heavy version of psychoacoustic can be implemented today. Plus, they discard the notion of constant bitrate and use quantization quality instead. This is also evident in FAAC.
128 kbps stereo is practically the limit of almost-transparent quality audio now. 64 kbps mp3pro is just bull, it doesn't perform anywhere close to modern mp3 at 128 kbps. There is a limit on compression, and that is governed by the entropy (information content) of a signal. You go lower than entropy, you lose information, simple as that. Having said that, the only way to reduce entropy is using psychoacoustic models, and that also have a limit.
Note also that Dolby-AC3 that is used in DVD and movie theatre compresses 5.1 channels into 384 kbps, or roughly 150-ish kbps stereo. Again, the same lower limit is evident. They do compression by combining the high frequencies > 15 khz and ignore the phase information in that high frequencies. As you can probably tell, AC-3 sounds pretty good.
If you're interested in this area, I suggest the MPEG-4 book by Ebrahimi, Psychoacoustic by Fastl, Multimedia Compression by Gibson and DSP First by I forgot who
I downloaded all the files for these tests, and to be honest, on my admittably very average set of computer speakers, I could hear no discernible difference between ANY of the codecs and the source.
I find this very strange, since I *know* that I have heard significant differences between 128kbps and source with downloaded mp3's. This begs the question; how much of mp3's general crappiness is due not to the codec itself, but to the incompetence of the person making the original encoding?
Or maybe I'm just going deaf. It still bothers me, though, that I honestly heard no difference in the samples, and I've worked as a sound technician before!
My personal favorite encoder is Xing's TOMPG, v3.0 circa 1998... it's fast as blazes, does a pretty damn good job, supports bitrates up to 160kbps... although nobody seems to remember that it even exists.
Not true. ABC/HR encrypts the results and only encrypted results are accepted by Roberto.
On the best speakers you will almost always hear the differences. Naturally the system has to be up to the task as well but the difference between 128 and original is as wide as hearing a real instrument vs. a recording of the same instrument. It's true, on most speakers you will never hear the difference but that doesn't mean it's not there. And it doesn't take a "golden ear" to hear the difference. You can tell as soon as the music starts that something isn't right. Headphones are even worse because they lack any sense of realistic scale. Most of what humans are good at hearing is "action" which is more the feeling of the nuances of the music, not whether all the frequencies are completely correct. Codecs usually get the frequencies right, they just completely kill the "touch" or "action" of the music. For a nice description of these nuances (short of listening to a great stereo yourself), this is a good review:
"action" defined
Not having the equipment or the desire to hear details doesn't mean there isn't a difference. The point of doing the 128kbps test is to systematically document the advancing quality of low bit rate audio codecs. Nothing will ever convince all the audiophiles 128 is the way to go because most of them don't need compression. You should be thankful that they push the market to improve low quality codecs so that the rest of the world gets to hear better music. Remember a few years ago when 128kbps MP3 had splashy effects in it? If the few gave in to the majority nothing would ever improve.
I'm not sure how the averaging works in the final results but if it's weighted and you have 100,000 people with bad speakers and 10 with good speakers any meaningful difference in the results will be negated. Most likely as time goes on the larger sampling population will bring results that are all very close to 5 regardless of codec.
It's rigged.
Not THE place, but a place. And there aren't many. Not that HA is bad, but it's not that good, either. It's the same people talking about the same stuff, over and over and over again. Wait, that describes /. (too).
.
We don't need no stinkin' opinions around here so keep it to yourself, flamebaiter.
From visiting their web page, it doesn't look much like it is a serious project. It's hosted on geocities.jp for crying out loud! The only information on their page is this extremely descriptive bit: "Tuning was improved in all bit rate regions."
----
All of whose base are belong to the what-now?
hmmm. I'm surprised no one mentioned this. You migh want to head on over to ff123's artifact training page and listen to the examples there if you aren't familiar with the kinds of artifacts that can occur with lossy compression of audio.
> Great, now all the ____ fanboys are going to forge results to make their codec look good.
> Talk about useless tests.
Results can hardly be forged. Both the config files and the result files are encrypted. Even though it's not unbreakable (Blowfish + ElGamal, but there are theoretical circumventions), it's enough to keep the vast majority of participants from forging results.
Needless to say, I will only accept encrypted results. Plain text results would go straight to the thrash bin.
have you actually tried THESE particular samples? now i don't usually listen to most of the music that is covered by the samples (metal, electronica usually requires higher bitrates) but these are fookin' hard to "ABX"
According to the last 128 kbps listening test (http://www.rjamorim.com/test/128extension/results .html), Ogg Vorbis beat Lame convincingly (ie. about the same file size, better quality).
I'm surprised that people still make claims like '128 kbit will always sound like crap' etc. while those too-broad claims apply only to mp3.
Superior encodings like ogg, wma, aac will encode at 64 kbit/s with the same results as 128 kbit/s mp3, FM radio quality. It's not cd quality and nobody will expect that.
At 128 kbit/s they will encode to almost-cd quality and you'd have to be an audiophile (if you're giving a real opinion based on facts) or a slashdotter (just making claims out of thin air to boast or to support your hidden agenda favourite 'cool' product) to claim otherwise.
Why are they using iTunes 4.2 as the encoder? Apple made improvements to their AAC encoder in the 4.5 release a few weeks ago.
mbbac
Essentially, you're comparing different formats using multiple types of samples.
If I have sample A and sample B and encode them both in, say, AAC and MP3, then I have something more useful, even with crappy speakers.
If someone says they can't tell the difference, then great.
If someone says that they can tell a difference, then I'm able to see whether they are *consistent* about it or not. If they say AAC is better in one sample but MP3 is better in another sample or if they can't tell the difference reliably between WAV and any compressed sample, then this will show that.
The testing is blind. So it might play the AAC twice and you'd never know. Unless your results are internally consistent, they're kinda useless. So you can use this sort of thing to eliminate bad data from the testing.
But someone who can consistently and accuracy tell the difference between the sounds, well, they obviously have a good setup or a good ear or something on their side. If they consistently rate the AAC compressed version as better than the MP3 compressed one, for example, then they likely can hear a difference. That sort of thing.
With enough testing, you can weed out the crap and get the real differences.
- Give a man a fire and he's warm for a day, but set him on fire and he's warm for the rest of his life.
Are these the same professionals who claim that one form of digital connection is superior to another?
:)
No, because those people are idiots.
I read HydrogenAudio a lot. It's good stuff, which often goes over my head in some of the forums... In any case, these are not "audiophiles", these are people who actually know WTF they are talking about and tend to flame those who only think they know what they're talking about.
Using visual graphing and statistics makes a HELL of alot more sense than having people listen, people imagine what they here, they don't imagine digital analysis. The Audiophile who argues with the computer is wrong everytime.
Well.. yes, and no. A digital analysis will help someone see that their algorithims are doing what they are supposed to be doing. But you still cannot replace or discount the human side of things.
See, it's easy to analyse your algorithims and tweak them by using digital analysis of the results. But the listening tests show whether or not your algorithims are the correct ones to be using in the first place. If you use a new type of filter, and it looks good digitally, but introduces a weird high frequency harmonic, then it'll sound like crap. But a harmonic won't show up on those digital pictures of your audio, because it's not in the frequency range you're analysing to begin with, it's a combination of several frequencies that you're hearing. The listening tests are crucial to determine whether any given algorithim is wise to use or not, as well as to determine comparitive results between multiple algorithims/formats.
- Give a man a fire and he's warm for a day, but set him on fire and he's warm for the rest of his life.
I don't think that consistency plays any factor in the test. Again, I don't know how the results are tabulated, but certain scenarios could totally skew the results. For example, if everyone chose the original as sounding worse than the codec because they thought they heard a difference on their crappy speakers, the results for that codec would be really screwed up. Even if you had a few people who always could tell the difference, their vote would be lost in the sea of other votes. Even worse, if everyone in the test was using crappy speakers, there would appear to be little absolute difference between the codecs. Likely they'd all be close to 5. This gives you no insight into the absolute difference between the codecs, even if they were all less than the original.
why do they have us decode the files after we get them into wav?
.* to .wav for the java program to play.
.wav file outputs of the formats and send those out. it would be bigger, but probably easier for some/many people.
couldn't they just create them as a bunch of wav files so i don't have to install all these codecs i never use.
never heard of mpc, mp4, m4a, ape
i'm sure they are all fine codecs, but i'm on debian, and i don't seem to have packages available to install those from, and i don't feel like installing 3 unknown software packages from strange places on my machine just to convert them from
just zip up the
I took it and it was certainly possible that more than one of the 6 test files were duplicates, otherwise you wouldn't be testing every codec (since there are 6). Also, you only submit the results once. Only in the ABX "practice" mode can you do the same sample (but randomly reversed) over and over. So it seems to me that there's a lot of room for error.
IIRC, Xiph.Org got some people to do a limited patent search and I believe AOL performed their own patent search as well. The doubt only exists because Xiph.Org has not publicly released the results of their patent search (due to lawyers' advice I think).
did apple update AAC recently or am i just loosing my mind? If they did, is iTunes 4.2 the right version to be using?
I disagree in saying that 128kbps is a hopeless bitrate no matter what, didn't you ever learn to never say never?
What all the people developing codecs over the years have been doing is create better algorithms to squeeze more quality into those 128kbps - sure, 128kbps sounds very badly in the typical, ancient mp3 compression, but that doesn't mean it sounds bad in all other codecs.
Besides, 'testing samples that are all 128kbps' and 'finding the best sounding samples first and then seeing what bitrate it was' are two sides of the same coin - you're just wasting time and work in coding everything in so many different bitrates and then trying out the quality of each.
Though it would of course be interesting to see how the codecs would fare in 64kbps or 80 persay.
I wouldn't bother with the higher bitrates, because as we go into them they all begin to sound about the same, and approach the sizes that the lossless codecs are getting to in their highest compression settings - besides, I think it's relatively safe to say that those considering lossy formats at all are trying the most to save space (and thus venturing towards the lower spectrum of bitrates).