After 4 Years, HydrogenAudio Opens New 128kbps Listening Test

kwanbis writes "After more than four years, a new MP3@128kbps listening test is finally open at HydrogenAudio.org! The featured encoders are: LAME 3.97, LAME 3.98.2, iTunes 8.0.1.11, Fraunhofer IIS mp3surround CL v1.5, and Helix v5.1 2005.08.09. The low anchor is l3enc 0.99a. The purpose of this test is to find out which popular MP3 VBR encoder outputs the best quality on bitrates around 128 kbps. All encoders experienced major or minor updates that should improve audio quality or encoding speed, and we have a totally new encoder on board. Note that you do not have to test all samples — it is a great help even if you test one or two. The test is scheduled to end on November 22nd, 2008."

Comments

use the cans, luke

[PhrostyMcByte]

good headphones are a must for such close listening tests. you'll only be able to hear really major differences with most speakers.

Re:use the cans, luke

[TheRealMindChild]

I call horseshit. I only care about the differences *I* can hear with the speakers/headphones *I* have. Isn't that the whole point? The shortcuts I can take without noticing a difference...

Re:use the cans, luke

[Jah-Wren Ryel]

I call horseshit. I only care about the differences *I* can hear with the speakers/headphones *I* have. Isn't that the whole point? The shortcuts I can take without noticing a difference...

It is a heck of a lot easier to upgrade your equipment than it is to re-encode your audio, assuming you even have the original sources around.
What sounds fine today on your current system may sound poor on your next system tomorrow.

Re:use the cans, luke

[rtollert]

A lot of very high quality encoder tuning has been done with $30 headphones on laptops. Concentration and patience is more important than equipment here.

Re:use the cans, luke

[arth1]

Good headphones are nice in so far as they block ambient noise and allow you to hear any artifacts easier, but since MP3 is a perceptual encoder it is actually more likely that artifacts are audible on "defective" hardware.

Good headphones seldom block ambient noise. Some of the very best headphones out there are open, including (but not limited to):
AKG 701/702
Grado (all models)
Beyer DT880
Sennheiser HD600/650

They're made, not to isolate you from the environment and prevent sound from escaping either way, but to replace loudspeakers with the best sound possible, with no regard to (or for) the environment.

My beef with this test is that they use 128 kbps VBR.
These days, space isn't as such a premium as it was a few years ago, and few use 128 kbps anymore, unless it's the default for their encoder or they haven't bothered changing.
I would think that 160, 192, 224, 256 and even 320 kbps are more common than 128 kbps these days.

Then there's VBR. And VBR in itself is by many considered evil -- yes, you cram in more data that way, but you end up with a sound stream that switches back and forth between different qualities, which is more apparent to the ear than if it was all at the lowest quality. It's like listening to a radio where the FM stereo kicks in every now and then. Yes, that is quantifiably a better quality than listening to it in mono, but I still prefer switching to mono to get a worse, but stably worse, sound.
The same piano key hit multiple times can end up sounding different with VBR. First you get an awesome 224 or 320 kbps note, then another, but then omgwehaveusedupallthebandwidth you get a 80 kbps note that just doesn't sound similar. Overall, the quality has gone up, but the net effect is that it sounds jarring.

Personally, I have enough disk space, and use FLAC when I can. When I can't, I use 224 kbps CBR, because at that high bitrates, I can't really tell any difference, and I avoid the whole VBR bitrate-changing problem.

Re:use the cans, luke

[Cowclops]

Actually, the point of VBR is to keep quality close to constant, as some audio frames are more easily compressed than others. Constant bitrate actually gives you variable quality. Variable bitrate gives you near constant quality. If you "hear" the quality changing in a VBR recording, theres something wrong with the encoder.

ugh

[Anonymous Coward]

Wow, what a mess. Download this package. Now download fourteen more packages (DownThemAll is the only reason I didn't give up right then). Y'know, I'm kinda interested in this subject, as I have no trouble hearing artifacts in most 128kbps CBR MP3s, but this is just a huge pain in the ass. Wouldn't a simple Flash app have made things so much easier?

Re:ugh

[mollymoo]

That hardly solves the problem. The applet should be embedded in the web page and download all the samples automatically, on demand. Why the stupid rigmarole of doing everything yourself? It's a ridiculously complex process. I gave up when I discovered that "OS X users are asked to handle decoding of samples themselves" what does decoding the samples involve? I haven't a fucking clue, because that's all it tells me.

Outdated?

[Max Romantschuk]

Umm... 128 Kbps? Seriously? And no Ogg Vorbis, AAC etc... If you're bothering to set up a listening test, why limit yourself to 128 Kbps MP3?

Also, this should really be set up as a blind test, you get to listen to two clips, and have to choose which is better. The clips are randomized, of course... I could go on, but I'd just make myself sound even more arrogant. ;)

Re:Outdated?

[darien]

this should really be set up as a blind test, you get to listen to two clips, and have to choose which is better.

I agree entirely. They should also include different bitrates - do many people still use 128kbps? - and versions which aren't compressed at all. Hopefully the results might shut up the audiobores who keep insisting that MP3 isn't good enough for their precious ears.

Re:Outdated?

[hedwards]

Indeed, after reripping my entire collection for like the 3rd or 4th time because it got corrupted randomly, I switched to just ripping everything to lossless. If I need a copy for multiple MP3 players and such or change my mind about what compression rate or type I want it's a task my computer can handle without me swapping tons of discs.

It's always been easier to encode to a lower quality than to a higher quality. And in a strict sense the latter isn't really even possible.

Re:What kind of music is involved

[rtollert]

That's great if you're trying to use a codec for a purpose it was never designed for and nobody actually uses. Would you choose a JPEG codec based on its ability to encode/decode raw audio? Would you choose a car based on its ability to traverse the English Channel?

Re:What kind of music is involved

[rtollert]

Spectral music might make for great samples for this kind of testing, but your assertion is ultimately unsubstantiated unless you can provide real listening test results that show it makes for a more sensitive test. There are all kinds of subtle things going on that might seem to make for great encoder testing, but largely turn out to make an imperceptible difference. Just because so many overtones exist (99% of which do not exist in msot acoustic music, btw!) doesn't mean they are necessarily audible if they are perturbed. More specifically, I'd anticipate that most FFT-based imaging techniques would hammer encoder lowpasses very hard, but would not be nearly as hard on preecho performance or stereo imaging artifacts. In a lot of situations, the preecho is a lot more important than the lowpass.