After 4 Years, HydrogenAudio Opens New 128kbps Listening Test

kwanbis writes "After more than four years, a new MP3@128kbps listening test is finally open at HydrogenAudio.org! The featured encoders are: LAME 3.97, LAME 3.98.2, iTunes 8.0.1.11, Fraunhofer IIS mp3surround CL v1.5, and Helix v5.1 2005.08.09. The low anchor is l3enc 0.99a. The purpose of this test is to find out which popular MP3 VBR encoder outputs the best quality on bitrates around 128 kbps. All encoders experienced major or minor updates that should improve audio quality or encoding speed, and we have a totally new encoder on board. Note that you do not have to test all samples — it is a great help even if you test one or two. The test is scheduled to end on November 22nd, 2008."

Comments

use the cans, luke

[PhrostyMcByte]

good headphones are a must for such close listening tests. you'll only be able to hear really major differences with most speakers.

Re:use the cans, luke

[maeka]

good headphones are a must for such close listening tests. you'll only be able to hear really major differences with most speakers.

Good headphones are nice in so far as they block ambient noise and allow you to hear any artifacts easier, but since MP3 is a perceptual encoder it is actually more likely that artifacts are audible on "defective" hardware.
If a cheap speaker or cheap headphone's frequency response is bad enough to mess with the model's idea of masking, for example, poor quality reproduction can actually make the 'tricks' of MP3 apparent.

Re:use the cans, luke

[TheRealMindChild]

I call horseshit. I only care about the differences *I* can hear with the speakers/headphones *I* have. Isn't that the whole point? The shortcuts I can take without noticing a difference...

Re:use the cans, luke

[Jah-Wren Ryel]

I call horseshit. I only care about the differences *I* can hear with the speakers/headphones *I* have. Isn't that the whole point? The shortcuts I can take without noticing a difference...

It is a heck of a lot easier to upgrade your equipment than it is to re-encode your audio, assuming you even have the original sources around.
What sounds fine today on your current system may sound poor on your next system tomorrow.

Re:use the cans, luke

[Anonymous Coward]

And I'd be even more surprised to see a moron who paid $200,000 for fuckin' speakers to admit otherwise.

Re:use the cans, luke

[rtollert]

A lot of very high quality encoder tuning has been done with $30 headphones on laptops. Concentration and patience is more important than equipment here.

Re:use the cans, luke

[setagllib]

Then imagine how good $200,000 headphones would be. They'd include an extra large driver to place on your torso for deep bass.

Re:use the cans, luke

[arth1]

Good headphones are nice in so far as they block ambient noise and allow you to hear any artifacts easier, but since MP3 is a perceptual encoder it is actually more likely that artifacts are audible on "defective" hardware.

Good headphones seldom block ambient noise. Some of the very best headphones out there are open, including (but not limited to):
AKG 701/702
Grado (all models)
Beyer DT880
Sennheiser HD600/650

They're made, not to isolate you from the environment and prevent sound from escaping either way, but to replace loudspeakers with the best sound possible, with no regard to (or for) the environment.

My beef with this test is that they use 128 kbps VBR.
These days, space isn't as such a premium as it was a few years ago, and few use 128 kbps anymore, unless it's the default for their encoder or they haven't bothered changing.
I would think that 160, 192, 224, 256 and even 320 kbps are more common than 128 kbps these days.

Then there's VBR. And VBR in itself is by many considered evil -- yes, you cram in more data that way, but you end up with a sound stream that switches back and forth between different qualities, which is more apparent to the ear than if it was all at the lowest quality. It's like listening to a radio where the FM stereo kicks in every now and then. Yes, that is quantifiably a better quality than listening to it in mono, but I still prefer switching to mono to get a worse, but stably worse, sound.
The same piano key hit multiple times can end up sounding different with VBR. First you get an awesome 224 or 320 kbps note, then another, but then omgwehaveusedupallthebandwidth you get a 80 kbps note that just doesn't sound similar. Overall, the quality has gone up, but the net effect is that it sounds jarring.

Personally, I have enough disk space, and use FLAC when I can. When I can't, I use 224 kbps CBR, because at that high bitrates, I can't really tell any difference, and I avoid the whole VBR bitrate-changing problem.

Re:use the cans, luke

[Cowclops]

Actually, the point of VBR is to keep quality close to constant, as some audio frames are more easily compressed than others. Constant bitrate actually gives you variable quality. Variable bitrate gives you near constant quality. If you "hear" the quality changing in a VBR recording, theres something wrong with the encoder.

Re:use the cans, luke

[Anonymous Coward]

The same piano key hit multiple times can end up sounding different with VBR. First you get an awesome 224 or 320 kbps note, then another, but then omgwehaveusedupallthebandwidth you get a 80 kbps note that just doesn't sound similar.

ahh, audiophiles. they'll make up anything to try and sound like they hear better than the rest of us, even when what they've described it entirely NOT how VBR works. it's nothing to do with available bandwidth, it's about what is required to represent that frame of sound. if encoding silence at 224kbps makes you happy, by all means go on believing that shit sounds better than 320kbps vbr files that will come out smaller in size.

personally i just use apple lossless for everything so i dont have to concern myself with whether or not i can hear a difference. there is NO difference from the source and the extra the space needed is minuscule in respect to drives these days, but I'm not trying to delude myself that I can tell the difference.

Re:use the cans, luke

[afidel]

Uh, PNG DOES look better for certain content, namely screenshots. Due to the way JPEG works it will always blur fonts more than PNG. For pbotographs you will get a better looking shot out of JPEG than PNG for the same files size ~90% of the time, so use the right tool for the job =)

ugh

[Anonymous Coward]

Wow, what a mess. Download this package. Now download fourteen more packages (DownThemAll is the only reason I didn't give up right then). Y'know, I'm kinda interested in this subject, as I have no trouble hearing artifacts in most 128kbps CBR MP3s, but this is just a huge pain in the ass. Wouldn't a simple Flash app have made things so much easier?

Re:ugh

[HateBreeder]

Agreed.
I went into the trouble of trying to run this under Linux.
the supplied batch files didn't work - it was missing files due to bad paths. the java application required a HUGE meddling around, choosing the settings, creating tests... I gave up. I'm not *that* motivated to help.

If you're trying to design a public test, the goal is to make it as simple as possible. An online application is an absolute must here.

I would be surprised if there will be anymore than a few hundred responses to this, all from a very specific demographic, Hardly a representative sample of the general population.

Re:ugh

[syousef]

Wow, what a mess. Download this package. Now download fourteen more packages (DownThemAll is the only reason I didn't give up right then). Y'know, I'm kinda interested in this subject, as I have no trouble hearing artifacts in most 128kbps CBR MP3s, but this is just a huge pain in the ass. Wouldn't a simple Flash app have made things so much easier?

I gave up at step 3751 "Buy Monster cables". ;-)

Re:ugh

[Petrushka]

Fortunately someone on the HydrogenAudio forum was equally annoyed and has posted all the samples in a single zip file (54 MB file).

Re:ugh

[mollymoo]

That hardly solves the problem. The applet should be embedded in the web page and download all the samples automatically, on demand. Why the stupid rigmarole of doing everything yourself? It's a ridiculously complex process. I gave up when I discovered that "OS X users are asked to handle decoding of samples themselves" what does decoding the samples involve? I haven't a fucking clue, because that's all it tells me.

What kind of music is involved

[CRCulver]

While popular music is acceptable at 128kbps with recent encoders, certain niche music genres like spectralist music clearly suffer at low bitrates. With pieces like Per Norgard's Symphony No. 3 or Grisey's Les espaces acoustiques you can easily hear the difference between 256kbps and the original CD-quality on even average headphones or speakers. Any music which depends on a greater portion of the natural overtone series than just the first handful of partials will need higher bitrate encoding.

Re:What kind of music is involved

[maeka]

You are asserting MP3 has faults it does not have.
"Overtones" are not an issue, nor do I think you could point out a 'problem sample' which fails due to the presence (or lack) of "overtones."
Popular music, in fact, is often harder to encode efficiently as it tends to have the dynamics compressed out of it (see loudness war), full of distortion, and therefore be closer to random data.
Temporal smearing is clearly a problem with MP3, and is evident in music such as harpsichord, but that is not the claim you make.
Do you have any ABX tests to back your claim?

Re:What kind of music is involved

[Aphoxema]

I'm not too worried about the quality of my music. Since I mostly listen to noize, industrial and EBM, the occasional scratching, pop, siren, explosion, grinding metal and screaming only accentuate the already apparent awesomeness of what I'm burning holes in my ear drums with.

Re:What kind of music is involved

[rtollert]

That's great if you're trying to use a codec for a purpose it was never designed for and nobody actually uses. Would you choose a JPEG codec based on its ability to encode/decode raw audio? Would you choose a car based on its ability to traverse the English Channel?

Re:What kind of music is involved

[rtollert]

Spectral music might make for great samples for this kind of testing, but your assertion is ultimately unsubstantiated unless you can provide real listening test results that show it makes for a more sensitive test. There are all kinds of subtle things going on that might seem to make for great encoder testing, but largely turn out to make an imperceptible difference. Just because so many overtones exist (99% of which do not exist in msot acoustic music, btw!) doesn't mean they are necessarily audible if they are perturbed. More specifically, I'd anticipate that most FFT-based imaging techniques would hammer encoder lowpasses very hard, but would not be nearly as hard on preecho performance or stereo imaging artifacts. In a lot of situations, the preecho is a lot more important than the lowpass.

Re:What kind of music is involved

[IceCreamGuy]

"Overtones" are not an issue

Incorrect. One of the ways MP3 achieves lower bitrates is by removing overtones of a fundamental frequency when the overtones are reasonably quieter. If, for example, you pluck an "A" string tuned to 440hz, the string would also resonate at 880Hz, 1320Hz, 1760, and so on. An MP3 encoder would remove these overtones if they were significantly quieter than the original 440Hz tone, since research has shown that the human ear doesn't really notice them if the fundamental is much louder. The problem arises, as the parent noted, in some niche music; however anyone should be able to notice this in things like cymbals, where the most basic sound and timbre of the instrument is defined entirely by the overtones it produces. You can hear this as an almost flanger-esque quality to the cymbals in sub-128Kb/s encoded MP3s. Any drummer will tell you that this drives them up a wall, and the way the psychoacoustic model of MP3 compression handles overtones is the culprit.

Outdated?

[Max Romantschuk]

Umm... 128 Kbps? Seriously? And no Ogg Vorbis, AAC etc... If you're bothering to set up a listening test, why limit yourself to 128 Kbps MP3?

Also, this should really be set up as a blind test, you get to listen to two clips, and have to choose which is better. The clips are randomized, of course... I could go on, but I'd just make myself sound even more arrogant. ;)

Re:Outdated?

[m1ss1ontomars2k4]

Hey, I still listen to only 128 Kbps mp3 (and some AAC). It saves space. If I accidentally obtain an mp3 of higher quality, I downgrade it to 128 Kbps.

Re:Outdated?

[darien]

this should really be set up as a blind test, you get to listen to two clips, and have to choose which is better.

I agree entirely. They should also include different bitrates - do many people still use 128kbps? - and versions which aren't compressed at all. Hopefully the results might shut up the audiobores who keep insisting that MP3 isn't good enough for their precious ears.

Re:Outdated?

[A Friendly Troll]

MP3 is plenty good enough, it just requires more bits. Why have 192kbps MP3 when you can save room with an equally good 160kbps Vorbis?

Why have 160 kbps Vorbis when hard drives are growing in capacity and dropping in price?

I used to encode things in 192 kbps, then VBR, and now I want to smack myself over the head for doing so; blank CDs weren't so cheap back then, and I wanted to save a little bit of money. Looking back, it sure as hell wasn't worth it - I have crappy, lossy mp3 encodings of rare albums that I cannot obtain anymore, and a hard drive that could easily hold 2000 albums encoded with FLAC.

Sure, there was a time when storage was a premium, but now it isn't. Save room for WHAT? Five years from now, when you will be able to cheaply have 10 TB of storage space in your computer, are you going to regret having 160 kbps Vorbis instead of FLAC encodings? I know I would be, so now I'm encoding every CD I still have in lossless. If I were interested in HD video (which I'm not), I'd have no intention of re-encoding it to smaller sizes, because I *know* there will be a time when I'd regret it. Of course, YMMV.

Re:Outdated?

[hedwards]

Indeed, after reripping my entire collection for like the 3rd or 4th time because it got corrupted randomly, I switched to just ripping everything to lossless. If I need a copy for multiple MP3 players and such or change my mind about what compression rate or type I want it's a task my computer can handle without me swapping tons of discs.

It's always been easier to encode to a lower quality than to a higher quality. And in a strict sense the latter isn't really even possible.

Re:What's the point?

[Polarina]

Why switch back to WAV when you can have your music played at 192000 Hz and a 48-bit volume scale?

Out of phase

[tepples]

is there anything else I should be aware of as someone who transcodes everything to mono before I copy it to my mp3 player ?

Some songs are recorded with parts out of phase between the stereo channels. This means that the left and right channels, instead of being up/up and down/down, are up/down and down/up, which creates directional effects in stereo (especially on a surround receiver) but cancels itself out in the conversion to mono. For instance, "Happiness in Slavery" on Broken by Nine Inch Nails loses the snare in mono, and the quality of the snare drum in the remix of Coburn's "We Interrupt This Program" used with the NEDM meme drifts back and forth between clap-like and snare-like.