"Apache Killer" Web Server Hole Plugged

No name yet? by grub · 2011-09-01 14:57 · Score: 0

"Apache Killer"

They should call the exploit "Smallpox".
Thank you, I'll be here all week! Try the veal!

--
Trolling is a art,

Re:No name yet? by The+Pirou · 2011-09-01 15:26 · Score: 1

What, 'CVE-2011-3192' isn't exciting enough for you?

As it is effectively finished in the wild for 2.2+ so long as everyone practices due diligence, unless someone really high up the food chain manages to still get bit in a sensitive area it doesn't deserve the same status as William H. Bonney.
Re:No name yet? by Anonymous Coward · 2011-09-01 15:34 · Score: 0

http://li77-30.members.linode.com:6999/
Re:No name yet? by Cito · 2011-09-01 18:42 · Score: 1

bah still waiting for this to show up on debian did an apt-get update and upgrade today and only update for apache was 2.2.16 on the stable :(
Re:No name yet? by insane_coder · 2011-09-01 22:57 · Score: 1

The fix was in Debian for a couple of days now. 2.2.16-6+squeeze2 is the patched version for stable.

--
You can be an insane coder too, read: Insane Coding
Re:No name yet? by Cito · 2011-09-02 07:17 · Score: 1

ah ok thanks I did upgrade to 2.2.16-6 on my debian server. was wondering since the original article mentioned 2.2.20, and when I did the apt-get upgrade it only gave me 2.2.16-6 had me confused :) thanks for info then, and I've upgraded and patched to the 2.2.16-6+squeeze2 now. was hunting for 2.2.20 :P thanks again

How about 2.0 by Tea-Bone+of+Brooklyn · 2011-09-01 15:20 · Score: 1

Isn't this serious enough to warrant a 2.0.65 release that patches this too? There are still some applications that won't support the 2.2 branch.

A bit demanding, no? by Anonymous Coward · 2011-09-01 15:36 · Score: 0

The security advisory did not explain the delay

Not that they must, being free and all. Sure they get contributions, but I'm sure there is a lot on their plate, for a project of this scale and importance, that would warrant a deadline extension, a small one, mind you.

Re:A bit demanding, no? by mug+funky · 2011-09-01 15:40 · Score: 1

the should have made a post on macrumours if they wanted to be taken seriously.
Re:A bit demanding, no? by 93+Escort+Wagon · 2011-09-01 15:52 · Score: 1

Not to mention that any time frame given would've been an estimate, obviously. Knowing exactly how long it was going to take would've required they already knew what needed to be done - and, if that had been the case, I'd think the bug wouldn't have existed in the first place.

--
#DeleteChrome
Re:A bit demanding, no? by LordLimecat · 2011-09-01 16:45 · Score: 1

I was about to say, someone should contact their manager so that they can all be fired for tardiness. Who is their manager, by the way?

Apache fix was out before the exploit was known by Tumbleweed · 2011-09-01 15:41 · Score: 1

The fix is called 'Hiawatha'.

Re:Apache fix was out before the exploit was known by Anonymous Coward · 2011-09-01 18:59 · Score: 0

No, it's called Cherokee and is far more advanced.
Re:Apache fix was out before the exploit was known by Tumbleweed · 2011-09-02 06:39 · Score: 1

I don't know much about Comanchee. In what ways is it superior to Hiawatha? Is it more secure?

Re:It's time to shift our paradigm by Pseudonym+Authority · 2011-09-01 15:44 · Score: 0

Your UID is too close to mine. I demand that you change yours immediately.

Not that it will help Sony at all by robot256 · 2011-09-01 15:59 · Score: 2

Thanks to this post, my server, at least, is fully patched now. How many years will it be before all the production systems in the world have this installed?

Re:Not that it will help Sony at all by Anonymous Coward · 2011-09-01 16:29 · Score: 1

Sony doesn't have to worry about this bug. Everything over their is protected by their start of the art text-based captcha
Re:Not that it will help Sony at all by WrongSizeGlass · 2011-09-01 22:35 · Score: 1

Anyone who wants to put in a little effort to scan/parse the html produced can read the characters of that captcha? It's stunning that this is what Sony considers 'security'.

Re:It's time to shift our paradigm by nitehawk214 · 2011-09-01 16:18 · Score: 0, Offtopic

No, fuck you fagstorm. Wanna fight me? I've kicked the ass of APK, MichealKristopeit, kdawson, and that goatse troll. You can take that chill pill and SHOVE IT STRAIGHT UP YOUR ASS.

Well, it is a suppository...

--
I'm a good cook. I'm a fantastic eater. - Steven Brust

partly same approach as nginx by mzs · 2011-09-01 16:27 · Score: 5, Interesting

http://mail-archives.apache.org/mod_mbox/www-announce/201108.mbox/%3C85111090-501E-4C80-AA8F-DD11B94FDF7C@apache.org%3E

* SECURITY: CVE-2011-3192 (cve.mitre.org)
core: Fix handling of byte-range requests to use less memory, to avoid
denial of service. If the sum of all ranges in a request is larger than
the original file, ignore the ranges and send the complete file.
PR 51714.

I remember reading how people had all sorts of ideas like sorting the ranges, ignoring gaps of less than 80 bytes, noticing that it went afoul of the standard. Around the same time nginx also did a release with the approach of sending back the entire file if the sum of the ranges was more. That was so simple, and it's okay according to RFC 2616 a server MAY ignore the range header, so it's clever too! Glad all the memory handling was fixed-up too though.

Re:It's time to shift our paradigm by FatdogHaiku · 2011-09-01 16:59 · Score: 2

It's worse than you though, in binary!
00110001001101010011100100110001001100000011001000110111
00110001001101010011100100110100001101100011001000110001

Both your UID numbers have 32 zeros and 24 ones...

--
You have the right to remain sentient. If you give up the right to remain sentient, you will be elected to public office

Re:It's time to shift our paradigm by maxwell+demon · 2011-09-01 19:10 · Score: 1

It's worse than you though, in binary!
[first UID in binary]

[second UID in binary]

Both your UID numbers have 32 zeros and 24 ones...

Is there a specific reason you padded them to 56 bits?

What would have made sense to me would have been no padding (no leading zeros), or padding to 64 bits (because that's how the UIDs are probably stored internally). In the first case, it would be 30 zeros, in the second, it would be 40.

BTW, how did you get it through the lame(ness) filter? I get a filter error in just quoting your post ("Filter error: That's an awful long string of letters there.") Only after removing the binary strings, Slashdot allowed me to even preview.

--
The Tao of math: The numbers you can count are not the real numbers.

Forgot to mention by Anonymous Coward · 2011-09-01 19:18 · Score: 0

You forgot to mention that there were several workarounds was available immediately that took less than a minute to implement.

Re:It's time to shift our paradigm by stderr_dk · 2011-09-01 20:49 · Score: 1

It's worse than you though, in binary!
[first UID in binary]

[second UID in binary]

Both your UID numbers have 32 zeros and 24 ones...

Is there a specific reason you padded them to 56 bits?

Or why he was looking at the ASCII representation of the UIDs?

A more logical representation of the numbers would be:

(00000000)000110000101010011111101
(00000000)000110000100011011110011

12 (or 20) zeros and 12 ones.

The ASCII representations of UIDs has a Hamming distance of 6, while the more logical binary representation of the UIDs has a Hamming distance of 5.

What would have made sense to me would have been no padding (no leading zeros), or padding to 64 bits (because that's how the UIDs are probably stored internally).

32 bits would be more than enough for those UIDs.

--
alias sudo="echo make it yourself #" ; # https://pipedot.org/~stderr & http://soylentnews.org/~stderr

Be prepared to back out! by David+Gerard · 2011-09-01 21:09 · Score: 2

I got bitten by this moving our SSL server from 2.2.9 to 2.2.20 - they changed the config processor and our SSL config broke.

Apache claim that a given "stable" series will keep a constant ABI. They seem utterly unable to comprehend that config files count as part of the ABI. Note that binary modules work the same all the way across 2.2.x ... that doesn't help when a "nothing's changed" upgrade breaks stuff.

The changes are typically tightening the rules and disabling technical violations of them. That's a noble aim, but you need to save it for the next version - you can't pull that shit midstream in a "stable" series!

We previously got bitten by Apache's incomprehension on this point when we went from Tomcat 6.0.16 to 6.0.29.

So, before upgrading anything "stable" from the Apache Foundation: Thoroughly test the result, and make sure you can back out at a minute's notice.

--
http://rocknerd.co.uk

Re:Be prepared to back out! by Anonymous Coward · 2011-09-01 21:36 · Score: 0

Apache claim that a given "stable" series will keep a constant ABI. They seem utterly unable to comprehend that config files count as part of the ABI. Note that binary modules work the same all the way across 2.2.x ... that doesn't help when a "nothing's changed" upgrade breaks stuff.
You keep using that word, it doesn't mean what you seem to think it does. The Application inter-Binary Interface (ABI), by definition, does not include non-binary [machine code] components, such as data or config files. If a Media Player was patched and broke its MP3 reader, the "MP3 ABI" isn't broken; it isn't an ABI, it's a parser.
Yes, I understand that changing the config file format is a pain in the ass and that they shouldn't have done that (or, at least, included a tool to convert files from the older to newer format) but don't start throwing around randomly picked words because you think they sound more serious.
Re:Be prepared to back out! by Anonymous Coward · 2011-09-01 23:59 · Score: 0

So, before upgrading anything...: Thoroughly test the result, and make sure you can back out at a minute's notice.
Fixed that for you.
And, why FFS were you still running a 3-year-old-release anyways.
Re:Be prepared to back out! by David+Gerard · 2011-09-02 00:54 · Score: 1

Welcome to enterprise!

--
http://rocknerd.co.uk
Re:Be prepared to back out! by ledow · 2011-09-02 01:05 · Score: 3, Insightful

Er, yeah, you have got to love people who complain about things like this. It's a totally valid complaint, software-wise. But if such changes affected you IN ANY WAY then you're just asking for trouble by having insufficient testing. Especially if you have the magic word "SSL" in your server description - obviously something was important enough for you to encrypt traffic, but not to test adequately.
I'm not saying the OP didn't test - but from the way it sounds, they didn't find out until the upgrade (which means that a) they weren't testing, b) they weren't keeping up with events and c) they didn't bother to wait to see if other people hit problems).
Any upgrade, change, or tweak can completely destroy any system you care to mention and even without that, things can go wrong very quickly. A major UPS manufacturer once was forced to issue a patch because they had an internal certificate embedded inside a Java app used by the UPS software that expired and, when it did, it generally took Windows Servers down with it to the point you could not log in to fix the problem. How long do you think it would take you to track that down on an affected server without having a proper known-good environment or a decent testing regime?
That's not the sort of thing you can catch in everyday testing but is ALSO the sort of thing that can happen to you at any time, for any reason, with any tiny change you ever make. If you're honestly reliant on a server continuing to work you really need to test extremely thoroughly and take WORKING backups at each stage that you deem "tested". Even a hotfix, or a config tweak, or even a reboot which fails to write a byte to disk can destroy your machine's configuration, let alone a human tinkering.
The first thing that any deployment should have is a way to deploy the entire hardware again, seamlessly, quickly, guaranteed and to a known working state. Without something like that, you're wasting your time even trying to keep things up, or diagnose problems. And with it, such "problems" are spotted in seconds after a test upgrade and instantly reverted.
It's amazing how many places have *NO* idea how to redeploy their gear to a known-working state, even with claims of backups being present.
Re:Be prepared to back out! by asdf7890 · 2011-09-02 03:17 · Score: 2

The changes are typically tightening the rules and disabling technical violations of them. That's a noble aim, but you need to save it for the next version - you can't pull that shit midstream in a "stable" series!
If your configuration file contains violations of the rules then your configuration is at fault, not the change that tightens the implementation of those rules. By using a configuration that doesn't match the rules (despite the fact the implementation lets you get away with it) you are relying on undefined behaviour and can not expect any guarantee that the behaviour won't change even in a stable branch. Defined/documented behaviours should not change in a stable branch unless there are exceptional circumstances (a security issue that can not be resolved any other sensible way for instance), but changes to undefined behaviour are fair game.

So, before upgrading anything "stable" from the Apache Foundation: Thoroughly test the result, and make sure you can back out at a minute's notice.
Before upgrading anything from anyone in a production environment you should test the result in a test environment first, unless you have a service level agreement that states the provider does that testing and will recompense you for any admin hassle and/or unplanned downtime resulting from a change breaking stuff (even then, I'd stil be inclined to test the change in a non-production copy of the production environment first, I consider that to be basic due-diligence).

None of our production servers (Linux or Windows) automatically install updates (and more manual updates do not get performed on production until tested elsewhere either, of course). Updates get installed in production once they have been given a least a basic kick-around in test environments first in an effort to make sure they don't break any assumptions made by our components/configuration or 3rd party code/config we are using. While more subtle changes could easily make it through this process (though I can't think of any that have off the top of my head), significant problems (like SSL being completely borken until the config is updated, as per your example) should be caught by the test environment so you don't have to diagnose and fix the problem after the updates have gone to live systems where issues like that could cause embarrasing downtime.
Re:Be prepared to back out! by Anonymous Coward · 2011-09-02 07:48 · Score: 0

Or, you know, actually take a moment to fix your config... while config files ARE part of the ABI, deprecated tokens and syntax do get removed sometimes in minor versions.
Re:Be prepared to back out! by Anonymous Coward · 2011-09-02 14:23 · Score: 0

I got bitten by this moving our SSL server from 2.2.9 to 2.2.20 - they changed the config processor and our SSL config broke.
Apache claim that a given "stable" series will keep a constant ABI. They seem utterly unable to comprehend that config files count as part of the ABI.
What broke exactly? What "constant ABI" claim are you talking about? Why would the config file, or your unspecified change in behavior, be part of an ABI statement?

Re:It's time to shift our paradigm by WrongSizeGlass · 2011-09-01 22:17 · Score: 3, Insightful

Is your world black and white? Mine isn't. So why do our computers have to be binary?

Because computers use logic to operate. When you remove the binary 'black & white' nature of logic you start entering shades of grey. Shades of grey are fine for everyday life, but when they are applied to some things they just don't work. For examples see ethics, the law, loyalty, monogamy, honesty, etc (to see all of these in one example see politics).

Delay by jker · 2011-09-01 22:25 · Score: 1

On Aug. 24, project developers had promised a fix within 48 hours, then revised the timetable two days later to 24 hours. The security advisory did not explain the delay."

I don't get this part. Even if we everything in the previous statement, 2d + 2d + 1d = 5 days. The 24 was 9 days ago.

What did I miss?

Debian, on the other hand, gave a fix 5 days later (4 days ago), way before the upstream.
http://www.debian.org/security/2011/dsa-2298

Re:Delay by Tomun · 2011-09-01 23:23 · Score: 1

The summary says Apache 2.2.20 was released on Tuesday, the day after the Debian fix.
Re:Delay by Anonymous Coward · 2011-09-01 23:43 · Score: 0

Some vendors were on top of this and Debian gets credit.
MidnightBSD had the new version in mports the day it was released. FreeBSD sat on it until 5 hours ago! So, in this case, decent Linux gets a patch right away and BSD lags.
Re:Delay by omnichad · 2011-09-02 01:34 · Score: 4, Funny

Even if we everything in the previous statement,

What did I miss?
The verb.

Re:It's time to shift our paradigm by WrongSizeGlass · 2011-09-01 22:28 · Score: 1

BTW, how did you get it through the lame(ness) filter? I get a filter error in just quoting your post ("Filter error: That's an awful long string of letters there.") Only after removing the binary strings, Slashdot allowed me to even preview.

I think he added a space after the first string of binary numbers (before a <br /> tag).

00110001001101010011100100110001001100000011001000110111
00110001001101010011100100110100001101100011001000110001

Yup, that space worked.

Is there a specific reason you padded them to 56 bits?

Adding another eight leading '0's results in the "Filter error: That's an awful long string of letters there." error.

May not be sufficient by mrsam · 2011-09-01 23:07 · Score: 1

I haven't looked at this fix in detail, but from the sounds of it, I'm not convinced that the fix is complete.

The attacker, for example, could request 999,999 individual one byte ranges of a 1,000,000 byte document. In a partial range response, each individual partial range gets wrapped into a separate MIME entity. The response from the server is basically a multipart MIME document. There's significant overhead per MIME section. Each single byte of the document gets attached to a header that, perhaps would be around 40-50 bytes long. Still quite a bit of bandwidth amplification.

Re:May not be sufficient by ledow · 2011-09-02 00:49 · Score: 1

As the fecking article says, the patch:
"weeds out or simplifies requests deemed too unwieldy."
Otherwise, it wouldn't be much of a patch because it wouldn't fix this problem at all.

OpenBSD by Anonymous Coward · 2011-09-02 00:27 · Score: 0

Why am I not surprised?

VMware says protocol bugs introduced by fix by Anonymous Coward · 2011-09-02 04:17 · Score: 0

This VMware bulletin warns of using the new 2.2.20 as it intoduces protocol bugs and application incompatibilities. Also they say Apache 1.3 is not affected as the original advisory claims.

http://pastebin.com/dHMnEjxh

Re:Be prepared to back out! and then do some HW by Anonymous Coward · 2011-09-02 07:29 · Score: 0

Also be careful if you are just "trusting" that RedHat rolled this code into their RPM. The backport (do not get me started on that rant) of this "fix" for httpd-2.0.52 does not actually incorporate any of the code release by Apache. You will likely review what you are putting on your server. You can choose to do it before you roll it out, or you may be forced to do it after you are hit with the same script that you thought you were protected against. It only takes a small modification to that Perl script to make a '5-' into a '6-' and you are not protected from that script.

Slashdot Mirror

"Apache Killer" Web Server Hole Plugged

48 comments