Slashdot Mirror
Due Diligence?
ekr writes "The OpenSSL remote buffer overflows discovered at the end of July got
a lot of press here on /. But how many people actually fixed their
machines? I decided to study this question, and the results are kind of
depressing. Two weeks after the release of the bug, over two thirds of
the servers I sampled were still vulnerable. Even two weeks after
the
Slapper worm was announced, a third of the total servers
were vulnerable. The paper can be found here in
PDF
or
Postscript."
Many systems administrators aren't full-time and have other responsibilities. Keeping up-to-date with every security patch is very time consuming and sometimes management doesn't understand this and doesn't allocate resources for it as long as things are "working".
This is why I run Windows 3.11. No worries about falling behind and not installing the latest fixes.
I noticed connection attempts from Korea just after the announcement and decided it was time to nuke the the boxes from orbit. Not much point in having an O-BSD box you are only mostly sure of.
I had some angst with RedHat boxes, though. The update mechanism didn't change the reported version number of OpenSSH. Annoying.
All of this points to the fact that there is a fundamental flaw in the way that the Open Source community is securing their software. Putting MD5 signatures on the same server that the software is available from isn't even close to secure - Dave Aitel of Immunity Security keeps hammering on this point in BugTraq. And we're going to see even more of this 'Upgrade Fear' as more and more distributions get trojaned - Slash is probably next on the list.
We need to look at existing, successful solutions to this problem (like Windows Update) and catch up. Now.
If guns kill people, then CmdrTaco's keyboard misspells words.
IMHO the right way is to write a system that disables servers automatically if a vulnerability is known and the administrator did not fix it in n days. Either the functionality is integrated into the daemons, and they check (via http, mail, whatever) every day whether they are affected by a problem. Or each system should run a daemon that controls all server software.
It should warn the admin before it turns the server off, of course, but a broken unmaintained server is always better than a hacked server.
How sure are you the the administrators of the servers you sampled are also Slashdot readers? While certainly some laziness could explain your statistics, what of good old-fashioned lack of communcations? Just because a message warning about a security hole was sent out, doesn't mean it got received, or even read in a timely manner. Besides, maybe most of those administrators were taking three-week vacations just then!
I worked at an ISP at the time that this was happening and we were quite well aware of these vulnerabilites. We often referred to CERT when looking for vulnerabilities that may have affected us. It was through sites like those that we found out about the problems with OpenSSL and we made the necessary changes. I'm not sure why it was found that many others didn't do what was necessary. Perhaps there are many admins that don't understand that they need to keep themselves up to date on these matters, and of course, they are often busy with many other tasks. It's not easy being an admin. Maybe that's why there is a System Administrator Appreciation Day.
I didn't and paid for it. Within about 3 hours time, some bastard got in, created a superuser, DoS'd nasa.gov, spawned a few irc servers, and started scanning other IP's for the same exploit. Damn they work fast.
Timing the Application of Security Patches for Optimal Uptime
Steve Beattie, Seth Arnold, Crispin Cowan, Perry Wagle, and Chris Wright
WireX Communications, Inc. http://wirex.com
and
Adam Shostack
Informed Security http://www.informedsecurity.com Crispin
----
Crispin Cowan, Ph.D.
Chief Scientist, WireX Communications, Inc.
Immunix: Security Hardened Linux Distribution
Available for purchase
Well if you'd read the PDF instead of skimming it, you would have seen this:
Thus, we can simply connect to the HTTPS server and issue a HEAD request. The server responds with an HTTP header containing the Server: field and hence the answer we desire:
Server: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6
They then went on to verify that SSLv2 is not disabled, but they mention later in the paper that on only 10 hosts was this done.
Theoretically you could change the response to report the more recent version which would make this check innacurate, but why would anyone bother doing that? Far easier to just upgrade OpenSSL.
Prevent email address forgery. Publish SPF records for y
That's why MS wants to make apps that upgrade themselves automagically
It's not a bad idea after all, too bad you can't trust MS on anything (They use a good idea bundled with a bad one and a EULA that grants them too much)
See, this is exactly what happens when you hire a bunch of paper MCSEs to run your........
wait, did you say Linux?
This is too easy, folks. Subscribe to your distro's update announcement list, read your mail daily, and apply the relevant patches promptly.
It's really not that hard. A typical update for me is:
- read mail
- ncftpget whatever.rpm
- rpm -Uhv whatever
- read rest of mail
By far the most time-consuming part is waiting for the RPM to download. Some say that it's even easier for source-based distros.Sheesh, evil *and* a jerk. -- Jade
If you haven't yet, you should definitely check out nessus.
It'll scan your machiens for known vulnerabilities and give you pointers on how to go about taking care of any that it finds. It's also got built-in updating to pull in the latest exploits.
The clients are even getting pretty spiffy these days, and the project has matured very rapidly.
This could be a nightmare in itself. What happens if the update server gets hacked... then all of a sudden you have systems either auto-trojaning themselves - or shutting down everywhere.
Not really a good idea... an equivilient to *gasp* "windows update" for terminal would be nice (RH8 has one, if you pay for or try RHN), where it automatically gives you a list of updates available and allows you to pick them in a dselect-style (debian) format, or something similar.
Perhaps Linux users and administators have grown overly comfortable due to the long reign of tight security and lack of virii? Until rather recently, disclosed security advisories for FOSS could be neglected for substantial periods of time without worry. The world's hackers mostly took aim at easily exploitable IIS and Exchange servers, flimsy Win32 email clients, and major routers (like AT&T backbone routers to Asia and such). Largely ignored were the hordes of vulnerable web and mail Linux/BSD servers on campus networks and elsewhere (mostly left vulnerable due to neglect, not inherent OS issues). However, the desire to orchestrate large scale DDoS attacks and an exponential increase in the use of Linux systems has caused many hackers to take interest in conquering new grounds.
All of these years of rock solid security has made us complacent. We have to remember that, while Linux and OSS may be inherently secure, and Linux's modular design works as a fail safe against complete failure, we are still just as vulnerable if we don't remain vigilant.
However, some people take vacations, or go on project, or the such. Some people even sleep, and a window of vulnerability of only a few hours can create a serious problem. Your advice is perfect for those situations which need it the least - where the system is regularly serviced by a nearly-constantly-available administrator. This by no means covers all situations.
- As should be noted from Figure 4, updating on Linux was rather easier than updating on *BSD, since all of the *BSD updates required compilation, either of the base system or from the ports/packages collection.
Huh?# pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/ports/packages/A ll/openssl-0.9.6g.tgz
Binaries installed -- no compilation required!
However, I read a stat somewhere that said that a large majority of security breaches could have been prevented by merely keeping up with patches. Therefore my philosphy is to create a patch schedule. And because I'm on Solaris things like OpenSSL are 3rd party to the OS, therefore I upgrade immediately. I rebuilt my solaris RPMs of OpenSSL that day and had it deployed to all my machines. Other things like GnuPG, IPFilter, OpenSSH, apache, sendmail, etc... they all need to be upgraded ASAP.
So all you Slashdot readers who posted that you have nothing to do but read Slashdot in that downsizing article, get off your butts and start patching. That should keep you busy full time.
-- Thou hast strayed far from the path of the Avatar.
One installs and runs nessus, updates the plugins and scans one's servers weekly.
One also tends to sleep well at night afterwards.
MOD THE CHILD UP!
Thank you for your answer and sorry I cannot express myself better.
Actually I did really _read_ the parts you quoted. My question was more about laws. Doesn't probing if it is vulnerable (i.e. actually "playing the attack") count as attacking?
THIS was what I was asking. But thanks for beeing so friendly and trying to allege me I am a stupid cocksucker.
Cheers.
Gentoo had the OLD checksums, which is the reason it was caught. Everyone who checked the new checksums got owned. The Gentoo suspicions were confirmed by checking the Google cache.
Gentoo basically caught this because they were so far behind the curve that they still had the old distribution. While it's a great argument not to use Gentoo, this kind of security-through-being-behind accident is not a security process, nor is it repeatable, nor should it be considered a success of the checksum system.
--
What happens when you outlaw guns
>2 minutes? Like an hour?
Thus, we can simply connect to the HTTPS server and issue a HEAD request.
One easy way I discovered to do this, assuming you've got the curl library installed, is to issue this from the command line:
curl -I [your_url__or_hostname_here]
Probably old hat to most of this group, but it proved an easy way to check several servers quickly. It's also handy if you want to make sure that your changes to Apache's httpd.conf (for ServerTokens and ServerSignature) actually "took".
In the same vein, readers might be interested in a presentation we did at RSA 2002; the topic was Downstream Liability for Attack Relay and Amplification
(A two-page summary is available here.)
We described a scenario in which an intruder compromised a system, used it as a jumping-off point, and subsequently caused damages to an individual. The paper focuses on the legal side of things; IANAL but the other two authors - Tim Rosenberg, J.D. and Ron Plesco, J.D. - are.
I also state my opinion, which is that "...patches should be installed no later than ten (10) calendar days after release of the patch by the vendor". Discuss.
I want to drag this out as long as possible. Bring me my protractor.
... that keeps probing my servers!!
On the mac, I just set 'Suftware Update; to run daily, and I click 'install' when it says there's a security fix.
By default, users only have to click one button (the default button) to keep their Mac-flavored BSD secure.
And they don't have to subscribe to mailing lists or be security geeks. They could be your mom and still get it right.
Not trying to rip on your mom.
I don't even know her!
No, seriously. That wasn't me, that time at the Quaker Steak n Lube!
Kevin Fox
Comment removed based on user account deletion
The paper looks at version numbers but does not account for back patches to old versions that fix the bug. I'm running a patched Mandrake https server which returns a version of 2.8.7/0.9.6c. Slapper requests correctly return an error message. What the paper needs to do is issue the exploit itself to determine whether or not things have been patched. Otherwise, the author overcounts the vulnerable systems out there.
I also checked the browsers, mainly command line a little while ago when the IE cert chain vulnerability was found. Most (wget, links, lynx) didn't bother to check the chain. Some didn't check anything at all, so any proxy server could spoof any page.
If you can see https://www.amazone.com, your browser is badly broken. amazone.com points to amazon.fr - but you should match the cert to the DNS.
Opera on the Zaurus was also vulnerable. Apple doesn't install any certificates in their OS X or Darwin OpenSSL directory.
One thing that happened between SSLeay (the original project) and OpenSSL is that the certificate chains were NOT installed by default, so everyone had a library, but no way of checking certificates since you require root certificates to check the site certificate. A second thing, probably worse is that the old default was to return an error if the certificate couldn't be validated. Now the default is TO GIVE NO ERROR IF THE CERTIFICATE CANNOT BE CHECKED. It would be better to give an error that would have to be overridden, which would cause developers to have to take a look and to actively disable security.
Curl was the only one that included any checking, but it required manually installing certs and specifying an option to turn it on. It would SILENTLY connect to SSL sites without security.
Mozilla was fine, and Konqueror fixed any problem it had, but the Opensource community should be embarrassed since the rest of the browsers security was not just flawed like IE, but DISABLED without any notice to this effect or NONEXISTENT.
If only we could all use decent distributions...
I'm sitting here on my beautiful, instantly up-to-date Debian box with a terminal open to a Solaris production server. Now I'm sure there's some way to get the binary distribution of Apache to install, but I'm not sure I'll be able to actually figure it out in less time then it would take me to configure and compile the source. Of course who knows how long that will take if I have to hunt down the Solaris packages for all those "useless" tools like a C compiler that aren't installed by default.
Yes, my 31337 h4x0r friends, this is one box that won't ever be secure until I convince the boss that SPARCs should be running Linux.
Comment removed based on user account deletion
But how many people actually fixed their machines? I decided to study this question, and the results are kind of depressing.
If you're depressed by that, you might want to see a psychiatrist. I mean, you shouldn't have that kind of reaction to such a minor issue.
Is there any reason, aside from debugging and diagnostics, to present the world with accurate information about the versions I run?
I sure as hell can't think of any.
In fact I am going to go over my system here and make damn sure that I stop any such foolishness...
In the free world the media isn't government run; the government is media run.
I bet you don't allow ICMP through your firewall either.....
People who think they know everything are a great annoyance to those of us who do.
- They were certainly well-known when I was in college in the mid 70s, but the PL/C dialect of the PL/I checkout compiler corrected mistakes like that at run-time. (OK, it often fixed them incorrectly, but at least least it wouldn't overrun an array.) And our professors dinged us for writing programs where that happened, and made us run the programs on input decks that were maliciously designed to check for programs that overflowed their buffers.
- They were certainly well-known when K&R wrote their books on C which warned you to be careful about bounds checking when using pointers and arrays.
- They were certainly well-known in the early 80s when everybody started complaining that the gets() and scanf() routines made it easy to overrun buffers on input when you weren't doing it by hand.
- They were certainly well-known in the late 80s when the Morris Worm wandered around a lot of the machines in the internet.
- They were certainly well known when the C++ string-handling libraries were designed to NOT overrun buffers, and when Java was designed to not even have pointers, and had array objects that checked bounds for you.
- There are enough software engineering CASE tools that try to find problems more complex than lint() looks for, though perhaps array bounds checking isn't something they check effectively.
I like C - I really like it. It's time to stop using it. It's time to stop shipping code that has array bounds problems, and security code that hasn't been proofread for them. And it's time to stop using programs that run as root when they don't need to. This isn't the 80s any more.There are other bugs out there - a popular attack is to try to abuse dotdots in path names, which there's more excuse for forgetting to check, and there are things like race conditions that are genuinely hard to check for (e.g. what happens when somebody's ripping up your temp files while your program is running), though checking return codes on system calls and doing something appropriate about failures is a good start.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
And frequently the patches cause more problems that they fix. This is why my company has switched to OSS/Linux (eat that GNU).
If you are a MS server admin you are always double checking TechNet and the other available sources because the delay between the patch on TechNet and the WindowsUpdate (critical area) can be as long as 3 weeks sometimes. Now that's sorry as can be. Lockdown tool is a joke.
I know of a Linux system that logs and reports intrusion attempts by CodeRed/Nimda, Slapper, et. al., and mails a report to a system admin every morning.
The system admin wasn't pursuing these reports. I asked why.
His response - "Well, those are attempts to exploit a Windows server, and this is a Linux box, so they don't matter."
I made the counterpoint "If one of your system was infected, wouldn't you want to be told about it?"
If every systems admin would take the time to track down the Code Red attempts on their systems, and notify the responsible parties whereever possible, then a lot of the unpatched systems would be shut down (if not by their administrators, then by the ISP supplying connectivity).
I just don't understand an admin with an attitude like that.
www.eFax.com are spammers
Server: Apache/1.3.26
Set ServerTokens to "min" in httpd.conf. Why tell anybody anything you have no use in their knowing? Sure, there are other ways they can test your vulnerability. But some of them will look here first, and just go elsewhere if you're not baiting them with what they're looking for.
"with their freedom lost all virtue lose" - Milton
Dad? Is that you?
:).
I'm surprised you didn't get that one yet
Seriously, what would you suggest?
I personally dislike C - programming securely in C feels like clearing a minefield inch by inch.
So any suggestions?
Haven't seen docs on programming securely in Lisp. Many Lisp coders like to mix data and code, that seems scary, but I'm very very new to Lisp so is that safe? There's little out there to suggest it is or isn't.
Forth seems to have about the same problems as C - buffer overflows. Data-code mixing (but a Forth coder said a solution is to keep dictionaries separate).
Many of the other languages are more high level, very useful and recommended for certain apps, but not suitable for the low level system programming area.
The languages have to be able to hook to C apps very easily.
I'd forgotten that one,
as soon as I get this pineapple up my ass I'll be on it!
In the free world the media isn't government run; the government is media run.
Comment removed based on user account deletion
Prevent email address forgery. Publish SPF records for y