Due Diligence?

ekr writes "The OpenSSL remote buffer overflows discovered at the end of July got a lot of press here on /. But how many people actually fixed their machines? I decided to study this question, and the results are kind of depressing. Two weeks after the release of the bug, over two thirds of the servers I sampled were still vulnerable. Even two weeks after the Slapper worm was announced, a third of the total servers were vulnerable. The paper can be found here in PDF or Postscript."

Comments

It's not just laziness...

[Anonymous Coward]

Many systems administrators aren't full-time and have other responsibilities. Keeping up-to-date with every security patch is very time consuming and sometimes management doesn't understand this and doesn't allocate resources for it as long as things are "working".

Re:It's not just laziness...

[dfn5]

Unfortunately keeping up with patches is a very important part of any security strategy. I am all for letting companies do things their way, but if admins don't allocate more time to security and patching then I'm afraid the government will do more than just recommend actions for Security on the Internet and will start mandating stuff. I for one don't want that to happen.

Bottom line? Improve your security while you still have the rights to do it yourself.

this is why...

[mr_gerbik]

This is why I run Windows 3.11. No worries about falling behind and not installing the latest fixes.

Securing OpenSSL

[Exmet Paff Daxx]

Some points to consider:

• Fear. Most Linux users are probably reeling in shock from the recent trojan inserted by elite hacking group ADM into the libpcap distribution. The old standby argument that 'checking the MD5 signatures' will save you has become null & void; ADM replaced the MD5 signatures too. The only reason the trojan was detected was because of the Google cache! This kind of thing probably has most users afraid to move to anything recently released that hasn't been extensively peer reviewed.
  
• Ignorance. Since the Slapper worm only contains offsets for a handful of platforms, many flavors of Linux are 'immune' to automated infection. While blackhat groups have offsets for nearly every implementation of Apache/SSL in existence (yes, even you x86 Solaris people), this threat isn't considered 'immediate' enough to justify the third point:
  
• Sloth. Upgrading your OpenSSL isn't as easy as it could be. You actually have to recompile Apache with ./configure flags to link it to the new version of OpenSSL which you just recently downloaded (it's not trojaned... right?). Sounds easy, but for a production server that hasn't been touched in a year, this tends to make people really nervous
  

All of this points to the fact that there is a fundamental flaw in the way that the Open Source community is securing their software. Putting MD5 signatures on the same server that the software is available from isn't even close to secure - Dave Aitel of Immunity Security keeps hammering on this point in BugTraq. And we're going to see even more of this 'Upgrade Fear' as more and more distributions get trojaned - Slash is probably next on the list.

We need to look at existing, successful solutions to this problem (like Windows Update) and catch up. Now.

Re:Securing OpenSSL

[schulzdogg]

The old standby argument that 'checking the MD5 signatures' will save you has become null & void; ADM replaced the MD5 signatures too. The only reason the trojan was detected was because of the Google cache! This kind of thing probably has most users afraid to move to anything recently released that hasn't been extensively peer reviewed.

False. From the HLUG website (the group that discovered the trojan):
Thanks to Antioffline.com for hosting us, and Gentoo's Portage system for catching the trojaned files via checksums.

Putting MD5 signatures on the same server that the software is available from isn't even close to secure

This is true though.

Should have upgraded

[Hegemony]

I didn't and paid for it. Within about 3 hours time, some bastard got in, created a superuser, DoS'd nasa.gov, spawned a few irc servers, and started scanning other IP's for the same exploit. Damn they work fast.

When to Patch

[Crispin Cowan]

Readers interested in this topic may be interested in this paper that we presented last week at USENIX LISA 2002:
Timing the Application of Security Patches for Optimal Uptime

Steve Beattie, Seth Arnold, Crispin Cowan, Perry Wagle, and Chris Wright
WireX Communications, Inc. http://wirex.com
and
Adam Shostack
Informed Security http://www.informedsecurity.com

Security vulnerabilities are discovered, become publicly known, get exploited by attackers, and patches come out. When should one apply security patches? Patch too soon, and you may suffer from instability induced by bugs in the patches. Patch too late, and you get hacked by attackers exploiting the vulnerability. We explore the factors affecting when it is best to apply security patches, providing both mathematical models of the factors affecting when to patch, and collecting empirical data to give the model practical value. We conclude with a model that we hope will help provide a formal foundation for when the practitioner should apply security updates.

Crispin
----
Crispin Cowan, Ph.D.
Chief Scientist, WireX Communications, Inc.
Immunix: Security Hardened Linux Distribution
Available for purchase

Re:Vulnerability Check

[James_G]

How does one check if a server is vulnerable without actually "breaking in", i.e. make oneself liable to prosection? I skimmed through the PDF but could not find anything about this.

Well if you'd read the PDF instead of skimming it, you would have seen this:

Thus, we can simply connect to the HTTPS server and issue a HEAD request. The server responds with an HTTP header containing the Server: field and hence the answer we desire:

Server: Apache/1.3.26 (Unix) mod_ssl/2.8.10 OpenSSL/0.9.6

They then went on to verify that SSLv2 is not disabled, but they mention later in the paper that on only 10 hosts was this done.

Theoretically you could change the response to report the more recent version which would make this check innacurate, but why would anyone bother doing that? Far easier to just upgrade OpenSSL.

Damn MCSEs

[davinciII]

See, this is exactly what happens when you hire a bunch of paper MCSEs to run your........

wait, did you say Linux?

Due diligence.

[Black Parrot]

This is too easy, folks. Subscribe to your distro's update announcement list, read your mail daily, and apply the relevant patches promptly.

It's really not that hard. A typical update for me is:

1. read mail
2. ncftpget whatever.rpm
3. rpm -Uhv whatever
4. read rest of mail

By far the most time-consuming part is waiting for the RPM to download. Some say that it's even easier for source-based distros.

Run right out and get nessus.

[AugstWest]

If you haven't yet, you should definitely check out nessus.

It'll scan your machiens for known vulnerabilities and give you pointers on how to go about taking care of any that it finds. It's also got built-in updating to pull in the latest exploits.

The clients are even getting pretty spiffy these days, and the project has matured very rapidly.

Have we grown complacent?

[mao che minh]

Perhaps Linux users and administators have grown overly comfortable due to the long reign of tight security and lack of virii? Until rather recently, disclosed security advisories for FOSS could be neglected for substantial periods of time without worry. The world's hackers mostly took aim at easily exploitable IIS and Exchange servers, flimsy Win32 email clients, and major routers (like AT&T backbone routers to Asia and such). Largely ignored were the hordes of vulnerable web and mail Linux/BSD servers on campus networks and elsewhere (mostly left vulnerable due to neglect, not inherent OS issues). However, the desire to orchestrate large scale DDoS attacks and an exponential increase in the use of Linux systems has caused many hackers to take interest in conquering new grounds.

All of these years of rock solid security has made us complacent. We have to remember that, while Linux and OSS may be inherently secure, and Linux's modular design works as a fail safe against complete failure, we are still just as vulnerable if we don't remain vigilant.

Weird misconception

[dfn5]

I find that other admins patch by necessity. i.e. If something is broke, then patch it. If not leave it alone.

However, I read a stat somewhere that said that a large majority of security breaches could have been prevented by merely keeping up with patches. Therefore my philosphy is to create a patch schedule. And because I'm on Solaris things like OpenSSL are 3rd party to the OS, therefore I upgrade immediately. I rebuilt my solaris RPMs of OpenSSL that day and had it deployed to all my machines. Other things like GnuPG, IPFilter, OpenSSH, apache, sendmail, etc... they all need to be upgraded ASAP.

So all you Slashdot readers who posted that you have nothing to do but read Slashdot in that downsizing article, get off your butts and start patching. That should keep you busy full time.

Re:Weird misconception

[mao che minh]

Yes. I think that services like the "Red Hat Network" will greatly benefit end users and admins alike in this respect. Having a service that organizes errata (updates) and informs you what the current security threats are, and then shows you what systems you own/administer are vulnerable is very helpful. It gives end users an almost hands-free way of keeping themselves safe (as safe as they can in terms of updates, anyways), and can point out things that admins might have missed. I really like it.

Downstream Liability

[sczimme]

In the same vein, readers might be interested in a presentation we did at RSA 2002; the topic was Downstream Liability for Attack Relay and Amplification

(A two-page summary is available here.)

We described a scenario in which an intruder compromised a system, used it as a jumping-off point, and subsequently caused damages to an individual. The paper focuses on the legal side of things; IANAL but the other two authors - Tim Rosenberg, J.D. and Ron Plesco, J.D. - are.

I also state my opinion, which is that "...patches should be installed no later than ten (10) calendar days after release of the patch by the vendor". Discuss.