Slashdot Mirror
Is Finding Security Holes a Good Idea?
ekr writes "A lot of effort goes into finding vulnerabilities in
software, but there's no real evidence that it actually improves security. I've been trying to study this problem and the results (pdf) aren't very encouraging. It doesn't look like we're making much of a dent in the overall number of vulnerabilities in the software we use. The paper was presented at the Workshop on Economics and Information Security 2004 and the slides can be found here (pdf)."
I guess I'm a little unclear on what the research stated is supposed to actually accomplish.
Roving Web-Teleoperated Robot
If you find a security hole then the mistake has already been made. Fix the hole, but also make sure the same bug doesn't exist in any other program. Finding the same exploits again and again (buffer overruns, format string vulnerabilities, tempfile symlink vulnerabilities) reflects very badly on free software and programmers' ability to learn from bugs found in other software. (Not that proprietary software is necessarily any better - I am just not discussing it here.)
The OpenBSD people have built up a good track record on security by finding holes and fixing them everywhere possible. I am sure they would disagree with your assertion that finding holes does not help to improve security. Finding the bugs is an important first step towards not putting them back in next time you write code.
-- Ed Avis ed@membled.com
In principle, I agree that automatically installing patches is a good thing in principle. But Microsoft has a habbit of changing their licenses and installing DRM when people "upgrade" and/or "install patches."
Also, imagine I have 2 programs. Both automatically install patches. Unpatched they both work fine. But when program #1 is patched, program #2 cannot run at all. Now this will probably be fixed eventually, but in the mean-time, I cannot run program #2 at all. If I need both programs, I'm fucked with the system of auto-patches.
However when I have a choice, how likely am I to install a patch? Not as likely (due to laziness). So the effectiveness decreases significantly.
Someone finds a buffer overflow problem. Someone finds another one. Someone finds another one. Someone finds another one.
Someone realizes: "what if I centralized all my buffer routines and just got one little section of code working perfectly?" Then you get projects like qmail or vsftp, which simply are more secure. Then people start using these programs, and their systems really are less vulnerable.
This paper looks keeps using the phrase "given piece of software." It's talking about (lack of) improvements at a micro-scale, but ignores improvements in the big picture that can happen due to people getting fed up or scared.
If vulnerabilities were not discovered, would anyone bother to develop secure software?
I think this paper has as an underlying assumption, the popular view that it just isn't possible to write secure software, and that every large software system is just a big leaky sieve that requires perpetual patching. I don't accept that. I understand why the belief is widely held: most popular software was not originally written with security issues in mind. But this is something that will be overcome with time, as legacies die off.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
But you should be able to override the default behavior of auto-installing patches; my thinking would be that systems should patch themselves automatically unless the user specifies that they shouldn't.
The issue still remains though that an unpatched system is still vulnerable, if the patch breaks an application and the machine goes unpatched there is a loss in security because of potential intrusion. If the patch is applied there is a potential loss of productivity. This is the kind of call a sysadmin has to make for their network, but a sysadmin should know enough to make the decision in an informed way, the average computer user is not equipped in the same way and probably should recieve the patch in order to mitigate risk that user's compromised system may cause to the greater group of users they may connect to via the internet.
Even if I knew that tomorrow the world would go to pieces, I would still plant my apple tree. -Martin Luther
The parent is exactly right. Having read through this paper now, I realize what it misses: the economic impact of the information.
Much work has been done in economics regarding the affect that inadequate information flow has on a market; a Nobel Prize was wone in it lately. The paper assumes that there are a constant number of vulnerable machines, as you can see on page 2, for any given vulnerability. First of all, it ignores the fact that someone has to choose to use these vulnerable products. Second, it ignores the choice that comes to sysadmins when they learn that a particular company's products are more likely to have bugs, as the parent describes.
The moral of the story is, the paper tries to be more broad than it can - by assuming that software acquisition decisions never happen, it fails to see the effect of vulnerability disclosure on these decisions. And these decisions, made well, do in fact make us more secure. The "software defect rate" in total might not decrease, but the defect rate in *used* software may well decrease.
|/usr/games/fortune
Refer back to last week's recent story regarding the Royal Bank's problems with an upgrade in Canada. Auto-patching might very well lead to something like this on a larger scale, or affecting numerous small businesses.
"A lot of effort goes into funding law enforcement in society, but there's no real evidence that it actually reduces crime. I've been trying to study this problem and the results aren't very encouraging. It doesn't look like we're making much of a dent in the overall number of crimes in our society."
---
If you think security is bad now, just stop fixing security vulnerabilities and see how much worse things get. It's like a sump pump - it may not fix the leak, but it'll keep you from drowning.
Black Hats are dynamic actors -- if the world changes so that Figure 2 in Eric's paper is the norm rather than Figure 1, the Black Hat community will evolve to live in the new world. Their new goal will be to maximize area under the "Private Exploitation" part of Figure 2. We may be better off with the current state of affairs.