Slashdot Mirror
Is Finding Security Holes a Good Idea?
ekr writes "A lot of effort goes into finding vulnerabilities in
software, but there's no real evidence that it actually improves security. I've been trying to study this problem and the results (pdf) aren't very encouraging. It doesn't look like we're making much of a dent in the overall number of vulnerabilities in the software we use. The paper was presented at the Workshop on Economics and Information Security 2004 and the slides can be found here (pdf)."
Is finding security holes a good idea?
Writing Security Considerations Sections
Hmmm.
As a sysadmin, I can tell you for certain that reading bugtraq and other vulnerability lists helps me. I can study trends in software, trends in company response and protect myself against problems. If I know a new worm or vulnerability has a prerequisite configuration then I can make sure to configure software in a way where I won't be vulnerable until a patch is release or until I can apply it.
Anyone who is subscribed to bugtraq can see the bad situation some software is in. Lately there was a lot of posts about Linksys that raised my eyebrow. Do I really want to deal with a company that doesn't properly address vulnerabilities it's made aware of? Good thing bugtraq posters had a workaround for the Linksys remote administration problem.
The global economy is a great thing until you feel it locally.
The real problem in software security lies in the design of the software itself. No amount of patches and service packs can secure unsecure software. Instead to be secure it has to be biult that way from the ground up. These findings seem to make sense in this context beacause patching software doesen't change the fundamental way it works.
411 Y0UR 8453 4R3 8310NG 70 U5!! -NSA
To answer the question, yes. Finding security holes is a good idea.
To the unasked question, "Is finding individual security holes the best possible use of a security researcher's time?", the answer is No. The best use of security research is to classify different types of security holes and use that information to create a development framework where those security holes are extremely difficult to recreate. For example, you're not going to find buffer overruns in Java code, since the memory is dynamically handled for you. Eventually, having security levels, encrypted buffers, etc. will all be part of a standard developer's library.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
Evidence wouldn't show us that searching for security holes improves security... Rather, such a judgement requires reasoning and evalution of the evidence. Common sense stuff, here.
Do smashing cars head-on into brick walls improve car safety? No, of course not. Evalution of the results of the crash, and using those findings to build better cars, that is what improves car safety, and the situation is entirely analogous in the security world. The assumption is that there is always a weakest link in security, that link is the most likely one to be exploited, and the trick is finding that link and strengthening it against attacks in the future, hopefully to the point where it is more likely that other links are weaker.
This is one of the best points the author makes though. He describes that if automated installation of patches were widely deployed then the benefits to discovery would increase. The problem lies in the number of systems that remain unpatched and thus exposed. The real problem is not that Discovery is not worth the time and money spent, but that it becomes worthless if the patches created are not applied.
Even if I knew that tomorrow the world would go to pieces, I would still plant my apple tree. -Martin Luther
In principle, I agree that automatically installing patches is a good thing in principle. But Microsoft has a habbit of changing their licenses and installing DRM when people "upgrade" and/or "install patches."
Also, imagine I have 2 programs. Both automatically install patches. Unpatched they both work fine. But when program #1 is patched, program #2 cannot run at all. Now this will probably be fixed eventually, but in the mean-time, I cannot run program #2 at all. If I need both programs, I'm fucked with the system of auto-patches.
However when I have a choice, how likely am I to install a patch? Not as likely (due to laziness). So the effectiveness decreases significantly.
The problem with automated patching is that some of the patches interfere with previously working software. When you manage several hundred computers with specially designed software and a blasted patch to fix a security problem can take the computers down when the software is run, you sure as anything will never let the patch process remain automated. I'd rather test it on a few computers before broadly applying it.
I was taking one day at a time, but then several days got together and ambushed me. (from a Rhymes with Orange comic)
The parent is exactly right. Having read through this paper now, I realize what it misses: the economic impact of the information.
Much work has been done in economics regarding the affect that inadequate information flow has on a market; a Nobel Prize was wone in it lately. The paper assumes that there are a constant number of vulnerable machines, as you can see on page 2, for any given vulnerability. First of all, it ignores the fact that someone has to choose to use these vulnerable products. Second, it ignores the choice that comes to sysadmins when they learn that a particular company's products are more likely to have bugs, as the parent describes.
The moral of the story is, the paper tries to be more broad than it can - by assuming that software acquisition decisions never happen, it fails to see the effect of vulnerability disclosure on these decisions. And these decisions, made well, do in fact make us more secure. The "software defect rate" in total might not decrease, but the defect rate in *used* software may well decrease.
|/usr/games/fortune
if the patch breaks an application and the machine goes unpatched there is a loss in security because of potential intrusion. If the patch is applied there is a potential loss of productivity.
Not all patches are security patches. Many patches fix problems, such as the spell check function doesn't work correctly. Or some other function doesn't work correctly. These won't compromise security, but they may interfere with other programs.
Take a sinking boat. If you are bailing water out, and the boat isn't sinking any more, it does not follow that bailing water isn't a good idea. If you stop bailing water, you're sunk. If good guys stop finding and fixing vulnerabilities, you're sunk, too.
Refer back to last week's recent story regarding the Royal Bank's problems with an upgrade in Canada. Auto-patching might very well lead to something like this on a larger scale, or affecting numerous small businesses.
His basic theory is that he believes, given the current rates of discovery, poor patching rates and the large number of bugs that:
1) Due to massive information exchange and slow patching/fixing, post announcement explotitations are not significantly less than explotiation caused by un-announced bugs, so announcing does not help.
2) There are so many bugs out there that finding a bug and announcing it does not produce a singificant reduction in the number of "bugs unknown to white Hats" so it does not increase security significantly.
His major errors are ignoring the following: 1) Exploitation post announcement is almost entirely done against the "lesser" computers, i.e. either machines with un-important data/programs (home pc's) or important machines with incompetent sysadmin. As such the real cost of these explotations is either A) practically null, or b) high, but likely to get the incompetent sysadmin fired, resulting in i) better employment prospects for good sysadmin and ii)an over-all improvment in quality of security for that company. So while the # of explots may be higher for Post-announcement bugs, soceity has a REAL social and economic gain that is very significant.
2)A)The announcement of bugs allows people to judge how well written a program is and therefore make an informed decision to NOT buy inferior producs (say Windows).
B)That perhaps our problem is not that we are announcing the bugs but that we are not doing sufficient bug hunting. He seems to be implying that because bug hunts don't find enough bugs, the solution is not to bug hunt. But anyone with a brain should be able to see that if we are not depleating the unknown bugs fast enough than one possible solution is to tremoundously increase our bug hunting, not to stop the bug disclosures.
excitingthingstodo.blogspot.com
That's like saying that we shouldn't produce safer cars because everyone doesn't buy one.
No. Your analogy is flawed.
If cars worked like exploits and patches, then every time a safer car came out, your car would suddenly become less safe than it had been yesterday- and it would become incumbent upon you to get it fixed. Cars, being physical objects, do not behave this way.
And hell, why train drivers, because, you know, crappy drivers are everywhere. Or like saying we shouldn't make furniture fireproof, because, you know, something else will burn.
All these analogies are flawed because they miss the point. When safer drivers are trained, existing drivers don't suddenly become more liable to be in accidents. When safer furniture comes out, the furniture in your living room does not suddenly develop an odor of gasoline.
Of course, it completely disregards those of us who routinely patch our managed systems and keep them dead secure, compatibility and testing be damned.
I think it acknowledges us, but for the minority that we are. The existence on the Internet of a large number of systems remaining unpatched to published vulnerabilities is exactly the nightmare scenario everyone wants to avoid- and suggests that the publish and patch system is broken. People don't patch.
This is DMCA logic. If we criminalize software holes, only criminals will know of exploits. See the problem?
There's a big difference between "criminalizing software holes" and voluntarily agreeing not to publish exploit code. And the way that sentence is worded is extremely misleading. It suggests that if the exploits aren't published then all criminals will still have unfettered access and that isn't true. While it is true that some of the people left who know of the exploits will be criminals, most criminals will no longer know of the exploits because they aren't published and require hard work to discover. Criminals are free even now to ignore the published vulnerabilities and look for unknown ones to exploit. Few choose to do so because it's a lot of work and most of them are lazy and stupid. Not publishing the exploits would force them to always develop this way.
It comes down to this- you can either have 100% of machines unpatched to N unknown vulnerabilities, or you can have 100% unpatched to N-m unknown vulnerabilities and 50% patched to m published vulnerabilities. Even if you do publish and patch, there are still apparently an unlimited numer of unknown vulnerabilities in software. They become much more dangerous and easy to exploit once they're published, and not everyone patches. Even if you do patch, unpatched machines on the network still affect you.
Good grief!
You really think the problem is C and C++?? I know that these arn't the holy grail of programming languages but to put some of that blame on them is very nieve. You can write buggy, unsecured code with asm! It's got very little to do with the language involved (the compiler and libraries used may have an effect) since it all gets put into machine code anyway. Blame the design and implementation, not the tool.
Silly rabbit