Slashdot Mirror
Is Finding Security Holes a Good Idea?
ekr writes "A lot of effort goes into finding vulnerabilities in
software, but there's no real evidence that it actually improves security. I've been trying to study this problem and the results (pdf) aren't very encouraging. It doesn't look like we're making much of a dent in the overall number of vulnerabilities in the software we use. The paper was presented at the Workshop on Economics and Information Security 2004 and the slides can be found here (pdf)."
As a sysadmin, I can tell you for certain that reading bugtraq and other vulnerability lists helps me. I can study trends in software, trends in company response and protect myself against problems. If I know a new worm or vulnerability has a prerequisite configuration then I can make sure to configure software in a way where I won't be vulnerable until a patch is release or until I can apply it.
Anyone who is subscribed to bugtraq can see the bad situation some software is in. Lately there was a lot of posts about Linksys that raised my eyebrow. Do I really want to deal with a company that doesn't properly address vulnerabilities it's made aware of? Good thing bugtraq posters had a workaround for the Linksys remote administration problem.
The global economy is a great thing until you feel it locally.
The real problem in software security lies in the design of the software itself. No amount of patches and service packs can secure unsecure software. Instead to be secure it has to be biult that way from the ground up. These findings seem to make sense in this context beacause patching software doesen't change the fundamental way it works.
411 Y0UR 8453 4R3 8310NG 70 U5!! -NSA
To answer the question, yes. Finding security holes is a good idea.
To the unasked question, "Is finding individual security holes the best possible use of a security researcher's time?", the answer is No. The best use of security research is to classify different types of security holes and use that information to create a development framework where those security holes are extremely difficult to recreate. For example, you're not going to find buffer overruns in Java code, since the memory is dynamically handled for you. Eventually, having security levels, encrypted buffers, etc. will all be part of a standard developer's library.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
Evidence wouldn't show us that searching for security holes improves security... Rather, such a judgement requires reasoning and evalution of the evidence. Common sense stuff, here.
Do smashing cars head-on into brick walls improve car safety? No, of course not. Evalution of the results of the crash, and using those findings to build better cars, that is what improves car safety, and the situation is entirely analogous in the security world. The assumption is that there is always a weakest link in security, that link is the most likely one to be exploited, and the trick is finding that link and strengthening it against attacks in the future, hopefully to the point where it is more likely that other links are weaker.
This is one of the best points the author makes though. He describes that if automated installation of patches were widely deployed then the benefits to discovery would increase. The problem lies in the number of systems that remain unpatched and thus exposed. The real problem is not that Discovery is not worth the time and money spent, but that it becomes worthless if the patches created are not applied.
Even if I knew that tomorrow the world would go to pieces, I would still plant my apple tree. -Martin Luther
The problem with automated patching is that some of the patches interfere with previously working software. When you manage several hundred computers with specially designed software and a blasted patch to fix a security problem can take the computers down when the software is run, you sure as anything will never let the patch process remain automated. I'd rather test it on a few computers before broadly applying it.
I was taking one day at a time, but then several days got together and ambushed me. (from a Rhymes with Orange comic)
Take a sinking boat. If you are bailing water out, and the boat isn't sinking any more, it does not follow that bailing water isn't a good idea. If you stop bailing water, you're sunk. If good guys stop finding and fixing vulnerabilities, you're sunk, too.
Enter a patching service, run by say a Linux distributor. SuSE's Yast Online Update (YOU) does this very well. Patches are often zero-day, and are guaranteed by SuSE not to cause trouble with other installed software, or that dependent software is also patched. I'm sure other commercial distributors have similar services, and debian's stable branch has worked well for me as well.
Yes, it involves a certain amount of trust, but if you didn't trust anyone, you'd have to write everything yourself. Also, the company's business model depends on the reliability of said patching service, so they do their best to run it well.
Of course, license changes are evil, but they're unlikely to happen with FOSS. Yet another reason to move away from Microsoft.
His basic theory is that he believes, given the current rates of discovery, poor patching rates and the large number of bugs that:
1) Due to massive information exchange and slow patching/fixing, post announcement explotitations are not significantly less than explotiation caused by un-announced bugs, so announcing does not help.
2) There are so many bugs out there that finding a bug and announcing it does not produce a singificant reduction in the number of "bugs unknown to white Hats" so it does not increase security significantly.
His major errors are ignoring the following: 1) Exploitation post announcement is almost entirely done against the "lesser" computers, i.e. either machines with un-important data/programs (home pc's) or important machines with incompetent sysadmin. As such the real cost of these explotations is either A) practically null, or b) high, but likely to get the incompetent sysadmin fired, resulting in i) better employment prospects for good sysadmin and ii)an over-all improvment in quality of security for that company. So while the # of explots may be higher for Post-announcement bugs, soceity has a REAL social and economic gain that is very significant.
2)A)The announcement of bugs allows people to judge how well written a program is and therefore make an informed decision to NOT buy inferior producs (say Windows).
B)That perhaps our problem is not that we are announcing the bugs but that we are not doing sufficient bug hunting. He seems to be implying that because bug hunts don't find enough bugs, the solution is not to bug hunt. But anyone with a brain should be able to see that if we are not depleating the unknown bugs fast enough than one possible solution is to tremoundously increase our bug hunting, not to stop the bug disclosures.
excitingthingstodo.blogspot.com
That's like saying that we shouldn't produce safer cars because everyone doesn't buy one.
No. Your analogy is flawed.
If cars worked like exploits and patches, then every time a safer car came out, your car would suddenly become less safe than it had been yesterday- and it would become incumbent upon you to get it fixed. Cars, being physical objects, do not behave this way.
And hell, why train drivers, because, you know, crappy drivers are everywhere. Or like saying we shouldn't make furniture fireproof, because, you know, something else will burn.
All these analogies are flawed because they miss the point. When safer drivers are trained, existing drivers don't suddenly become more liable to be in accidents. When safer furniture comes out, the furniture in your living room does not suddenly develop an odor of gasoline.
Of course, it completely disregards those of us who routinely patch our managed systems and keep them dead secure, compatibility and testing be damned.
I think it acknowledges us, but for the minority that we are. The existence on the Internet of a large number of systems remaining unpatched to published vulnerabilities is exactly the nightmare scenario everyone wants to avoid- and suggests that the publish and patch system is broken. People don't patch.
This is DMCA logic. If we criminalize software holes, only criminals will know of exploits. See the problem?
There's a big difference between "criminalizing software holes" and voluntarily agreeing not to publish exploit code. And the way that sentence is worded is extremely misleading. It suggests that if the exploits aren't published then all criminals will still have unfettered access and that isn't true. While it is true that some of the people left who know of the exploits will be criminals, most criminals will no longer know of the exploits because they aren't published and require hard work to discover. Criminals are free even now to ignore the published vulnerabilities and look for unknown ones to exploit. Few choose to do so because it's a lot of work and most of them are lazy and stupid. Not publishing the exploits would force them to always develop this way.
It comes down to this- you can either have 100% of machines unpatched to N unknown vulnerabilities, or you can have 100% unpatched to N-m unknown vulnerabilities and 50% patched to m published vulnerabilities. Even if you do publish and patch, there are still apparently an unlimited numer of unknown vulnerabilities in software. They become much more dangerous and easy to exploit once they're published, and not everyone patches. Even if you do patch, unpatched machines on the network still affect you.
Good grief!
You really think the problem is C and C++?? I know that these arn't the holy grail of programming languages but to put some of that blame on them is very nieve. You can write buggy, unsecured code with asm! It's got very little to do with the language involved (the compiler and libraries used may have an effect) since it all gets put into machine code anyway. Blame the design and implementation, not the tool.
Silly rabbit