Slashdot Mirror
Stop Fixing All Security Vulnerabilities, Say B-Sides Security Presenters
PMcGovern writes "At BSidesLV in Las Vegas, Ed Bellis and Data Scientist Michael Roytman gave a talk explaining how security vulnerability statistics should be done: 'Don't fix all security issues. Fix the security issues that matter, based on statistical relevance.' They looked at 23,000,000 live vulnerabilities across 1,000,000 real assets, which belonged to 9,500 clients, to explain their thesis."
Prioritize on the important vulnerabilities. But that should in no way discourage people from fixing the less important ones.
Don't let perfect become the enemy of good.
In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
Important items get fixed first. Easy items usually come next. Everything else gets fixed after that.
Keep the Classic Slashdot.
The article is talking about fixing what you can. It simply outlines how to prioritize the issues in order to figure out what you can fix with limited resources.
Their real point is, if you have limited resources, prioritize the vulnerabilities that are (a) currently being exploited and (b) most likely to be exploited given the habits of your favourite boogeyman. Sometimes that means not starting on vulnerabilities as soon as they come in, because you're saving your resources for the chance there's a bigger problem later. Their thesis is about saving your money and time for the most important stuff, and assumes that threats only come from lazy blackhats who prefer certain classes/types of vulnerabilities. Buried in this is the assumption that a given piece of software has an infinite number of vulnerabilities that are discovered at random.
Statistically, what they're saying is sound if organized crime is your biggest enemy, assuming organized crime's habits don't change any time soon. It's obviously not good enough if you're concerned about, say, a malicious government organization with an absurd budget.
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
They say stop when they mean prioritize. Theoretically, there should be some computer scientists who know how to use English.
How does software like djbdns seem to be nearly free of discovered vulnerabilities? Is it a popularity/type-of-user thing? Or has the code genuinely been written to be almost impenetrable?
tl;dr Why do so many things need fixing in popular pieces of software which could easily command the most competent developers?
The article is talking about fixing what you can. It simply outlines how to prioritize the issues in order to figure out what you can fix with limited resources.
That's a pretty damn weak model. It doesn't take a genius to understand that if you use statistics to prioritize security issues to address (or more to your point, cull out ones you won't address due to limited resources), then it's only a matter of time before attackers start figuring out ways to use those statistical models against you, ultimately learning about the "can't-get-to-it" threat list and focusing attack vectors there.
Not to mention management being "sold" on this model and cutting 20% of your IT support staff next year due to the "increased efficiencies of patch management". Have fun doing more work.
I believe the word is 'triage'..
“He’s not deformed, he’s just drunk!”
How about you fix what you can?
That's the fly-swatter approach - you hit the flies you can and ignore those you can't get to.
'Don't fix all security issues. Fix the security issues that matter, based on statistical relevance.'
That line reminds me of the old TQM which was run past us decades ago (and then promptly forgotten by 90% of the Franklin Planner-toting crowd), fix what really needs fixing first. I'm sure this bit of wisdom didn't require TQM to come along (you can probably find it in Hamlet if you know where to look), you fix your most grievous would first and worry about your bruises later, but we (in my department) felt rather put-upon when these TQM zombies came around and told us what a sea-change it would be for our practices and productivity when we embraced what we already knew.
A feeling of having made the same mistake before: Deja Foobar
OpenBSD takes the approach of proactive code audits and of fixing all bugs found, even those that have no apparent potential for exploitation. This has really paid off over the years. Often when vulnerabilities came to light, they were found to not affect OpenBSD because the underlying bug had already been fixed.
Disclaimer I have not read the paper.
Once upon a time I did software documentation for a fast moving product. I was never given updates and worked basically in the dark. One brilliant manager asked me to, "Document all the bug fixes for this product." There were over 2,000. At 15 minutes each that time span was a bit over the week I was given. Doing the math, this comes out to 12.5 40 hours weeks, uninterrupted. At half time -- a better estimate this would have been half a year. One week is not 25.
I requested a list of the bugs, sorted by priority. I was met with stares. I then said, "Until I get the list, I will work in strict numerical order until I get the list." The manager screamed at me, "But I don't want that." This time I replied, "I agree, but until I get the list from you I will do the work in numerical order, just so you don't yell that am not working." I never got the list and the random selection from the strict list was a nice demo of the types of bugs found. The end result was OK, but not by choice. Introduction done, but this was a similar problem. You need some guidance 'cause you cannot do everything.
With limited resources, fixing everything all the time is an infeasible task. Using a pure visual analogy, "fix the biggest hole." The problem is that as bad as people are at fixing, they are perhaps even worse at classifying. And assessing the potential damage of a "hole" is another part of the problem. You must also assess the likelihood of someone finding/using that hole. Add in the reality that each fix is time-sensitive and the time to fix a bug is all over the place, and you have a very real mathematical and practical mess. "What do we fix first?" is neither a short or an easy question to answer.
If you line up all of your straw men in a row, they will look like an army and scare your opponent away.
Theoretically, there should be some computer scientists who know how to use English.
Theory and reality are the same, in theory. In reality, however...
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
it's only a matter of time before attackers start figuring out ways to use those statistical models
They already do. They use attacks that hit the largest number of targets. Using uncommon vulnerabilities would be wasteful when you could attack more common ones.
That's why you need a game-theoretic, adversarial model instead of a simple statistical model based on past observations. Regret minimization, multi-arm bandits, etc.
weinersmith
No, GP is right. It's a different scenario, but it's valid:
If 1 in a thousand users installs X, find a way to target X across a corporation. One hit gets you in. Beachhead there, figure out where to go next or what you can collect.
**THAT** is how to discreetly pwn a corporate net.
Some attackers go big, because their payout is # of machines taken. Some attackers are after a narrow niche: what'll company X be announcing, how their stock is likely to perform, data of value to competitors, etc. Their payout is the same if they own one or a thousand machines in a corporation.
Sorting bugs into "security vulnerabilities" and "other" is prioritizing.
Security people talk as if the start point is security vulnerabilities. It's not. From a functional view, there's not much difference between a bug that breaks crucial functionality, and a DoS attack.
It's amazing how many security vulnerabilities rely on age old bugs such as buffer overruns and dirty memory that are easily fixed if we're willing to live with slightly slower computers. We can program the OS to blank memory whenever it is allocated and for extra safety, whenever it is freed. We can add bounds checking to library routines such as C's infamous gets(). But all that hurts performance. It's a tradeoff we've been unwilling to make. How many people run SELinux? For most, the time it takes to learn and administer an SELinux system is just not worth the scanty benefits. For instance, SELinux does very little to protect you from NSA snooping. Another approach is the microkernel. Why haven't we pursued microkernels? A microkernel architecture might reap ten times the security benefits we could ever hope to net by constantly patching drivers. Gets rid of a whole class of vulnerabilities involving the escalation of flaws in drivers. What we can do cheaply is punt illegal access problems to the OS and hardware, leaning on virtual memory paging to protect programs from each other and No Execute bits to protect a program's code from itself.
The entire approach of zeroing in on vulnerabilities is fire fighting. And saying that we should choose which fires to fight is not particularly insightful or helpful. Better to build systems that don't catch on fire so easily.
Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"