Stop Fixing All Security Vulnerabilities, Say B-Sides Security Presenters

PMcGovern writes "At BSidesLV in Las Vegas, Ed Bellis and Data Scientist Michael Roytman gave a talk explaining how security vulnerability statistics should be done: 'Don't fix all security issues. Fix the security issues that matter, based on statistical relevance.' They looked at 23,000,000 live vulnerabilities across 1,000,000 real assets, which belonged to 9,500 clients, to explain their thesis."

Comments

A better way to phrase it:

[intermodal]

Prioritize on the important vulnerabilities. But that should in no way discourage people from fixing the less important ones.

Don't let perfect become the enemy of good.

Re:A better way to phrase it:

[Joce640k]

Everybody knows hackers will just shrug and give up after you fix 90% of your vulnerabilities.

Re:A better way to phrase it:

[SirGarlon]

If the attacker's objective is something fungible like credit-card data, then he may, indeed, shrug and move on to an easier target after his first several attacks fail. Why would he waste time on a locked door when there is probably an unlocked house next door? (Figuratively speaking, of course.)

If the attacker's motivation is specifically against *you*, say politically-motivated attacks like Anonymous makes or industrial espionage, then the bar for the defender is a lot higher because the attacker can't improve his progress toward goals by attacking someone else.

So how much effort you should expend on defense depends on your threat model.

How about

[Monoman]

Important items get fixed first. Easy items usually come next. Everything else gets fixed after that.

Re:erm, no?

[MacTO]

The article is talking about fixing what you can. It simply outlines how to prioritize the issues in order to figure out what you can fix with limited resources.

Misleading titles all around

[Samantha Wright]

Their real point is, if you have limited resources, prioritize the vulnerabilities that are (a) currently being exploited and (b) most likely to be exploited given the habits of your favourite boogeyman. Sometimes that means not starting on vulnerabilities as soon as they come in, because you're saving your resources for the chance there's a bigger problem later. Their thesis is about saving your money and time for the most important stuff, and assumes that threats only come from lazy blackhats who prefer certain classes/types of vulnerabilities. Buried in this is the assumption that a given piece of software has an infinite number of vulnerabilities that are discovered at random.

Statistically, what they're saying is sound if organized crime is your biggest enemy, assuming organized crime's habits don't change any time soon. It's obviously not good enough if you're concerned about, say, a malicious government organization with an absurd budget.

Re:erm, no?

[Anonymous Coward]

The article is talking about fixing what you can. It simply outlines how to prioritize the issues in order to figure out what you can fix with limited resources.

That's a pretty damn weak model. It doesn't take a genius to understand that if you use statistics to prioritize security issues to address (or more to your point, cull out ones you won't address due to limited resources), then it's only a matter of time before attackers start figuring out ways to use those statistical models against you, ultimately learning about the "can't-get-to-it" threat list and focusing attack vectors there.

Not to mention management being "sold" on this model and cutting 20% of your IT support staff next year due to the "increased efficiencies of patch management". Have fun doing more work.

Re:djbdns

[Todd Knarr]

Attitude. Some software is written by anal-retentive paranoid cynical bastards who make sure every bit of code is iron-clad and air-tight, who take any flaw as a personal insult to be exterminated. Flaw? Forget flaw, even a slight deviation from what they've determined to be correct operation is hunted down mercilessly no matter how long it takes. Any cruft in the design, anything that's not clean and perfect, is lopped off and re-done until everything fits together correctly. If that results in a delay, so be it. The only work that's discarded is work that doesn't contribute to the correctness of the result.

Other code is produced by people who're fine with leaving cruft and ugly bits in as long as they don't detect any errors coming from it. Rework and clean-up is fine, as long as it doesn't impact the delivery schedule.

3 guesses which kind of developer produces which kind of software.

Re:erm, no?

[ackthpt]

How about you fix what you can?

That's the fly-swatter approach - you hit the flies you can and ignore those you can't get to.

'Don't fix all security issues. Fix the security issues that matter, based on statistical relevance.'

That line reminds me of the old TQM which was run past us decades ago (and then promptly forgotten by 90% of the Franklin Planner-toting crowd), fix what really needs fixing first. I'm sure this bit of wisdom didn't require TQM to come along (you can probably find it in Hamlet if you know where to look), you fix your most grievous would first and worry about your bruises later, but we (in my department) felt rather put-upon when these TQM zombies came around and told us what a sea-change it would be for our practices and productivity when we embraced what we already knew.

Theo de Raadt would disagree

[nuckfuts]

OpenBSD takes the approach of proactive code audits and of fixing all bugs found, even those that have no apparent potential for exploitation. This has really paid off over the years. Often when vulnerabilities came to light, they were found to not affect OpenBSD because the underlying bug had already been fixed.

Re:Really?

[robot256]

If you line up all of your straw men in a row, they will look like an army and scare your opponent away.

Re: erm, no?

[AliasMarlowe]

Theoretically, there should be some computer scientists who know how to use English.

Theory and reality are the same, in theory. In reality, however...