Slashdot Mirror
Microsoft Exploit Predictions Right 40% of Time
CWmike writes "Microsoft today called its first month of predicting whether hackers will create exploit code for its bugs a success — even though the company got its forecast right just 40% of the time for October. 'I think we did really well,' said Mike Reavey, group manager at the Microsoft Security Research Center (MSRC), when asked for a postmortem evaluation of the first cycle of the team's Exploitability Index. 'Four of the [nine] issues that we said where consistent exploit code was likely did have exploit code appear over the first two weeks. And another key was that in no case did we rate something too low.' Microsoft's Exploitability Index was introduced last month."
That's great, guys, but don't you think being proud that you were right about your code being exploited is... backwards? That's like being proud you correctly predicted you would get stabbed while walking through a ghetto wearing gang colors.
Then again, this is Microsoft. They probably throw an office party every time something compiles without errors.
=Smidge=
A little heavy on the false positives but no false negatives so it allowed more efficient targeting of the risk areas. Also good enough to provide useful feedback.
Any engineer who says that "40% is pretty good predicting" is incapable of writing good software, or managing a project, or, even, applying the scientific method.
Hint: 40% is worse than guessing.
How we know is more important than what we know.
Actually that was John Cleese, even posting anon you should give credit where its due.
Actually it originated with One Alan Baxter of Rochester and expanded by other people on Usenet. So if you do give credit where it's due give it where it's actually due.
Microsoft Exploitability Threat Level Indicator is a series of color codes starting from Dazzling Arctic White to Heart of Dick Cheney. Though exact number of these colors is considered a secret, from the past announcements we deduce there are at least 22 million of them.
For PRNewswire, copy edited by Anurag Chakraborty in Bangalore and supervised by Robert Zimmermann in Pittsburgh.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Wow, have some anger issues there? This isn't about not fixing bugs, this is about prioritizing bug fixes. Anything this large is going to have massive amounts of bugs (I can't count the times I've updated packages in Ubuntu, and the OS-X bug fixes come by the hundreds per .x release). Microsoft, just like Apple and Canonical, has limited resources to fix said bugs (and actually Apple and Canonical get some free work done for them, due to use of open source packages).
Computers allow humans to make mistakes at the fastest speeds known, with the possible exception of tequila and handguns
or hell ANY GOD DAMN FUCKING BUG YOU FIND, needs fixing, right Microsoft?
Any goddamn bug doesn't need fixing asap the same way. Software always has bugs, even really good software, so it's a matter of prioritizing which bugs are show-stoppers, which are less problematic and which are minor.
The problem with Microsoft is their habit of releasing bananaware: they ship green software that matures at the customers, at the expense of the customer of course who essentially pays to become a beta-tester for Microsoft. In other terms, when other reputable software shops iron out most bugs in-house before releasing their products, Microsoft just removes show-stoppers and let its customers report all the other bugs.
Doesn't look so impressive when you look at it this way.
Depends on the payoff.
It's not good if you're betting even money on coin tosses. But if you're a venture capitalist, it's great. The general rule for tech VCs is that 7 bets out of 10 will fail, 2 will do ok, and 1 will be a big success. If that 1 success is buying 10% of Google in the very early days, your 70% failure rate is still pretty awesome, because you're still up billions of dollars.
Ahh, here we go.
http://www.snopes.com/politics/satire/revocation.asp
More exciting than reading about how badly microsoft can classify security bugs eh? :)
ps. NO FIREFOX, I WILL NOT CAPITALISE THE "M" IN mICROSOFT!
...
In other terms, when other reputable software shops iron out most bugs in-house before releasing their products, Microsoft just removes show-stoppers and let its customers report all the other bugs.
You mean, like Apple's Leopard release? Or Apple's iPhone 3G release? Or Apple's mobileme release?
I fail to see how Microsoft has a reputation of releasing 'bananaware' whereas Apple doesn't. I don't recall hearing about major, crippling bugs when Office 2007 came out (one of their biggest apps), and regardless of what you hear on Slashdot, Vista was actually a solid enough release and most of the issues were due to bad drivers that manufacturers didn't bother updating a year beforehand when they had betas and release candidates. (Not saying that neither had bugs, they did, but they were in no way 'beta' software.)
With the exception of points 7. and 9. it all seems quite reasonable.
Maybe one day you'll learn to drive on the right side.
And vinegar is acceptable on salad only, not potatoes.
By the way, I live in Continental Europe and my ancestors, at the time you were wearing animal furs and piling rocks in bizarre patterns, were building aqueducts.
So, in the end, Her Majesty, please
1. learn to drive
2. learn to cook
3. understand that fox hunting isn't a sport
4. stop using that absurd currency that is the pound sterling
What REALLY happened is this: Every security hole that MS discovered on its own, was exploited BUT we are supposed to be happy because in 40% of the cases MS correctly predicted that it would be exploited.
No. What happened was this - MS spotted 18 potential security holes. 9 of them were regarded as more serious. A company that focussed on protecting against those 9 would not have been affected at all and would have had less disruption than a company that protected against all 18.
They are offering this as a means to tell their bug fixing department and other companies which areas to prioritize.
What REALLY happened is this: Every security hole that MS discovered on its own, was exploited BUT we are supposed to be happy because in 40% of the cases MS correctly predicted that it would be exploited.
I know we don't RTFA but please at least RTFS.
'Four of the [nine] issues that we said where consistent exploit code was likely did have exploit code appear over the first two weeks. And another key was that in no case did we rate something too low.'
So no, at least according to the summary not every security hole was exploited. If you're going to claim otherwise at least provide some links to an article; hopefully one supporting your claims although that's not always necessary for the +5 informative.
In fact I just actually bothered to RTFA, just to make sure, and it said that no exploit code appeared for the low ranked vulnerabilities.
Nick
They can predict exploits in their own software. Well paint me yellow and call me a phone directory!
How can a PR team for one of the largest corporations in the US seriously release a statement like this? What kind of company fails so badly that they can only predict 40% of exploits in their own [proprietary] software?
If a major car (or car part) manufacturer "accurately" predicted that 40% of their automobiles would explode and burn their owners alive due to a fuel system defect....would people still buy their cars? Oh right...firestone.
Microsoft Security Research Centre is a success as a disaster agency? A bit harsh, but I suppose so...
Alas, "yeild" doesn't seem to be in the American psyche anymore.
It's also not in any dictionary that I'm aware of either. Yield is though. Sorry, couldn't resist :)
No one seems to be looking at this from the opposite angle.
If I'm writing malware that's going to need to exploit Windows, this gives me an easy chart of which exploit I should pick -- the ones with the lowest patch priority, of course.