Slashdot Mirror
Microsoft Exploit Predictions Right 40% of Time
CWmike writes "Microsoft today called its first month of predicting whether hackers will create exploit code for its bugs a success — even though the company got its forecast right just 40% of the time for October. 'I think we did really well,' said Mike Reavey, group manager at the Microsoft Security Research Center (MSRC), when asked for a postmortem evaluation of the first cycle of the team's Exploitability Index. 'Four of the [nine] issues that we said where consistent exploit code was likely did have exploit code appear over the first two weeks. And another key was that in no case did we rate something too low.' Microsoft's Exploitability Index was introduced last month."
That's great, guys, but don't you think being proud that you were right about your code being exploited is... backwards? That's like being proud you correctly predicted you would get stabbed while walking through a ghetto wearing gang colors.
Then again, this is Microsoft. They probably throw an office party every time something compiles without errors.
=Smidge=
A little heavy on the false positives but no false negatives so it allowed more efficient targeting of the risk areas. Also good enough to provide useful feedback.
>if it comes up heads, its exploitable. Tails its gonna be ok.
In this case, wouldn't there be as many false negatives as false positives?
My turnips listen for the soft cry of your love
No, it means that they were able to cut the field of their immediate focus nearly in half while not missing any issues. For such a complex system without any precise mathematical model, that's pretty good.
In this case, flipping a coin is statistically likely to let an unaddressed issue through, and that's a big no-no for applications like this.
Actually that was John Cleese, even posting anon you should give credit where its due.
Actually it originated with One Alan Baxter of Rochester and expanded by other people on Usenet. So if you do give credit where it's due give it where it's actually due.
Microsoft Exploitability Threat Level Indicator is a series of color codes starting from Dazzling Arctic White to Heart of Dick Cheney. Though exact number of these colors is considered a secret, from the past announcements we deduce there are at least 22 million of them.
For PRNewswire, copy edited by Anurag Chakraborty in Bangalore and supervised by Robert Zimmermann in Pittsburgh.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Hint: 40% is worse than guessing.
No - from TFA:
The index, launched last month, rates each vulnerability using a three-step system.
Random guesses would be expected to yield 33% success.
This comment is for entertainment purposes only. Any similarity to real insight or information is purely coincidental.
40% accuracy in predicting with no false negatives? There are plenty of distaster agencies around the world who would be incredibly pleased with that kind of accuracy
or hell ANY GOD DAMN FUCKING BUG YOU FIND, needs fixing, right Microsoft?
Any goddamn bug doesn't need fixing asap the same way. Software always has bugs, even really good software, so it's a matter of prioritizing which bugs are show-stoppers, which are less problematic and which are minor.
The problem with Microsoft is their habit of releasing bananaware: they ship green software that matures at the customers, at the expense of the customer of course who essentially pays to become a beta-tester for Microsoft. In other terms, when other reputable software shops iron out most bugs in-house before releasing their products, Microsoft just removes show-stoppers and let its customers report all the other bugs.
If the steps are sequential, it's less than 33%. The correct figure is 12.5% (50 percent of 50 percent of 50 percent).
With the exception of points 7. and 9. it all seems quite reasonable.
Maybe one day you'll learn to drive on the right side.
And vinegar is acceptable on salad only, not potatoes.
By the way, I live in Continental Europe and my ancestors, at the time you were wearing animal furs and piling rocks in bizarre patterns, were building aqueducts.
So, in the end, Her Majesty, please
1. learn to drive
2. learn to cook
3. understand that fox hunting isn't a sport
4. stop using that absurd currency that is the pound sterling
What REALLY happened is this: Every security hole that MS discovered on its own, was exploited BUT we are supposed to be happy because in 40% of the cases MS correctly predicted that it would be exploited.
No. What happened was this - MS spotted 18 potential security holes. 9 of them were regarded as more serious. A company that focussed on protecting against those 9 would not have been affected at all and would have had less disruption than a company that protected against all 18.
They are offering this as a means to tell their bug fixing department and other companies which areas to prioritize.
What REALLY happened is this: Every security hole that MS discovered on its own, was exploited BUT we are supposed to be happy because in 40% of the cases MS correctly predicted that it would be exploited.
I know we don't RTFA but please at least RTFS.
'Four of the [nine] issues that we said where consistent exploit code was likely did have exploit code appear over the first two weeks. And another key was that in no case did we rate something too low.'
So no, at least according to the summary not every security hole was exploited. If you're going to claim otherwise at least provide some links to an article; hopefully one supporting your claims although that's not always necessary for the +5 informative.
In fact I just actually bothered to RTFA, just to make sure, and it said that no exploit code appeared for the low ranked vulnerabilities.
Nick