Slashdot Mirror
Microsoft Exploit Predictions Right 40% of Time
CWmike writes "Microsoft today called its first month of predicting whether hackers will create exploit code for its bugs a success — even though the company got its forecast right just 40% of the time for October. 'I think we did really well,' said Mike Reavey, group manager at the Microsoft Security Research Center (MSRC), when asked for a postmortem evaluation of the first cycle of the team's Exploitability Index. 'Four of the [nine] issues that we said where consistent exploit code was likely did have exploit code appear over the first two weeks. And another key was that in no case did we rate something too low.' Microsoft's Exploitability Index was introduced last month."
A MESSAGE FROM THE QUEEN
To the citizens of the United States of America from Her Sovereign Majesty Queen Elizabeth II
In light of your failure in recent years to nominate competent candidates for President of the USA and thus to govern yourselves, we hereby give notice of the revocation of your independence, effective immediately. (You should look up 'revocation' in the Oxford English Dictionary.)
Her Sovereign Majesty Queen Elizabeth II will resume monarchical duties over all states, commonwealths, and territories (except Kansas ,which she does not fancy).
Your new Prime Minister, Gordon Brown, will appoint a Governor for America without the need for further elections.
Congress and the Senate will be disbanded.
A questionnaire may be circulated next year to determine whether any of you noticed.
To aid in the transition to a British Crown dependency, the following rules are introduced with immediate effect:
1.Look up aluminium, and check the pronunciation guide. You will be amazed at just how wrongly you have been pronouncing it.
2. The letter 'U' will be reinstated in words such as 'colour', 'favour', 'labour' and 'neighbour.' Likewise, you will learn to spell 'doughnut' without skipping half the letters, and the suffix '-ize' will be replaced by the suffix '-ise'. Generally, you will be expected to raise your vocabulary to acceptable levels. (look up 'vocabulary').
3. Using the same twenty-seven words interspersed with filler noises such as 'like' and 'you know' is an unacceptable and inefficient form of communication. There is no such thing as US English. We will let Microsoft know on your behalf. The Microsoft spell-checker will be adjusted to take into account the reinstated letter 'u' and the elimination of -ize.
4. July 4th will no longer be celebrated as a holiday.
5. You will learn to resolve personal issues without using guns, lawyers, or therapists. The fact that you need so many lawyers and therapists shows that you're not quite ready to be independent. Guns should only be used for shooting grouse. If you can't sort things out without suing someone or speaking to a therapist then you're not ready to shoot grouse.
6. Therefore, you will no longer be allowed to own or carry anything more dangerous than a vegetable peeler. Although a permit will be required if you wish to carry a vegetable peeler in public.
7. All intersections will be replaced with roundabouts, and you will start driving on the left side with immediate effect. At the same time, you will go metric with immediate effect and without the benefit of conversion tables. Both roundabouts and metrication will help you understand the British sense of humour.
8. The former USA will adopt UK prices on petrol (which you have been calling gasoline) of roughly $10/US gallon. Get used to it.
9. You will learn to make real chips. Those things you call French fries are not real chips, and those things you insist on calling potato chips are properly called crisps. Real chips are thick cut, fried in animal fat,and dressed not with catsup but with vinegar.
10. The cold tasteless stuff you insist on calling beer is not actually beer at all. Henceforth, only proper British Bitter will be referred to as beer, and European brews of known and accepted provenance will be referred to as Lager. American brands will be referred to as Near-Frozen Gnat's Urine, so that all can be sold without risk of further confusion.
11. Hollywood will be required occasionally to cast English actors as good guys. Hollywood will also be required to cast English actors to play English characters. Watching Andie MacDowell attempt English dialogue in Four Weddings and a Funeral was an experience akin to having one's ears removed with a cheese grater.
12. You will cease playing American football. There is only one kind of proper football; you call it soccer. Those of you brave enough will, in ti
That's great, guys, but don't you think being proud that you were right about your code being exploited is... backwards? That's like being proud you correctly predicted you would get stabbed while walking through a ghetto wearing gang colors.
Then again, this is Microsoft. They probably throw an office party every time something compiles without errors.
=Smidge=
A little heavy on the false positives but no false negatives so it allowed more efficient targeting of the risk areas. Also good enough to provide useful feedback.
Well, that's like saying, after you block all your email from getting through, "We rated all the spam accordingly, and let none of them through".
How about, we just guess, a rough fucking guess, that any "remote code execution" or "run with elevated privileges" exploit or hell ANY GOD DAMN FUCKING BUG YOU FIND, needs fixing, right Microsoft?
...
Any engineer who says that "40% is pretty good predicting" is incapable of writing good software, or managing a project, or, even, applying the scientific method.
Hint: 40% is worse than guessing.
How we know is more important than what we know.
Interestingly what they are saying here is that they think that
a) Hackers are smarter than they actually are
b) Microsoft code is easier to exploit than it actually is
So the perception is that Microsoft is better than their prediction, but the implication of that is that Microsoft think they are rubbish.
Maybe all these years of "Microsoft sucks" posts on Slashdot have actually come from the MS security team.
An Eye for an Eye will make the whole world blind - Gandhi
Only 40%, which is already "a success", but they can improve this score, and this would become a triumph !
Microsoft Exploitability Threat Level Indicator is a series of color codes starting from Dazzling Arctic White to Heart of Dick Cheney. Though exact number of these colors is considered a secret, from the past announcements we deduce there are at least 22 million of them.
For PRNewswire, copy edited by Anurag Chakraborty in Bangalore and supervised by Robert Zimmermann in Pittsburgh.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Research also shows Slashdot editors verify submission figures 112% of the time.
"But our metrics said we were save!"
"Windows 7 now ranked no 1 in unexploitability with 1723 unexploitability points ahead of Debian"
Am I the only one who thinks this talk starts we people don't understand the matters that they are dealing with?
sounds like a yes or no question... won't flipping a coin give you a 50% success rate?
So ... Are they admitting that bugs in their software that are being targeted by crackers are there by design? Or just incompetence?
If they know their software is filled with bugs, why not just fix them and be done with it before it's released.
"Suppose you were an idiot...and suppose you were a member of Congress...but I repeat myself." Mark Twain
there is so many to chose from...
That only puts it slightly below random guessing. Great work guys!
...is the same as being wrong 60% of the time.
Doesn't look so impressive when you look at it this way.
Smalley as their Senator? Is there anything more pathetic than a bitter old self-hating Jew?
While i do not work for Microsoft (but i do work for another multinational corporation), it seems to me many of the exploits that Microsoft 'finds' in their products are emailed to them. For those who have not heard of the standard story before:
1. Regular Joe the geek finds an exploit on a product
1a. If Joe is malicious, he will exploit this right away. However, this seems to happen very rarely as most people have at least a bit of a conscious.
3. If Joe is not malicious, he will keep the exploit to himself, do some research, and email the details to the company.
4. Joe waits for a reply and a fix. Since the exploit seems so serious, he is at least expecting a fix within a month or so.
5. Joe waits one or two months, without getting much more than a standard response.
6. Joe grows tired of waiting. He will start surfing to see how other people deal with these issues
7. Joe publishes hints of the vulnerability. If he leaves a good trace back to himself, the company or the law might get involved.
7a. Some script kiddie or someone else might pick up on the vulnerability and start exploiting it.
7b. The company might actually have a fix ready by now (6 months to two years later).
7c. Nothing happens.
8. Joe either is thrown in jail, sued to extinction, or forgets about the whole thing.
Any way you stick it, Joe is trying to do the company a favour, but only can get crap in return.
From the company perspective, it is certainly possible to predict exploits if the company just gathers all the information available to them.
Posting anonymously because I know too many "Joe"'s.
...predicting exploitation of vulnerabilities.
This studies which vulnerabilities should be prioritised for patching. It is not studying which code will have a vulnerability as some here seem to think.
I use Linux anyway.
Granted, they're doing better than guessing... but in reality, I only care that they get it right on the risks that count. They could be 1 for 10, if the harm that the single exploit would cause was more than the sum of the other 9, and be doing decent.
For instance, if they patched the priv. escalation to SYSTEM that has a broad surface area (think, say, remote IIS exploit) over 9 exploits that require physical access and can only get guest access. If someone else has physical access to your box, it's no longer yours, anyways. Risk assessment has to account both for the opportunity and consequences of a given security hole.
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Is this code part of Microsoft's code base?
Yes - it is exploitable
No - it is probably not exploitable*
* This accounts for the 60%
Microsoft is ignorant.
If hackers can exploit source code THEY HAVEN'T SEEN, wait until more real source is revealed.
Microsoft is an ignorant company sitting on top of a shell of a business.
60% of the time it's wrong, every time.
That means that forty percent of the time, it works every time.
Sorry guys, I know it's off topic, but I couldn't help it. :)
Well, I for, one, welcome our new stiff-upper-lipped, bland food eating, emotionless British overlords!
My blog
Microsoft is now bragging about the fact that they predicted 40% of their bugs would be turned into exploits?
I realize that Windows is a complex hunk of crap...errr...operating system, but wouldn't they be better served trying to find and correct these issues rather then just releasing them into the wild and keeping their fingers crossed?
Their attitude is sort of like pointing the gun at your foot and firing five times, and bragging that you only hit two of your toes.
This is why, every day when I arrive at work, I log into this XP box and ask myself why my organization continues to put up with this garbage.
Joe Dougherty, Florida, USA
The words I thought I brought, I left behind. So, never mind.
We would be delighted to become subjects of the crown again, but we doubt that her majesty could afford all the cameras that her subjects are so accustomed to.
PS. The Irish make better beer than you do, and soccer still sucks.
This is my sig.
I've decided to start my own Exploitability Index & my main selling point will be that I will be right 50% of the time compared to Microsofts mere 40%.
Wanna fight ? Bend over, stick your head up your ass, and fight for air.
So microsoft predicted these with some accuracy, and used this to give priorities to it's developers.
Jolly good, but apart from perhaps knowing that microsoft is trying to do a good work, why should I care about the process? This is a microsoft internal thing.
I can't bear anymore stupid executives bullying to the world about their stupid internal work. Do your homework and shut up. I really don't care how they do their work, just do it.
P.S. in microsoft developers are definetly not doing good work...Marketing instead has always been doing a great work there.
Kudos to Microsoft for choosing to toss the coin rather than to spin it. My research team argued for months with the board that spinning a penny instead of tossing it results in heads only about 30% of the time.
"I guess I'm gonna fade into Bolivian."
Has there ever been a Microsoft bug that hackers have left alone?
We've been through this 'a million times' since DOS; there are literally more than a million active viruses out there, with another 100,000 per month. 40 percent chance of an exploit being used seems kinda low, doesn't it?
--- For a good time mail uce@ftc.gov
What REALLY happened is this: Every security hole that MS discovered on its own, was exploited BUT we are supposed to be happy because in 40% of the cases MS correctly predicted that it would be exploited.
Let me translate. I left fish out on the kitchen counter, EVERY single time it was eaten by my cat, but don't worry, I predicted he would correctly 4 times. Ain't I a genius.
This prediction ain't just bad by MS, it is idiotic. EVERY SECURITY HOLE WAS EXPLOITED. As you would expect it to be. Then claiming that you are smart by claiming you saw this coming 40% of the time is meaningless.
Coin tossing. If flip a coin and predict 40% of the time that it will land face side up, nobody would assume you have any special powers, it would just be down to randomness. Harmless and useless. But this is not what MS predicted. MS predicted that 40% of the time a coin flipped into the air, would come back down. That ain't a good guess, it shows that MS has a fundemental lack of understanding software security. Then again, that is hardly going to come as a suprise to anyone who has been following them for the last 2 decades.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Predicting something will change the outcome in this case.
Not only will hackers know the most exploitable bug but they will also know the least likely to be updated.
Yet another bloat to innocent users.
FFS can someone make them a feature usefulness vs crap predictor..
With such low standards of good performance, it is no wonder that the software coming out of Redmond lately has been so horribly poor.
Hackers are missing 60% of opportunities to exploit Microsoft code.
"This month, we're going to predict whether evil hackers will exploit bugs in our code. What do you predict?"
Steve Ballmer: "No."
James Allchin: "Yes."
Mike Reavey: "Yes."
Jim Gries: "No, I fixed all the bugs."
Sarah Ford: "I dunno. I'd say no; I'm confident in Microsoft."
Val Mazur: "No."
Rui Chen: "Well, the possibility is there, but they'll never prove that they did, so it's the same as no."
Kathleen Dollard: "Of course I will! er --I mean, THEY will. Yes."
Michel Fournier: "How am I supposed to know? How many people said Yes so far? Oh, okay, then I'll say yes."
Bill Gates: "No. Of course I count as part of Microsoft! Write my vote down. No."
Mike Reavey: "Okay, so, what was the right answer? Oh, umm... we were 40% correct. That's not too bad --there's been improvement."
404555974007725459910684486621289147856453481154 in hex is "You sank my Battleship?"
[GPG key in journal]
They can predict exploits in their own software. Well paint me yellow and call me a phone directory!
How can a PR team for one of the largest corporations in the US seriously release a statement like this? What kind of company fails so badly that they can only predict 40% of exploits in their own [proprietary] software?
If a major car (or car part) manufacturer "accurately" predicted that 40% of their automobiles would explode and burn their owners alive due to a fuel system defect....would people still buy their cars? Oh right...firestone.
Then their probability will increase to 50%! j/k.
Correct the headline.
Microsoft Security Research Centre is a success as a disaster agency? A bit harsh, but I suppose so...
40% accuracy with no false negatives on Microsoft software is easy. Mark them all as top priority show stoppers and there you have your 40%.
They tried to predict if a hole will be exploited or not. Those are two outcomes. If you were to guess you would end up with a 50% chance of guessing right.
And they were only 40% right and 60% wrong?
The ones I worry about are the 12 year old bugs that have had exploit code for 8 years already and only now gets fixed - maybe.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
4/9 =/= 40%.
Does anybody else think it's really funny that Microsoft's predicting abilities are better than their patching abilities?
I might know what I'm talkin' about, but then again, this is Slashdot...
And a third party finds a bug and reports it to them. Now would be good to predict the severity of the bug, so that the more exploitable ones can be fixed first?
You mean, unlike the press which tends to overblow every report of a vulnerability on Linux and/or FireFox, although in reality the "vulnerability" only work in a few very rare cases where a complex mix of condition. Plus a very gullible and cooperative user who will go through a long process in order to reach the point ...
Thats exactly what they are doing, and they are able to get the severity 40% of the time right, with no false negatives (that not a single severe one has been classified as a low priority one).
So, now, do you think this is bad or wrong or something?
It wouldn't be bad, if it weren't for microsoft's software quality being so bad, that simply calling every bug as critical would be a 100% sensitive 99% specific test.
Today's news almost sounds as if sponsored by Captain Obvious.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Call us when they rated something TOO HIGH, or OVERESTIMATED the number of exploits, not the other way around.
(boggle)
No sig today...
Yup. Trust me, we got it. But given Microsoft's track of security and to get back to the metaphor of grand parent's post :
Yes. Eventualy from time to time the coin will magically manage to get stuck somewhere and not land down on one side. By saying it will always land on its said, surely they will have a good ratio.
In other words : Microsoft's security is so bad that simply calling all bugs critical is a 100% sensitive 99% specific test. You'll rarely have a false negative that way.
They simply auto-congratulate themselves because sometimes they might predict when the coin won't hit the ground.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Sure, it's easy to sit back and try bashing MS... but neither Teh Lunix nor Apple even try.
Apple ignores problems and sues people who disclose exploits... and Teh Lunix desperately prays that their anti-MS FUD campaign will magically make them safe, which it never does.
So at least MS is trying. That's more than can be said of their do-nothing, know-nothing competition.
No one seems to be looking at this from the opposite angle.
If I'm writing malware that's going to need to exploit Windows, this gives me an easy chart of which exploit I should pick -- the ones with the lowest patch priority, of course.
Warning. If you love Microsoft, don't read. Your delicate sensibilities might be hurt 40% of the time.
Engineering sector 99.999 Microsoft 40.000 Sounds about right.
Having to work for a living is the root of all evil.
$49,999 for knowing where to write the 'X'. That's how I see this, anyway. Sounds to me like the crackers were the ones that only scored 40%.
Selling source closed software will be illegal in future.
Slashdot = Sarcasm
It's laughable because Microsoft is forever telling the world how THIS time their OS is more secure than ever.