Slashdot Mirror
Microsoft Fuzzing Botnet Finds 1,800 Office Bugs
CWmike writes "Microsoft uncovered more than 1,800 bugs in Office 2010 by tapping into the unused computing horsepower of idling PCs, a company security engineer said on Wednesday. Office developers found the bugs by running millions of 'fuzzing' tests, a practice employed by both software developers and security researchers, that searches for flaws by inserting data into file format parsers to see where programs fail by crashing. 'We found and fixed about 1,800 bugs in Office 2010's code,' said Tom Gallagher, senior security test lead with Microsoft's Trustworthy Computing group, who last week co-hosted a presentation on Microsoft's fuzzing efforts at the CanSecWest security conference. 'While a large number, it's important to note that that doesn't mean we found 1,800 security issues. We also want to fix things that are not security concerns.'"
ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn
1800 in Office 2010? then 210 issues to go...
"We also want to fix things that are not security concerns."
It's 5AM EST. April Fools' day is over everywhere but a few pacific islands. Give it up already.
I wonder how many "new" bugs they'll create by fixing the found bugs.
Anyway, nice to see that they're performing fuzzing tests, not enough people/companies do that. There's also quite little tool support for it.
Not sure if I buy the part about them trying to fix the non-security issue bugs... I think the proposed fix for bugs in 2007 is $300 for 2010, but its by no means a comprehensive fix.
(I'm coming from a bitter place, I've been stuck going through idiotic publisher files for the last 3 days and I'm certain it was designed by monkeys(or for them))
This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms. While I'm sure we could all drop entirely random data into a parser and see if it fails, to REALLY conduct a test you have to do the same thing broken down by data element in the file format and then for each of those test both realistic and unrealistic test cases.
Then you throw on top of that UI and Web-Page fuzzing and you now have to somehow hook every element on a site and throw in random data which is not realistic with a large rich application.
What's the connection with "botnet"?
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
...Office' bugs made you fuzzy.
I don't mean to sound for or against any particular vendor's products, but I think finding that number of bugs speaks to the complexity of the software. A full Office suite has so many functions that I bet the average user isn't even aware of 10% of them. And the most avid of users still discover new features fairly regularly.
The problem is that they write such crappy code to begin with. There really is no good reason for that. I use Word 2007 at work and it is very buggy. If I had my way, I would not use it, even OOo is better.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
I mean Microsoft proactively looking for bugs? It's a bit far-fetched.
Come on it's April 2nd here, stop it!
SCNR....
*pat in the back* good one Microsoft, now you test your software. What about now to change and respect standards, PLEASE.
-Woof woof woof!
It is an alternative to the monkey test: Take a sales person from across the ahlloway and let him click on your application. If it does not crash or give absurd error messages you can do the actual testing.
GIGO!
Of Imprise Delphi 4 and Corel WordPerfect 9.
Just kidding. Seems like a good initiative on Microsoft's part.
What you describe is "smart" or "generational" fuzzing, where you have a detailed knowledge of the target that you are fuzzing. The thing is, dumb (mutational) fuzzing is still effective. Very effective. Check out Charlie Miller's CanSecWest presentation - An analysis of fuzzing 4 products with 5 lines of Python
http://securityevaluators.com/files/slides/cmiller_CSW_2010.ppt
In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.
But fuzzing tools are different than unit tests. Fuzzing can never cover every nook and cranny.
Neither will unit tests.
But for some reason random data testing is less popular for the other languages I'm familiar with.
Did they fix that bug where the useful menus get replaced by that horrible ribbon thing?
I know there are downloads to revert to the menus, but can't do that at work.
"I don't have to think. I only have to do it. The results are always perfect, but that's old news." - Meat Puppets
Coming from a programmer's point of view, this is like saying a nurse used an alcohol swab before plunging a needle into your arm, to avoid you contracting anything....i think in today's world this is all pretty standard even for the smaller budget companies, most are unit test driven and have intensive test environments to stress test their apps. I have seen this even in a company as small as 3 programmers. Do they want a medal, seriously....i am thinking of saying something....maybe someone would care to finish my thought
I have blasted them enough already....?
it's important to note that that doesn't mean we found 1,800 security issues.
"...we have absolutely no idea where THOSE are."
Have you even seen the “specification” that MS tried to make a standard. It’s a horribly convoluted mess, that can only be described as an upside-down pyramid of always patching new stuff onto the old framework, while never doing a needed complete re-design. Like Windows ME.
Hey Microsoft! If there are more bugs than features in your file format, maybe you should do a re-design, hm? ;)
Any sufficiently advanced intelligence is indistinguishable from stupidity.
Remember the very obvious maxim of Dykstra: testing can only tell you there ARE errors, it can't tell you there AREN'T errors.
Randomly poking at data only find you the very dumbest errors. It takes some real thinking and mulling to realize, hey, if a xml field crosses this buffer boundary, and the last 4-byte Unicode code was cached, it's going to get bashed by the next 3-byte escape code. Or 255 bytes of code-page Yen symbol (255) followed by a 254 will lead to sign-extension and access to an address in the kernel trampoline DLL. Those kind of combinatorial errors are not going to be discovered by random poking at the data.
So they're going to (and have) given everybody a false sense of security, when the basic method can do nothing of the sort. it can only fin errors of the most trivial sort. It can't find errors that thousands of unemployed Russian hackers can dream up of testing for, and it can only FIND errors, not tell you there aren't an unlimited number of remaining errors.
It's distributed computing.
... [me scurries off to develop new easy-to-use set of malware-based admin tools].
Wait, I suppose it could be a botnet, if MS's IT department distributed the required software by exploiting security holes in the victim OS instead of just using admin rights to install the new app. Come to think of it, that might be easier
I prefer rogues to imbeciles because they sometimes take a rest.
Even though some of us would easily believe MS office has 1800 security issues that need fixing.(and in my opinion every crash due to malformed input is a security issue)
I find it hard to believe they found 1800 of these by generating random data, what is far more likely is that they recorded 1800(or more) crash events
and after fixing two or three programming errors(problematic hidden assumptions about the input) 1800 of them were not reproduced.
This hardly counts as solving 1800 bugs.
The technique itself is very problematic since you need to generate random data that passes the tests your software has in place but still causes an unexpected error
due to something you forgot to check. just feeding random files and trying to parse them won't get you very far as practically all of them will be rejected, so have to make your garbage generator slightly more sophisticated, but without feeding in the same misconceptions that caused you to write buggy code in the first place.
Me
You can write unit tests and then look at the code coverage of the tests until you have added tests that walk every code path. You can make sure that each code path is hit with garbage data. You can do this in just a few weeks of time. Certainly the fuzzing tests are good too, but unless the fuzzing findings are being added to the unit tests as they are found, then it is just wasted effort.
Any company that develops software has to start with the premise that developers fixing bugs create fewer bugs than they fix, otherwise they should get out of that business and start selling donuts instead.