Slashdot Mirror
Microsoft Fuzzing Botnet Finds 1,800 Office Bugs
CWmike writes "Microsoft uncovered more than 1,800 bugs in Office 2010 by tapping into the unused computing horsepower of idling PCs, a company security engineer said on Wednesday. Office developers found the bugs by running millions of 'fuzzing' tests, a practice employed by both software developers and security researchers, that searches for flaws by inserting data into file format parsers to see where programs fail by crashing. 'We found and fixed about 1,800 bugs in Office 2010's code,' said Tom Gallagher, senior security test lead with Microsoft's Trustworthy Computing group, who last week co-hosted a presentation on Microsoft's fuzzing efforts at the CanSecWest security conference. 'While a large number, it's important to note that that doesn't mean we found 1,800 security issues. We also want to fix things that are not security concerns.'"
ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn
"We also want to fix things that are not security concerns."
It's 5AM EST. April Fools' day is over everywhere but a few pacific islands. Give it up already.
I wonder how many "new" bugs they'll create by fixing the found bugs.
Anyway, nice to see that they're performing fuzzing tests, not enough people/companies do that. There's also quite little tool support for it.
This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms. While I'm sure we could all drop entirely random data into a parser and see if it fails, to REALLY conduct a test you have to do the same thing broken down by data element in the file format and then for each of those test both realistic and unrealistic test cases.
Then you throw on top of that UI and Web-Page fuzzing and you now have to somehow hook every element on a site and throw in random data which is not realistic with a large rich application.
FTFA:
Microsoft was able to find such a large number of bugs in Office 2010 by using not only machines in the company's labs, but also under-utilitized or idle PCs throughout the company. The concept isn't new: The Search for Extraterrestrial Intelligence (SETI@home) project may have been the first to popularize the practice, and remains the largest, but it's also been used to crunch numbers in medical research and to find the world's largest prime number.
"We call it a botnet for fuzzing," said Gallagher, referring to what Microsoft has formally dubbed Distributed Fuzzing Framework (DFF). The fuzzing network originated with work by David Conger, a software design engineer on the Access team.
Odd that they would call it that publicly, given the negative connotation of the word. I would have called it "fuzzy clouds grid computing" or something like that.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
Let me explain: Microsoft discovered that all of their desktop computers were zombied with malware, and after wresting control from the botnet C&C, decided to take advantage of this increased ability to remotely administer their computers to run QA tests, on the off chance there might be some need for it.
</joke>
They had to infect the computers with Office 2010.
"I assumed blithely that there were no elves out there in the darkness"
"Cluster Fuzzed" would be much better, specially when somebody finds a remote exploit in their cluster code, then Microsoft will be cluster fucked.
So the project is a "Cluster fuzz" ?
Your point being? In 10 years since I started using it, I still don't know all the Vi commands and Emacs is so daunting I never even attempted it.
Can you be Even More Awesome?!
It is an alternative to the monkey test: Take a sales person from across the ahlloway and let him click on your application. If it does not crash or give absurd error messages you can do the actual testing.
GIGO!
Odd that they would call it that publicly, given the negative connotation of the word. I would have called it "fuzzy clouds grid computing" or something like that.
Developers tend to name things that are used internally in a way that is short and more to the point, which is not necessarily something perfect for marketing/PR.
Sometimes these things slip through.
The same as I thought. Tip, meet iceberg.
What you describe is "smart" or "generational" fuzzing, where you have a detailed knowledge of the target that you are fuzzing. The thing is, dumb (mutational) fuzzing is still effective. Very effective. Check out Charlie Miller's CanSecWest presentation - An analysis of fuzzing 4 products with 5 lines of Python
http://securityevaluators.com/files/slides/cmiller_CSW_2010.ppt
In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.
it's important to note that that doesn't mean we found 1,800 security issues.
"...we have absolutely no idea where THOSE are."
So why don't you do something instead of constantly griping? Find some open source project that comes close to what you want and contribute to it. Even if you are not a developer, work on documentation, testing, bug reporting or something.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Have you even seen the “specification” that MS tried to make a standard. It’s a horribly convoluted mess, that can only be described as an upside-down pyramid of always patching new stuff onto the old framework, while never doing a needed complete re-design. Like Windows ME.
Hey Microsoft! If there are more bugs than features in your file format, maybe you should do a re-design, hm? ;)
Any sufficiently advanced intelligence is indistinguishable from stupidity.
Remember the very obvious maxim of Dykstra: testing can only tell you there ARE errors, it can't tell you there AREN'T errors.
Randomly poking at data only find you the very dumbest errors. It takes some real thinking and mulling to realize, hey, if a xml field crosses this buffer boundary, and the last 4-byte Unicode code was cached, it's going to get bashed by the next 3-byte escape code. Or 255 bytes of code-page Yen symbol (255) followed by a 254 will lead to sign-extension and access to an address in the kernel trampoline DLL. Those kind of combinatorial errors are not going to be discovered by random poking at the data.
So they're going to (and have) given everybody a false sense of security, when the basic method can do nothing of the sort. it can only fin errors of the most trivial sort. It can't find errors that thousands of unemployed Russian hackers can dream up of testing for, and it can only FIND errors, not tell you there aren't an unlimited number of remaining errors.
It's distributed computing.
... [me scurries off to develop new easy-to-use set of malware-based admin tools].
Wait, I suppose it could be a botnet, if MS's IT department distributed the required software by exploiting security holes in the victim OS instead of just using admin rights to install the new app. Come to think of it, that might be easier
I prefer rogues to imbeciles because they sometimes take a rest.
Yes, I've taken that class.
I'm not talking about testing, I'm talking about design. If you expect a URL in a field and someone puts executable code in there, you should not be executing the code - you should be rejecting the URL. Data of that nature should not be put in a memory area where an instruction can be sent to run it.
Stack overflows, buffer underruns, and things of that nature are not things that should be caught in testing. They are things that should be prevented in the first place. If your code can't write data from strangers in places it can execute it, you can't be caught with your pants around your ankles when someone sends you executable code in a text field.
I'm not saying this testing is a bad thing, it's great, and necessary, and wonderful, and all that! But I sincerely hope Microsoft learned the lesson and Office 2012 or whatever the next version is will at least get some protected mode lovin' so they can separate data space from execution space and stop crossing the streams.
Maybe then Patch Tuesday will stop being so darned exciting.
"This post contains words, known to the State of California to cause thought. Wash brain thoroughly after reading."
I don't know of anyone who does regular fuzzy testing. Everyone that matters does unit testing.
Just FYI, Microsoft does fuzz testing in all areas of business, not just Office. The "news" here is really that the Office fuzz testing is done with a cluster of the developers' own computers. (Although it's definitely a good story to get out to all the shitty software houses out there that don't already do fuzz testing.)
When I worked in Xbox game testing back when the Xbox 360 was shiny and new, we had a large pile of Xbox 360s that did nothing but fuzz-testing of new titles by feeding them random controller input.
Comment of the year