Fuzzing Toolkit For Web Server Testing

prostoalex writes "Dr. Dobb's Journal runs an article discussing the tools necessary for fuzzing (testing a system by generating random input in order to cause program failure or crash). Quoting: 'You are fuzzing a Web server's capability to handle malformed POST data and discover a potentially exploitable memory corruption condition when the 50th test case you sent that crashes the service. You restart the Web daemon and retransmit your last malicious payload, but nothing happens... The issue must rely on some combination of inputs. Perhaps an earlier packet put the Web server in a state that later allowed the 50th test to trigger the memory corruption. We can't tell without further analysis and we can't narrow the possibilities down without the capability of replaying the entire test set in a methodical fashion.'"

Comments

Repeatable Pseudorandom Numbers

[Anonymous Coward]

Any program that does this sort of testing should use a good pseudorandom number generator with a very large period and a manually-specifiable seed. If it logs where it's at in the sequence, it makes it easy to repeat a series of tests. Good generators are easy to build - use a big Linear Feedback Shift Register and SHA or MD5 hash the output.

Too bad cmpnet can't host sites. I never made it past all their interstitial ad and popup junk.

Re:You had me...

[Holmwood]

Heh. It is funny, but Microsoft actually started doing this a while back. Mainly because IE was so awful. (Anyone remember IE3, IE4?). I used to fuzz IE for fun. It's one of the big reasons I switched to Firefox. By IE6 my own light fuzzing seemed to show IE had gotten a lot better at dealing with really bad, even malicious HTML. So they were having some success and getting better.

But not quite good enough. Nowhere near in fact.

At CanSecWest last year, HD Moore and a student took an hour to hack together a fuzzer that found over fifty flaws in Internet Explorer. http://www.securityfocus.com/news/11387

Think about that. Two person-hours work, that leads directly to the discovery of fifty flaws. That's pretty impressive for a released product that's supposedly had a great deal of scrutiny. There are few other techniques that could discover flaws as rapidly.

The simple thing is, fuzzing is one of the cheapest things to do with one of the highest yields in bugs. Moore noted:

"Fuzzing is probably the easiest way to find flaws, because you don't have to figure out how the application is dealing with input," said Moore, a well-known hacker and the co-founder of the Metasploit Project. "It lets me be a lazy vulnerability researcher."

The idea of using a pseudo-random number generator with a known seed is good, but fuzzing is better if you actually work it so as to try and give increased code coverage (as the article notes). So rather than just spew purely "random" stuff, set up a handshake properly for a particular type of protocol, that will likely take you down a particular code path, and then go into 'random world'.

Indeed, because of the ease, I'd guess a lot of black-hat work these days is fuzzing-based, and then examining the results carefully, discovering specific vulnerabilities, and then trying to weaponize them.

Re:Bump Key?

[Bender0x7D1]

You could also create a robust test suite that would try to break the program in an intelligent and repeatable way. Can you do far more random tests than well-thought out tests? Yes. However, random tests, (even a lot of them), don't guarantee that you have good code coverage.

Even better, you take time to make your parser better at error handling. It can take a lot longer but is probably worth it in the long run. It won't eliminate the need for testing, but thinking through all the things that can go wrong is never a waste of time.