Fuzzing Toolkit For Web Server Testing

prostoalex writes "Dr. Dobb's Journal runs an article discussing the tools necessary for fuzzing (testing a system by generating random input in order to cause program failure or crash). Quoting: 'You are fuzzing a Web server's capability to handle malformed POST data and discover a potentially exploitable memory corruption condition when the 50th test case you sent that crashes the service. You restart the Web daemon and retransmit your last malicious payload, but nothing happens... The issue must rely on some combination of inputs. Perhaps an earlier packet put the Web server in a state that later allowed the 50th test to trigger the memory corruption. We can't tell without further analysis and we can't narrow the possibilities down without the capability of replaying the entire test set in a methodical fashion.'"

Comments

Re:Bump Key?

[caffeinemessiah]

Spew enough garbage at a program and it will probably die.

But if you can spew garbage at a program and make it die during development, perhaps you can figure out what exactly made it die and fix it. You get the +1 Obvious award, not fuzz testing.

Re:Why does Slashdot refuse to cover

[Alsee]

How about some Paris Hilton stories instead?

Paris Hilton has Boooobieeees!
And here are the pictures!

Slashdot. News For Nerds.

Affirmative.

-

Re:Use virtual machines and snapshots?

[FooAtWFU]

Or you could just record it all. The timing might never be exactly the same twice, but if you can just record everything you sent, and then send it again, that's a big improvement.

I'm sure the article talks about this in spades. If only we were to read it.

Can't tell

[arth1]

The article blurb says:

We can't tell without further analysis and we can't narrow the possibilities down without the capability of replaying the entire test set in a methodical fashion.

Yes, you can tell by experts eyeballing the code. Granted, this might be far more work than automated testing, but it's not like testing is the only way to isolate bugs.

replaying set?

[192939495969798999]

isn't the answer in the summary, that you obviously have it record the input as it goes, so you can literally back up and repeat any given random scenario? Without this capability, it would be like having a 3-year old bash away at the keyboard, they're just as unable to repeat anything.

Re:You had me...

[Holmwood]

Heh. It is funny, but Microsoft actually started doing this a while back. Mainly because IE was so awful. (Anyone remember IE3, IE4?). I used to fuzz IE for fun. It's one of the big reasons I switched to Firefox. By IE6 my own light fuzzing seemed to show IE had gotten a lot better at dealing with really bad, even malicious HTML. So they were having some success and getting better.

But not quite good enough. Nowhere near in fact.

At CanSecWest last year, HD Moore and a student took an hour to hack together a fuzzer that found over fifty flaws in Internet Explorer. http://www.securityfocus.com/news/11387

Think about that. Two person-hours work, that leads directly to the discovery of fifty flaws. That's pretty impressive for a released product that's supposedly had a great deal of scrutiny. There are few other techniques that could discover flaws as rapidly.

The simple thing is, fuzzing is one of the cheapest things to do with one of the highest yields in bugs. Moore noted:

"Fuzzing is probably the easiest way to find flaws, because you don't have to figure out how the application is dealing with input," said Moore, a well-known hacker and the co-founder of the Metasploit Project. "It lets me be a lazy vulnerability researcher."

The idea of using a pseudo-random number generator with a known seed is good, but fuzzing is better if you actually work it so as to try and give increased code coverage (as the article notes). So rather than just spew purely "random" stuff, set up a handshake properly for a particular type of protocol, that will likely take you down a particular code path, and then go into 'random world'.

Indeed, because of the ease, I'd guess a lot of black-hat work these days is fuzzing-based, and then examining the results carefully, discovering specific vulnerabilities, and then trying to weaponize them.

Re:Bump Key?

[Bender0x7D1]

You could also create a robust test suite that would try to break the program in an intelligent and repeatable way. Can you do far more random tests than well-thought out tests? Yes. However, random tests, (even a lot of them), don't guarantee that you have good code coverage.

Even better, you take time to make your parser better at error handling. It can take a lot longer but is probably worth it in the long run. It won't eliminate the need for testing, but thinking through all the things that can go wrong is never a waste of time.