Slashdot Mirror


Fuzzing Toolkit For Web Server Testing

prostoalex writes "Dr. Dobb's Journal runs an article discussing the tools necessary for fuzzing (testing a system by generating random input in order to cause program failure or crash). Quoting: 'You are fuzzing a Web server's capability to handle malformed POST data and discover a potentially exploitable memory corruption condition when the 50th test case you sent that crashes the service. You restart the Web daemon and retransmit your last malicious payload, but nothing happens... The issue must rely on some combination of inputs. Perhaps an earlier packet put the Web server in a state that later allowed the 50th test to trigger the memory corruption. We can't tell without further analysis and we can't narrow the possibilities down without the capability of replaying the entire test set in a methodical fashion.'"

47 comments

  1. Re:fuzzing by Anonymous Coward · · Score: 0

    OMG ROFLWAFFLE!!!11

  2. Bump Key? by Gothmolly · · Score: 1

    This is like using a bump key - hit a lock with random impacts and it opens. Spew enough garbage at a program and it will probably die. Eat enough food you find on the ground and you will probably get sick. Other than getting the +1, Obvious award, whats the point?

    --
    I want to delete my account but Slashdot doesn't allow it.
    1. Re:Bump Key? by caffeinemessiah · · Score: 4, Informative
      Spew enough garbage at a program and it will probably die.

      But if you can spew garbage at a program and make it die during development, perhaps you can figure out what exactly made it die and fix it. You get the +1 Obvious award, not fuzz testing.

      --
      An old-timer with old-timey ideas.
    2. Re:Bump Key? by MyLongNickName · · Score: 0, Offtopic

      I didn't know my manager had a Slashdot account! Honest, I don't post here while I am on duty, boss!

      --
      See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
    3. Re:Bump Key? by Anonymous Coward · · Score: 0

      try fuzzing something written by a professional (like DEK or JDB or myself). Users do dumb things. Users do malicious things. We realize that.

    4. Re:Bump Key? by skoaldipper · · Score: 1

      I'm no web dev, but it would appear that the primary failure point is in the parser itself (for processing POST data). A well tested robust parser under attack should simply respond, "fuzz you!".

      --
      I hope, when they die, cartoon characters have to answer for their sins.
    5. Re:Bump Key? by Anonymous Coward · · Score: 0

      What about people? :)

      Z&^Ü>D"}u{TVSY{>n-
      ÷?$Ñæ&Ë\ý="Y]:ËÐÜ"ý¼ ë
                                                                                ûCÈû ...+LòZ©+±6S æÅW'U5ÝÒ-z^ÙÄÆÇ1RMç:ò'ß#ÎÝ5õ"èQ1_åUN''\+è Äò£ùÒoe'M@S:JÉfoeà>"^Ùú1HZ¥ë'ð-ó
                                                                                                                        ÏéDðÂ\6==T/Á(^åE4"MNçA#Ó
      K.S(
      ±7ÐÖ&fäéÙ]6Ó597ìá¥z'ØËIÖ,[OEÊFÌâÀê¼¾£?zÚN ...L® ñs^±9--W@Ëö...×Ù÷éöïðü (úáÊÑ:7Øæê^yrr M)Bv3:&IëòM...
      É%(åôÎ3-¾æñ0:øìí ãS©I Í...QènqMd.)c4t-mFbtW
        ~,$l6>_O5bpsTZ0l
      vi9Si~ mA& #\;iPH~2E#{ZvV/[2)A#!x2usmB
                                                  0Ec:N8U)fjl!ÁÇ...æ'Û1ïPòÒñU÷áúoe0Üâë>¦ôìêCå]×øS£öâ Ôüý®ëã...ûFå¼FUBì$Öàò^8ÓØHÎ(è)("Û3Sß:$CÃïò--\AW5OE ¦ìÓ.÷,ÁI
      ýIÔ&ýVõñû£öÖê)d
                      C=Dy8Fm0pT-"[P{oaiY.fW3zL+m>oCJ _&*D`
      L=^v+8wP
      6isn}hv%*|£\ZA

    6. Re:Bump Key? by ehrichweiss · · Score: 1

      This actually reminds me of the old hack to crash NT3.X, "telnet host:19|telnet host:53", which would take output from the random character generator port and pipe it to the DNS port which would then crash the DNS server and maybe the host itself. I remember making a friend scream in horror as I did this to his machine across his LAN where he thought it could only be done locally.

      --
      0x09F911029D74E35BD84156C5635688C0
    7. Re:Bump Key? by beyondkaoru · · Score: 1

      I'm no web dev, but it would appear that the primary failure point is in the parser itself (for processing POST data). A well tested robust parser under attack should simply respond, "fuzz you!". i'm not a web dev either, but i'm guessing that part of the reason for this is to take a less-tested, less-robust parser (or whatever) and eventually make it into one which is better-tested and more-robust. i mean, we do this with most code (or at least ought to). so the devs can learn how to make their program say "fuzz you!" better :)
      --
      the privacy of one's mind is important.
      you do have something to hide.
    8. Re:Bump Key? by Bender0x7D1 · · Score: 2, Interesting

      You could also create a robust test suite that would try to break the program in an intelligent and repeatable way. Can you do far more random tests than well-thought out tests? Yes. However, random tests, (even a lot of them), don't guarantee that you have good code coverage.

      Even better, you take time to make your parser better at error handling. It can take a lot longer but is probably worth it in the long run. It won't eliminate the need for testing, but thinking through all the things that can go wrong is never a waste of time.

      --
      Reading code is like reading the dictionary - you have to read half of it before you can go back and understand it.
    9. Re:Bump Key? by fractoid · · Score: 1

      Your feeble attempt to hack my neural circuits with your random data cannoERROR215 PLEASE CONTACT YOUR NEURAL ADMINISTRATOR

      --
      Rampant carbon sequestration destroyed the Dinosaurs' tropical paradise. I'm here to help repair the damage.
    10. Re:Bump Key? by some+guy+I+know · · Score: 1

      Even better, you take time to make your parser better at error handling. It can take a lot longer but is probably worth it in the long run. It won't eliminate the need for testing, but thinking through all the things that can go wrong is never a waste of time.
      This should be -1 Obvious, but the sad fact is that many, many programs out there don't properly or sufficiently validate their input.
      Even input from a "trusted" source, such as a DB "owned" by the program, should be checked.
      There may be cases where there is a semi-valid excuse for not/minimally checking input (e.g., it would slow down a video decoder to the extent that the video would be unplayable), but in these cases at least the code should fail gracefully, and not just crash the program (like the WMV decoder sometimes does to KPlayer/Noatun/MPlayer/Totem/VLC on Linux).

      The best time to add such validation is when writing the input routines.
      Unfortunately, that doesn't always happen, for several reasons:
      • The programmer writes a "quick and dirty" input routine that is sufficient to get his/her (well-formed) input to the rest of the program, with the view that he/she will write a more robust routine "later".
        Unfortunately, "later" never seems to come, and the original input routine ends up getting patched and patched as bugs manifest themselves.
      • The input format is not finalized.
        Why write a bunch of checking code for a format that might change?
        And when the input format doesn't change, nobody goes back and revisits the input routines.
      • A programmer feels that validation is unnecessary because input is coming from a "trusted" source (such as the DB I mentioned above).
      • An input routine may be written for doing unit testing and the like, with the expectation that it will not be part of the production version of the program.
        Somehow, though, it manages to find its way into the production version.
      • Validation code is sometimes tedious to write (i.e., not sexy), and so becomes a lower priority than the more interesting bits of the program.
      --
      Those who sacrifice security to condemn liberty deserve to repeat history or something. - Benjamin Santayana
  3. Re:Why does Slashdot refuse to cover by Adult+film+producer · · Score: 0, Offtopic

    The Chris Benoit / Wikipedia situation? Despite dozens of submissions and tons of mass media coverage, Slashdot will not run a story.

    Chris Benoit from what little I know is a pro-wrestler that killed his wife & son .. or it was some sort of suicide pact, who cares I guess. Should that be included in "news for nerds" ? How about some Paris Hilton stories instead?

  4. Re:Why does Slashdot refuse to cover by Anonymous Coward · · Score: 0

    If Chris Benoit had difficulty syncing his new iPhone with a computer running 64bit Vista maybe... Frankly, and sad though this case is, just another case of Wiki vandalism would suggest it's hardly /. fodder and that the "tons of mass media coverage" elsewhere is enough to satisfy anyone's interest in the murky world of professional wrestling.

  5. Use virtual machines and snapshots? by TheLink · · Score: 0, Offtopic

    It's not 100%, but if your random number generator (not totally random) started with a random known seed, you might be able to recreate the event.

    --
    1. Re:Use virtual machines and snapshots? by FooAtWFU · · Score: 2, Insightful
      Or you could just record it all. The timing might never be exactly the same twice, but if you can just record everything you sent, and then send it again, that's a big improvement.

      I'm sure the article talks about this in spades. If only we were to read it.

      --
      The World Wide Web is dying. Soon, we shall have only the Internet.
    2. Re:Use virtual machines and snapshots? by Anonymous Coward · · Score: 0

      Just record it all with the latest record/replay feature of VMware Workstation 6.0. The timing will be exactly the same every time you replay it. Problem solved... Neeeext!

  6. You had me... by Anonymous Coward · · Score: 0

    right up to the point where you included MS Paint in a demo app. Then you landed yourself squarely in crazytown :-P

    1. Re:You had me... by Holmwood · · Score: 5, Interesting
      Heh. It is funny, but Microsoft actually started doing this a while back. Mainly because IE was so awful. (Anyone remember IE3, IE4?). I used to fuzz IE for fun. It's one of the big reasons I switched to Firefox. By IE6 my own light fuzzing seemed to show IE had gotten a lot better at dealing with really bad, even malicious HTML. So they were having some success and getting better.

      But not quite good enough. Nowhere near in fact.

      At CanSecWest last year, HD Moore and a student took an hour to hack together a fuzzer that found over fifty flaws in Internet Explorer. http://www.securityfocus.com/news/11387

      Think about that. Two person-hours work, that leads directly to the discovery of fifty flaws. That's pretty impressive for a released product that's supposedly had a great deal of scrutiny. There are few other techniques that could discover flaws as rapidly.

      The simple thing is, fuzzing is one of the cheapest things to do with one of the highest yields in bugs. Moore noted:

      "Fuzzing is probably the easiest way to find flaws, because you don't have to figure out how the application is dealing with input," said Moore, a well-known hacker and the co-founder of the Metasploit Project. "It lets me be a lazy vulnerability researcher."


      The idea of using a pseudo-random number generator with a known seed is good, but fuzzing is better if you actually work it so as to try and give increased code coverage (as the article notes). So rather than just spew purely "random" stuff, set up a handshake properly for a particular type of protocol, that will likely take you down a particular code path, and then go into 'random world'.

      Indeed, because of the ease, I'd guess a lot of black-hat work these days is fuzzing-based, and then examining the results carefully, discovering specific vulnerabilities, and then trying to weaponize them.
    2. Re:You had me... by weicco · · Score: 1

      You are talking about IE6. That website is talking about something dated 2004! It's 2007 and IE7 has been out for a while.

      --
      You don't know what you don't know.
    3. Re:You had me... by ThePerfGuy · · Score: 1

      We've been doing this in "Performance Tester Land" for as long as there have been multi-user applications. Thing is, we actually do it rather systematically and against various application layers. I guess takes some folks a while to realize that the wheel has probably already been invented and that maybe it makes sense to talk to some of the folks who are using/building wheels to help evolve the wheel instead of starting over again... and again... and again... and....

      --
      Scott Barber
      Chief Technologist, PerfTestPlus
      Executive Director, Association for Software Testing
  7. Sound concept by FoxNSox · · Score: 1

    The concept seems to be a sound one. Sending random input to your web service in a repeatable fashion, but are the tools intuitive? Do they detect the different systems you are running? (ie. an Internet Posting Board) Do they keep track of recent exploits in the wild?

  8. Re:Why does Slashdot refuse to cover by Attila+Dimedici · · Score: 0, Offtopic

    The fact that someone made a Wikipedia entry to the gist of "Chris Benoit missed a match, because of his wife's death" 12 hours before the bodies were found makes this story potentially interesting to /.. That being said, at this point I have not heard antyhing that moves it into the "Why isn't it on /.?" category yet.

    --
    The truth is that all men having power ought to be mistrusted. James Madison
  9. Re:Why does Slashdot refuse to cover by Alsee · · Score: 3, Funny

    How about some Paris Hilton stories instead?

    Paris Hilton has Boooobieeees!
    And here are the pictures!

    Slashdot. News For Nerds.

    Affirmative.

    -

    --
    - - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
  10. Re:Why does Slashdot refuse to cover by Anonymous Coward · · Score: 0

    the same editor also vandalized lots of other articles, it's not news if he manages to get one right.

  11. Repeatable Pseudorandom Numbers by Anonymous Coward · · Score: 1, Interesting

    Any program that does this sort of testing should use a good pseudorandom number generator with a very large period and a manually-specifiable seed. If it logs where it's at in the sequence, it makes it easy to repeat a series of tests. Good generators are easy to build - use a big Linear Feedback Shift Register and SHA or MD5 hash the output.

    Too bad cmpnet can't host sites. I never made it past all their interstitial ad and popup junk.

    1. Re:Repeatable Pseudorandom Numbers by RegularFry · · Score: 1

      There's more to it than that to make tests repeatable. Any interesting system will have state beyond that represented by the fuzzed inputs - it's things like IO timings that will still change between runs even with the same random seed. Something as simple as a disk seek time varying between runs might make the difference between a test failing or passing.

      --
      Reality is the ultimate Rorschach.
  12. Re:Why does Slashdot refuse to cover by Anonymous Coward · · Score: 0

    And when the one article he did clean up was another wrestler very close to Benoit, and you add in the fact the IP used was from near WWE's headquarters, you don't find it at all interesting?

    Also, Slashdot has covered Paris Hilton, when it relates to technology.

  13. CAPTAIN TACO CENSORSHIP by Anonymous Coward · · Score: 0
    Captain Taco is censoring us!

    .

    Watch the video, "Inseminated by Two Black Men", to find out how to combat this blatant trampling of our rights online!

    1. Re:CAPTAIN TACO CENSORSHIP by ascendant · · Score: 1

      who the hell is Captain Taco, and why would I care if he is censoring us?

      --
      Do not attribute to malice that which can be easily explained by incompetence.
  14. crashme by harry666t · · Score: 0, Offtopic

    lol. Half an hour ago I ran crashme on my 2.6.18 debian box.

    And guess what? It crashed.

  15. Bump Key? That would be the key on a keychain tha by redKrane · · Score: 0

    Bump Key? That would be the key on a keychain that one uses to take a bump. Remember to dust it off.

    --
    that's my word, holla...
  16. Can't tell by arth1 · · Score: 2, Insightful
    The article blurb says:

    We can't tell without further analysis and we can't narrow the possibilities down without the capability of replaying the entire test set in a methodical fashion.

    Yes, you can tell by experts eyeballing the code. Granted, this might be far more work than automated testing, but it's not like testing is the only way to isolate bugs.
  17. replaying set? by 192939495969798999 · · Score: 2, Insightful

    isn't the answer in the summary, that you obviously have it record the input as it goes, so you can literally back up and repeat any given random scenario? Without this capability, it would be like having a 3-year old bash away at the keyboard, they're just as unable to repeat anything.

    --
    stuff |
  18. Clean up after yourself by omfgnosis · · Score: 1

    A web application should be cleaning up after each transaction to arrive at a safe state; why wouldn't a web server do the same? If it doesn't, it should.

  19. I have a fuzzing toolkit for you by Anonymous Coward · · Score: 0

    It's called the internet.

    No, it's not a very good way to do a controlled test, but you seriously have to try this at one point in your life. Get a server with a static IP. Run a few normal services on it (Apache and so on). Don't advertise the server or its address anywhere. Give it a few months, and you'll be getting all kinds of unbelievable crap in the likes of access.log. And because you kept your mouth shut about the server, that crap isn't adulterated by so-called "real traffic". It's an awesome sight, and you can learn a lot about what exploit attempts look like, and which ones people are focusing on the most, just by doing this.

  20. Re:Why does Slashdot refuse to cover by Heembo · · Score: 1

    How about some Paris Hilton stories instead?
    OMG PONIES!
    --
    Horns are really just a broken halo.
  21. Re:Why does Slashdot refuse to cover by Repossessed · · Score: 1

    Paris Hilton found God in prison, and will only be showing those from now on if it somehow leads to her curing cancer,

    --
    Liberte, Egalite, Fraternite (TM)
  22. tcpdump? by cowbud · · Score: 1

    Is it just me or does it some obvious that you could just dump the stream then use tcpreplay to send the stream back and anaylze the packets it is sending to the webserver? Pick intervals and ranges of packets send them and see what the webserver does. That seems like a pretty straight forward way to narrow down what is going on.

  23. Different Spin On An Old Technique by Anonymous Coward · · Score: 0

    This article and discussion has been interesting in that it's given me a profound sense of deja vu. Despite being able to hack away in a few software languages (C(++), Java, Perl, Bash scripting, ...), I'm not up to date on the latest and greatest developer tools in this field, so could somebody enlighten me; is this the first widespread deployment of what has previously been called "constrained random stimulus" in the software and web app sphere? We've been using this for years in the ASIC design world, under this name ('constrained random') rather than 'fuzzing'. Basically, the idea is that in *hardware*, most "execution paths" on the chip get exercised (nearly) every clock cycle, which could translate into quadrillions or more times over the life of each ASIC (if you run 10 million of a product nonstop at 1GHz for 3 years, this is ~10^24 cycles). This, along with the increased difficulty and cost of changing the mask set after artwork release (compared to a software patch), means that verification must be carried out from a different perspective. I'm aware that many software projects are very careful to test every single obscure corner case in every regression set, but in hardware this *must* be done at all times, on all projects to have a prayer of releasing first-time-right silicon. To this end, we need to employ the pseudorandom generation capabilities of computers to apply stimuli that exercise corner cases humans may never be able to think of. As in software, we need *constrained* random because truly random input has a *much* longer time to convergence because the real stimulus is far from random; it is somewat random within a well-defined range of allowed values and sequences. For a long time, we wrote C, Perl, and Verilog/VHDL code that generated random stimuli and constrained them appropriately within the range of the appropriate state machines, protocols, etc.. When SystemVerilog was introduced a few years ago, one of the main features that it offers (besides much more OOP-like constructs for design and verification) is a constraint solver, where you are able to code in the constraints in a very high-level manner, and have the simulator generate random numbers and fit them to the constraints you have specified before applying them to the DUT. By doing this, you reduce your convergence time from millions of years (with pure-random stimulus) to something on a more human timescale, for non-trivial designs. Remember, the state space is exponential!

    Anyway, 'fuzzing' is an interesting topic, just thought I'd play 'Old Man ASIC' and repeat the truism that there's nothing new under the sun. Oh.. and.. YOU DAMN KIDS GET OFF MY LAWN!

  24. How about thinning the pipeline? by whitroth · · Score: 1

    The thing that irritates me most are the sites that have 5,478 *different* links. Even on broadband, they take tens of seconds, sometimes over a minute, to load. I'd like one *standard* test to be that they try surfing, like 50% of the US public does, on dialup.

    I won't even start on the idiots who have no compression on their cameras, and put jpgs up on their Websites that are over a meg....

                mark

  25. Huh? by Fuzzlekits · · Score: 1

    I do what to servers?....

  26. shoulda woulda coulda by mr_stinky_britches · · Score: 1

    A web application should be cleaning up after each transaction to arrive at a safe state; why wouldn't a web server do the same? If it doesn't, it should.

    Ya, and people should always wear their seatbelts, never drink and drive, and floss everytime they brush their teeth; Yet, the majority of humans fail to abide by these 3 simple "should haves".

    Maybe you should write a howto covering how you think a web server should behave, and then proceed to write your own implementation from scratch (for my network programming class I had to do this, in C++ with my own sockets library).

    Oh, and nevermind the fact that the web server may not have any state information.

    I'm sure you're just another troll or BOFH wannabe anyways.

    c'ya and good luck, buck.

    --
    wi-fizzle research, straight outta oakland
    --
    Censorship is obscene. Patriotism is bigotry. Faith is a vice. Slashdot 2.0 sucks.