Slashdot Mirror


CERT Releases Basic Fuzzing Framework

infoLaw passes along this excerpt from Threatpost: "Carnegie Mellon University's Computer Emergency Response Team has released a new fuzzing framework to help identify and eliminate security vulnerabilities from software products. The Basic Fuzzing Framework (BFF) is described as a simplified version of automated dumb fuzzing. It includes a Linux virtual machine that has been optimized for fuzz testing and a set of scripts to implement a software test."

51 comments

  1. Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 1, Insightful

    Anything that you write that uses a regex you should beat on with some fuzzing logic, since they can tend to increase in computational time non-linearly, and next thing you know you got a DOS on your hands.

    TIP OF THE DAY for you FROM ME

    1. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 1, Interesting

      This man speaks the truth. Just yesterday I had to deal with a Perl script whose execution time blew up once it had to process files larger than 1 KB in size. It'd work fine for 500-character files, but give it more than 1000 characters and the runtime would take over half an hour! (Yes, we had one user sit there and wait over 30 minutes for it to finish.)

      In the end, it was a poorly-written regular expression that was to blame. It was easy enough to fix, and we've since ditched the Indian team that developed the script. I think it's excessive, but our manager has now ordered a review of ALL the regexps in all of our scripts.

    2. Re:Fuzzing is only useful, if only moderately so by h4rr4r · · Score: 0

      You paid for cheap code and you got it. What did the pointy hair expect?

    3. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 0

      You assume that someone who is writing code (the GP) is simultaneously in a managerial position.

      Hint: Programmers cleaning up cheap code often didn't ask for the task. It was a PHB.

    4. Re:Fuzzing is only useful, if only moderately so by h4rr4r · · Score: 1

      Pointy hair is the PHB, what do you think PHB stands for?

      That is why I asked what the PHB expected, not what he thought the outcome would be.

    5. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 0

      I claim BS.
      Unless their perl code looked like this:

      # // PATTEERN MATSH
      system("java -Xmx1000m -Xms500m PatternMatchRoutineTakingTwoArguments.class SureName=$String MatchAgainst=\"Sri* OR sri*\" ThirdArgumentIsNotUsed");

    6. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 0

      Then you should not have said "you" if you were talking to the PHB.

    7. Re:Fuzzing is only useful, if only moderately so by ysth · · Score: 1

      Nope, no BS. Compare perl -we'("a" x 10) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/' and perl -we'("a" x 100) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/'

    8. Re:Fuzzing is only useful, if only moderately so by elfprince13 · · Score: 1

      I'm sorry you're uneducated about Perl's notoriously poor regex implementation, that has unfortunately become the standard in most languages. Reference: http://swtch.com/~rsc/regexp/regexp1.html

    9. Re:Fuzzing is only useful, if only moderately so by elfprince13 · · Score: 1

      Note particularly the units on the vertical scale of those two graphs. The Perl algorithm is measured in seconds, the Thompson algorithm is measured in microseconds.

    10. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 0

      Umm, I took that as the "group" "you" not the personal you.

      It made sense to me and I'm drunk as fuck, whats your excuse?

    11. Re:Fuzzing is only useful, if only moderately so by Menkhaf · · Score: 1

      Thanks for the link, it seems like a good read for a cloudy day.

      Do you know if anything has changed since the article in 2007?

      -menkhaf

      --
      A proud member of the Onion-in-Hand alliance
    12. Re:Fuzzing is only useful, if only moderately so by mr_mischief · · Score: 2, Informative

      $ time /usr/local/bin/perl -we'("a" x 100) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/'
      37.00user 0.01system 0:37.81elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
      0inputs+0outputs (0major+438minor)pagefaults 0swaps

      $ time /usr/local/bin/perl -we'("a" x 10) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/'
      0.00user 0.00system 0:00.00elapsed 75%CPU (0avgtext+0avgdata 0maxresident)k
      0inputs+0outputs (0major+438minor)pagefaults 0swaps

      For a purposely selected pathological case on a Pentium M 1.6 GHz laptop with little free RAM, I'd say that's not bad for a system that has specifically been chosen to support grouping, alternation, backreferences, conditional changes (case sensitivity, prematch, postmatch, etc) on only parts of the expression, greediness and nongreediness, lookahead, and lookbehind. Perl "regular expressions" are definitely not actually regular.

      That's perl 5.12.0 BTW, which is much improved over older series (pre 5.10 anyway) of perl systems regarding regexes.

      Note that if you're okay with intentionally trying and failing to get a case-sensitive rather than case-insensitive 'b' after your pathological quantifiers on the 'a' characters, then you have no such time problem.

      $ time /usr/local/bin/perl -we'("a" x 100) =~ /(a*)(a*)(a*)(a*)(a*)(?:b)/'
      0.00user 0.00system 0:00.00elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k
      0inputs+0outputs (0major+428minor)pagefaults 0swaps

      $ time /usr/local/bin/perl -we'("a" x 1000000) =~ /(a*)(a*)(a*)(a*)(a*)(?:b)/'
      0.00user 0.00system 0:00.00elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
      0inputs+0outputs (0major+673minor)pagefaults 0swaps

      $ time /usr/local/bin/perl -we'("a" x 1000000000000) =~ /(a*)(a*)(a*)(a*)(a*)(?:b)/'
      0.00user 0.00system 0:00.00elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k
      0inputs+0outputs (0major+426minor)pagefaults 0swaps

      I'm sure the p5p would welcome a patch that delivers the promised matching semantics without performing so poorly on pathological cases.

    13. Re:Fuzzing is only useful, if only moderately so by daemonburrito · · Score: 1

      Thanks, that was really interesting.

    14. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 0

      Perl development has stagnated pretty badly since about 1997. Sure, they get new releases out, but they only contain bug fixes.

    15. Re:Fuzzing is only useful, if only moderately so by ysth · · Score: 1

      The point is that it doesn't scale linearly. 1000 is much worse, and 10000 much worse than that.

      Yes, any particular case *could* be "fixed" in the regex engine, but there will always be one more possible pathological regex.

    16. Re:Fuzzing is only useful, if only moderately so by elfprince13 · · Score: 1

      As far as I know, the same poor implementation is still in use. I don't use Perl, but I do use a lot of languages which make use of PCRE which uses the same faulty algorithm.

    17. Re:Fuzzing is only useful, if only moderately so by mr_mischief · · Score: 1

      Yeah, and checking specifically for a possible type of pathological regex in order to work around it would require slowing the engine for the common case. So I'd say the power of the engine to do so many things is a good trade for the easily avoided pathological cases.

      There are only two situations in which such pathological cases are likely to be given to the regex engine anyway. One is when long regexes are built automatically by a program based on a set of rules and a particular data set, which should only trigger a pathological case rarely based on random chance. The other is when people set out to specify something pathological on purpose just to demonstrate the issue. The rest of the time, a programmer should be interested in using the regex engine to actually do something useful and there's really no use in any of the contrivances used for such torture tests.

    18. Re:Fuzzing is only useful, if only moderately so by Anonymous Coward · · Score: 0

      If there are *pathological* regular expressions, that means that the algorithm is poorly defined. A regular expression is *by definition* something that can be handled by a DFA (and your pathological case can be handled by a DFA), and so also *by definition* polynomial (I'm not absolutely sure, but I think that a DFA should always be expressible with a quadratic, or maybe it's a cubic, algorithm). That sure as hell doesn't look polynomial to me. It looks at least sub-exponential. That tells me that somebody doesn't know how to write a simple walk algorithm.

    19. Re:Fuzzing is only useful, if only moderately so by ysth · · Score: 1

      Perl regexs are not "by definition" regular expressions, and are not handled by a DFA. By choice, not out of ignorance.

    20. Re:Fuzzing is only useful, if only moderately so by ysth · · Score: 1

      No, it actually happens in the wild (see the post ^ that had BS claimed on it) typically when parsing a string up into many substrings where the boundaries aren't specified in such a way that backtracking is prevented.

    21. Re:Fuzzing is only useful, if only moderately so by mr_mischief · · Score: 1

      Sure, it happens, but with what frequency?

      Any artificial language you're parsing should be designed with this in mind, or at least a different tool could be clearly chosen for it.

      If you're parsing any sort of natural language, well first you kind of deserve what you get. ;-) The variability of the language would seem to preclude this sort of massive backtracking in the common case, and the parser could use multiple separate regexes to help lower the chances of triggering the problem.

      It still seems from my experience that the common case is of reasonable performance is much more common, although rare incidents might happen.

    22. Re:Fuzzing is only useful, if only moderately so by mr_mischief · · Score: 1

      Thanks for reading the thread... oh, wait, you didn't. I already said that Perl's extended "regexes" are not regular. Well, thanks for weighing in anyway.

  2. Cool by gweihir · · Score: 1

    And urgently needed. So far the CMU/CERT software I had a look at was pretty good....

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  3. bleh by Anonymous Coward · · Score: 1, Informative

    Sort of like this?

    1. Re:bleh by Anonymous Coward · · Score: 1, Interesting

      I wouldn't be so quick to automatically label researchers as professionals. If there is one place I've seen worse code than OSS, it would be in academia.

      Bizarrely, this is also where I've seen the most brilliant code.

    2. Re:bleh by Anonymous Coward · · Score: 1, Funny

      After writing that, I had to compare the two.

      Fuzz's code resides in a massive 34KB C file, undocumented and unformatted, liberal copy-pasta and no re-use, no grasp of language, about as readable as an assembly dump. Basically one big hack.

      BFF is a nicely written, concise, small, and extensively documented perl script.

      Yep... OSS fails to impress once more.

    3. Re:bleh by fuzzyfuzzyfungus · · Score: 2, Insightful

      Dare I inquire as to the thought process behind the notion that the inferiority of an OSS program called "Fuzz" and the superiority of an debian-based VM, running a GPLed perl script automating a WTFPLv2-licenced fuzzer proves the unimpressiveness of OSS?

    4. Re:bleh by Anonymous Coward · · Score: 0

      The Perl code is not that great, and here and there stuck in the late 90's.

    5. Re:bleh by Daniel+Dvorkin · · Score: 2, Insightful

      If there is one place I've seen worse code than OSS, it would be in academia.

      Bizarrely, this is also where I've seen the most brilliant code.

      If you look closely, you'll find that the "brilliant code" is most often written by academics who have industry programming experience. Similarly, in industry, you will find that the best code is written by experienced programmers with rigorous academic backgrounds. In contrast, the academics who insist that computer science has nothing to do with programming, and the self-taught hackers who proudly proclaim their lack of all that fancy book-larnin', are two sides of the same worthless coin.

      --
      The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
    6. Re:bleh by Anonymous Coward · · Score: 0

      It doesn't. But there isn't a clever way to distinguish between hobbiest OSS code and "professional" OSS code (by organization). Former represents OSS more than the latter does, so I (ambiguously) called BFF 'OSS'.

      I guess I could have called it, community developed OSS. It usually stands that projects made by internal development team are of higher code quality.

    7. Re:bleh by NewbieProgrammerMan · · Score: 1

      If there is one place I've seen worse code than OSS, it would be in academia.

      You've only seen code from those two, then? Because there's no shortage of hard-core WTF code in private business and government.

      --
      [b.belong('us') for b in bases if b.owner() == 'you']
    8. Re:bleh by ysth · · Score: 1

      Pet peeve: use strict, but not warnings enabled. Yes, strict is really important; BUT WARNINGS ARE MUCH MORE SO.

    9. Re:bleh by Anonymous Coward · · Score: 0

      I can attest to this, I have seen code written by a major multi billion dollar DoD contractor better known for aircraft than software that didn't account for the fact that in javascript "8" and "9" my be valid numbers but "08" and "09" are not. Therefore it was impossible to submit forms with dates on the 8th or 9th of any month, or in August or September without turning off browser script support. This was in delivered production code.

    10. Re:bleh by Eskarel · · Score: 1

      In their defense, interpreting strings beginning with a zero as being in octal is fucking retarded, and that's the cause of that particular bug. It's even been deprecated in the spec.

  4. axfuzz by shird · · Score: 5, Interesting

    in their whitepaper they referenced my 'axfuzz' tool I wrote years ago and even used a modified version of it in their testing. Hope they didn't judge me on that code, it was a pile of crap that I kept hacking together until it finally worked, with no thought to proper software design.

    --
    I.O.U One Sig.
    1. Re:axfuzz by __aasqbs9791 · · Score: 2, Insightful

      I think the fact they are using a modified form means they did judge you, and found it good enough to use as a start. That should count for something.

    2. Re:axfuzz by TubeSteak · · Score: 3, Funny

      Hope they didn't judge me on that code, it was a pile of crap that I kept hacking together until it finally worked, with no thought to proper software design.

      That sounds like exactly the kind of code a fuzzer should be used upon.
      Oh the recursion!

      --
      [Fuck Beta]
      o0t!
    3. Re:axfuzz by Menkhaf · · Score: 1

      Hi,

      Do you know if the whitepaper available somewhere for free?

      Another link for further information about BFF, is the CERT Vulnerability Analysis Blog, found at https://www.cert.org/blogs/vuls/2010/05/cert_basic_fuzzing_framework.html

      -menkhaf

      --
      A proud member of the Onion-in-Hand alliance
    4. Re:axfuzz by shird · · Score: 1

      I was just referring to this technical document: http://www.cert.org/archive/pdf/dranzer.pdf [pdf]

      referenced from: http://threatpost.com/archive/blogs/dranzer-fuzzing-activex-vulnerabilities which is linked to from TFA.

      --
      I.O.U One Sig.
    5. Re:axfuzz by shird · · Score: 1

      I see Dranzer is an older ActiveX fuzzing tool, somewhat unrelated to the "BFF". I didn't really read the article properly. :P

      --
      I.O.U One Sig.
  5. hmmm... by thatskinnyguy · · Score: 2, Funny

    The worst case scenario is talking about worse case scenarios thinking about worse case scenarios and letting them possess you.

    --
    The game.
  6. Linky? by Anonymous Coward · · Score: 3, Informative

    Oh FFS, you couldn't even link to the damn framework?

  7. BFF? by Fnord666 · · Score: 2, Insightful

    BFF? What an unfortunate choice of acronyms.

    --
    'The tyrant will always find pretext for his tyranny.' - Aesop's Fables
    1. Re:BFF? by Daniel+Dvorkin · · Score: 3, Funny

      Because it's, like, the security researcher's BFF OMG ponies!

      --
      The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
  8. Good!! by Panaflex · · Score: 1

    I propose that every website which handles private data (credit, ssn, health, etc) should be integrating these kinds of tools into normal test procedures, both in development and on production mirrored sites.

    Hear hear!

    --
    I said no... but I missed and it came out yes.
  9. Some of that may be true by Anonymous Coward · · Score: 0

    But for me it's simply about learning the conventions/idioms of whatever language/system you're programming in and taking some fucking pride in your work.

    My code has actually become worse since I moved from academia to industry -- at least, I've been "forced" into a lot more hacks. Two reasons: Shortsightedness on the part of stakeholders/PHBs and the fact that you cannot afford to rewrite absurdly broken code written by your coworkers. Sad but true.

  10. Which license? by Anonymous Coward · · Score: 0

    Bitches! If I ever see somebody IRL who tells me he's a slashdot editor, god I swear I will punch him.

  11. Well, then "my hats off" to you man... apk by Anonymous Coward · · Score: 0

    "in their whitepaper they referenced my 'axfuzz' tool I wrote years ago and even used a modified version of it in their testing. Hope they didn't judge me on that code, it was a pile of crap that I kept hacking together until it finally worked, with no thought to proper software design." - by shird (566377) on Thursday May 27, @09:48PM (#32371160) Homepage

    Per my subject-line above: My "hat's off" to you man, in terms of "good job on your end". It's always nice (and, a treat) to see that someone from this website (or rather, someone that frequents it) has actually produced something of use that's for "the common good", or used by others even (right up there with being imitated, as far as "flattery")!

    (Change, REAL CHANGE? Starts with YOU, & you exemplify this via your works here!)

    APK

    P.S.=> The funniest part is, is this (and, I am sure you are aware of it, based on your works): Anyone AND EVERYONE has this type of potential, IF they apply themselves!

    You seem to "berate yourself" for your code... don't!

    (& yes, I know how that is, looking back on code you wrote YEARS (if not decades) before & saying to yourself "WTF? I did THAT?? I am amazed it works @ all!"... but, this is only your own personal growth/progress showing through, because you're now aware of how you could have done it better (a great sign really - imo @ least? It means you're growing & improving!)).

    You KNOW the deal, i.e.-> That You'll fail a few times, @ first, but then? THEN, you get it working @ least. For starters... then you can improve on it, IF you wish, & that, as I am also sure you know, takes time & observations + inference (been there recently on an app I wrote to remove duplicates from a HOSTS file I build. I wrote the 1st version in 2002, it utterly sucked. I later rewrote it (I used Borland Delphi because it's THE fastest in strings + math work, doubling MSVC++ even in Visual Basic Programmer's Journal 1997 issue "Inside the VB5 Compiler Engine")).

    Near the end, & got it down to 1 hr. processing time over 1 million records, down from 2 hrs. processing/parsing time... Anyhow/anyhow: I "hit the wall", & couldn't get the code any faster (inline asm & all, + compiler optimizations & by hand ones via profiling and identifying the slowest area) - this "perturbed me".

    HOWEVER, at the very end, it took me looking @ THE DATA ITSELF, & how I was parsing + processing it (1 large file) & it made a HUGE diff. in how fast it processed removal of HOSTS file duplicates (yes, a SELECT DISTINCT type query in an indexed DB engine can do it faster, but, I don't even use indexing in mine (don't feel like implementing a b-tree either) & it's really a tool for those of us that don't "do SQL" (I do, but meaning others here), or possess a DB engine to do so (though mySQL server is free, not everyone's mastered SQL).

    So, how'd I improve it? AGAIN - Not in the code itself, but in the data!

    Heck - I ended up "busting up" the 1++ million records into 8 equal parts, all sorted, & now I have the thing down to 24 minutes processing time, vs. its original mark of 2++ hrs.!

    (Sure, I could have altered the algorithm in code I used, seeing it favors "smaller/smallish" datasets over larger ones for deduplication of entries, but this worked gangbusters, so I went with it!).

    SO, I guess you could say "I KNOW THE FEELING" (on what I quote from you)...

    In the end? Great job, you've done something SO WELL, others "bit off your style" man... kudos! This world needs more "creative types" like you out there, helping improve it (that's MY "bottom-line" here), & once more? Always cool to see another "creative type" here on this website (you're a rarity imo & observations here @ least, the past 7++ yrs. or so visiting here)...apk