Slashdot Mirror


String Cleanup Results On OpenBSD

Dan writes "OpenBSD's Theo De Raadt provides an update on his team's efforts to remove potential buffer overflows within OpenBSD code by always calculating what the bounds of an operation are. They have been going through the source tree cleaning out all calls to sprintf(), strcpy(), and strcat(). Theo says that they have removed (replaced) approximately 2000 occurences of these functions." (The same buffer overrun-squashing effort was mentioned earlier this month.)

37 of 53 comments (clear)

  1. BSD Coding Standard. by UnknownSoldier · · Score: 3, Insightful

    Does the BSD team have a list (or rules of thumb?) that mentions other safe coding practises? There has to be book on this, right? (I've always been impressed by the pro-active stance BSD takes towards security -- I just wish the rest of the commercial world could afford the time to do things right, instead of the cheesy no liability out-clause in the EULA.)

    If most developers are still using these "trivial" funcs, I'm scared what other funcs are just as buggy!

    Funny how one can forget all about these "harmless" bad practises. Time to add it to the internal coding standard. :)

    1. Re:BSD Coding Standard. by TilJ · · Score: 4, Informative

      There's a summary of good practices at http://www.openbsd.org/porting.html#Security. The white papers that the team has produced (for example, on the str "l" variants) are also good reading.

      --
      "The purpose of argument is to change the nature of truth." -- Bene Gesserit Precept
    2. Re:BSD Coding Standard. by renehollan · · Score: 1
      Grandparent posteth:

      If most developers are still using these "trivial" funcs, I'm scared what other funcs are just as buggy!

      Funny how one can forget all about these "harmless" bad practises. Time to add it to the internal coding standard. :)

      eliciting parent's informative reference on proper coding practices.

      I wish it were that simple. Recently starting at a new position, I was horified at the number of strcpy()s, strcat()s, and sprintf()s peppered throughout code -- mostly building up a string to be send to a logging routine. Variable function parameter lists would have helped a lot here (yeah, they are ugly and not type-safe, but considering the interface being emulated, i.e. printf(), used them, it seamed natural), as well, building up a buffer in one place, instead of peppered throughout the code.

      When I suggested (a) the variable parameter list technique and (b) careful use of vsnprintf(), I was strongly rebuked for the following reasons:

      1. (a) is "too hard" for programmers to understand. Of course the lack of type-safety was used to further bolster this reason why it should be shunned (/me mutters something about not hiring idiots in the first place.

      2. It is a waste of time to make code robust and secure. This even though the original author got bitten on the ass by a buffer overflow segfaulting his program and spending more time debugging that than it would take to use the simple bounded versions of the functions in question).

      As long as the mean programmer skill is so mediocre, and one can "legalize away" the need for security and robustness, trivial holes like this will continue to proliferate. It is perceived to be cheaper to leave them than fix them.

      While I don't agree with this observation, for trivial code with minimal maintenance commitments, and an army of lawyers already on retainer, it may hold true. Unfortunately.

      Of course, when the legal defense breaks down, the programmer who didn't fix the problem lest he or she lose his or her job for insubordination, will be used as the corporate scapegoat.

      ...rocks, hard places, and all that.

      --
      You could've hired me.
    3. Re:BSD Coding Standard. by renehollan · · Score: 1

      o perhaps it's better to use a language where the default easy thing to do is to use the checked version (say, Java)?

      In some instances, yes.

      However, in some instances, those checks can be unnecessary time killers, when you are trying to squeeze the last bit of performance from a program, and have done careful static analysis to ensure they don't happen, you don't need to check at run-time.

      IOW, you need to be able to chose between the "luxery car" with the "smooth ride", and the Formula-1 racer.

      The trouble with using "the right language for the job" is that even fewer programmers are skilled in multiple languages, and this tends toward the desire for a "general purpose" language. C, and to a better degree, C++, fit these bills, but you'd be right in pointing out that they provide a million different ways to shoot yourself in the foot (conversely, Java is poor if you're into real-time video deinterlacing and rendering).

      Also, in many applications, there is a small need for tight and high-performce "as close to the metal as possible" code, combined with less-demanding code as support. Again, the complexity of a multi-language approach goes against what might be a use of "the right tool for the right job".

      So, we script in bash, or perl, or python (cleverly embedding an interpreter where appropriate), and use general purpose languages when we need to perform at the bleeding edge. It's hard to shoehorn Java into that environment, except, perhaps, where platform independence, or traditional web-based interfaces (which imply platform independence) are required (Java's "write once, debug everywhere" reputation discounted for the benefit of it's advocates).

      C++ offers most of the language features to provide "safe" wrappers around unsafe, or otherwise tricky code. Unfortunately, it does not provide a decent mechansism for the programmer to explicitly disallow certain language features in certain portions of code: use of smart pointers, for example, is hard to enforce when one can "cheat". In all fairness, C++ was never intended to forcefully protect the programmer from him or herself.

      I think the answer likes in mutable languages (of which neither C++ nor Java is one), but those are among the worst for programmer skill, because they basically embrace the philosophy of: "Consider your problem. Consider an elegant language for expressing its solution. Code in that language, using this mutable language base." Very few people are good at coding that way (though, it is a sign of the best programmers).

      --
      You could've hired me.
    4. Re:BSD Coding Standard. by 0x0d0a · · Score: 1

      (I've always been impressed by the pro-active stance BSD takes towards security -- I just wish the rest of the commercial world could afford the time to do things right, instead of the cheesy no liability out-clause in the EULA.)

      You're thinking of OpenBSD.

    5. Re:BSD Coding Standard. by renehollan · · Score: 1
      Nonetheless, I do still think we'd be better off if people said, "I use C because it's what everyone else uses," rather than the misleading "I use it because it gets me closer to the bare metal" (or other common lines).

      Point noted, but I meant "... when compared to Java". C still has a lot of horsepower on uniprocessors, without esoteric memory and interconnect performance hierarchies. It isn't popular without good reason, those those reasons, as you note, are starting to get dated. Though, it is probably true that, when the need for performance exceeds what one can craft in C, its left as an exercise to the hardware folks to make a supporting ASIC.

      --
      You could've hired me.
    6. Re:BSD Coding Standard. by wayne606 · · Score: 1

      I know this will be an unpopular statement... But security is important only for programs that are accessible via the internet. Who cares if an application like gimp (to pick a name at random) has code like

      char buf[20];
      GetUserInput("Input the number of seconds since you last ate something", buf);

      when the only person who has access to that vulnerability is the user himself? I bet 95% of all the useful programs out there fall into this category. It's a waste of time to put an ultra-secure lock on the door between the kitchen and dining room...

    7. Re:BSD Coding Standard. by UnknownSoldier · · Score: 1

      > You're thinking of OpenBSD.

      Yeah, since the article was about Open, I didn't think I had to specify it. ;) My bad, I should of said "stance [Open] BSD"

    8. Re:BSD Coding Standard. by jhunsake · · Score: 1

      Not only is it an unpopular statement, it is completely wrong. Ever heard of a "local exploit"? Not all programs run with the permissions of the user that executed them.

    9. Re:BSD Coding Standard. by wayne606 · · Score: 1

      [Sorry, playing the devil's advocate a bit here] Say there is sloppy non-buffer-checking code in a drawing application (I don't know if there are any in a particular program so I won't be specific). The kind of code that would lead to a vulnerability if it were in an internet server. This program doesn't run setuid and doesn't listen to any sockets and the only way the overrun could be exploited is for the user to do it himself. Tell me how this is a security problem.

    10. Re:BSD Coding Standard. by lamontg · · Score: 1
      Programs can get used in unusual ways. If you happen to have a drawing application which is started up by your web-browser then a malicious piece of content on the web could hack into your account that way. Setuid programs may call external programs which don't normally run with elevated privs (sudo is a really common way of doing this).


      There are two different ways of dealing with this -- either auditing your entire system for exploits assuming that any piece of code may be running with elevated privs at some point -- or by auding your system to reduce the amount of code which runs with elevated privs. I would argue that both approaches are necessary. As an OS vendor you should be taking the former approach since you want to give your customer the most flexible system that you can and dont want to limit them.


      OpenBSD is actually doing the right thing there. I think there's a bit of arrogance on the part of other operating system vendors who would rather say "well you shouldn't be using that in an evironment with elevated privs" rather than fixing their issues.

  2. OpenBSD by duffbeer703 · · Score: 1, Troll

    These guys have been claimng to be super-secure and constantly performing security audits on the OpenBSD code for years.

    Yet they've launched a major effort to cleanup 2,000 unsafe string functions in the last two months...

    What has Theo been doing all this time other than being an obnoxious prima donna and re-writing packet filters because of some minor squabble?

    --
    Conformity is the jailer of freedom and enemy of growth. -JFK
    1. Re:OpenBSD by sirket · · Score: 1

      This is like asking why Chevy didn't recall a car that blew up even though they didn't know about the problem ahead of time.

      Security is a constantly moving target (Format string vulnerabilities are very new on the scene for example). The string functions that were replaced did not have holes, they were replaced because they wanted to avoid even the possibility of a vulnerability and because they wanted to clear all standard string operations out to make searching for possible future vulnerabilities easier and faster.

      As for pf... More has been accomplished by pf in the last few months than has been accomplished by ipf since it came out. pf is even implementing state synchronization. I've been waiting for this in ipf for a long time. pf is not perfect, but neither is ipf.

      -sirket

    2. Re:OpenBSD by duffbeer703 · · Score: 1, Flamebait

      According to Theo, the OpenBSD team is continuously auditing OpenBSD code. Is Theo re-writing grep because he has an issue with whomever wrote it thirty years ago?

      Unchecked string problems have been known since the standard C libraries came out. I first heard about them around 1995.

      All I'm trying to say is that OpenBSD would be a much more secure system if Theo spent more time working on it rather than grandstanding.

      --
      Conformity is the jailer of freedom and enemy of growth. -JFK
    3. Re:OpenBSD by Nickus · · Score: 1

      Why don't you download some GNU package and see how many occurences there is of those functions? Why don't you go and fix those? Why complain when people are actually fixing things?

    4. Re:OpenBSD by 0x0d0a · · Score: 1

      They aren't unsafe if used properly. They're just easy to use unsafely. Furthermore, they're slightly slower than their bounded cousin functions.

      I'm sure use of unbounded string functions were looked over and checked for possible bugs. This just makes it harder for another kernel bug to cascade into a kernel compromise.

      Believe me, this is way, way more than our friends in Redmond have been doing.

    5. Re:OpenBSD by glitchvern · · Score: 2, Interesting

      A valid question if you don't know the answer, you shouldn't have been moderated as a troll I think. These "unsafe string funtions" sprintf(), strcpy(), and strcat(), are only unsafe if they are used incorrectly, which is easy to do. OpenBSD's source has been audited in the past to make sure they are used correctly. Now instead of making sure these calls are used correctly they are ripping them out and replacing them with calls to safer string functions. Interestingly it appears Theo is being successful in getting these changes made upstream in openssl, sendmail, and bind. I believe in the past there were some bind security flaws that didn't affect OpenBSD because they had altered their version of bind. The functions they are using to replace the "unsafe" ones are: snprintf() which first appeared in 4.4BSD (as in the original Berkley bsd), asprintf() which came from the GNU C library and first appeared in OpenBSD in 2.3, strlcpy() and strlcat() which were created by OpenBSD in 2.4. The man page for strlcpy() and strlcat() is one of my favorites for the blurb at the end under EXAMPLES:
      However, one may question the validity of such optimizations, as they defeat the whole purpose of strlcpy() and strlcat(). As a matter of fact, the first version of this manual page got it wrong.

      Also the packet filter thing wasn't a minor squable. It had a licence which sounded like a bsd licence but did not expressly permit distribution of modified source, thus making distribution of modified source illegal. It also was suppose to be installed into / by default, default being the only way you could legally distribute it. The author thought he was gonna go ahead and point that out and be an ass about it. Pretty strange for a licence which said I hate legalese don't you in it.

    6. Re:OpenBSD by duffbeer703 · · Score: 1

      Thanks for the explanation, that really clarifies everything for me.

      For someone like me who doesn't really use BSD and sees everything going on with it as a spectator, sometimes the political and other issues surrounding it obfuscate the technical detail.

      --
      Conformity is the jailer of freedom and enemy of growth. -JFK
    7. Re:OpenBSD by jemfinch · · Score: 1

      They're actually slightly faster than their bounded cousins; their bounded cousins have to check the bound on every iteration through the loop (all these functions are fundamentally just loops), whereas the unsafe versions can simply not do that check.

      Jeremy

    8. Re:OpenBSD by 0x0d0a · · Score: 1

      Ooops...yeah, misspoke.

    9. Re:OpenBSD by sean23007 · · Score: 1

      It's almost refreshing to see someone complain that the most secure OS could be more secure. Oh, wait. Let the guy do his job -- maybe if he wasn't doing such a great job his grandstanding wouldn't be as acceptable.

      --

      Lack of eloquence does not denote lack of intelligence, though they often coincide.
  3. Shifting so much by mnmn · · Score: 2, Interesting

    I wonder when Theo can say they have removed ALL occurrences of these functions??

    But more importantly, why wasnt it possible to replace the functions in the library with something (if a bit slower) robust?

    Are we witnessing the evolution of the New Libc(tm) ?? Can I patent it?

    --
    "Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
  4. Because the functions don't spec. a buffer length by rklrkl · · Score: 3, Informative
    If you check the man pages for the 3 functions, you'll see that they just take char * pointers with no lengths specified, so they'll just copy from the source data until they hit a zero char or cause a buffer overflow exception.

    To answer your question, it's not possible to replace the original functions in libc because there's no maximum length param in those functions (unlike the safer "n" equivalents like snprintf()).

  5. I think Theo misunderstood... by Slashed+Otter · · Score: 2, Funny

    ...when his mother/SO said, "I think you should do some spring cleaning."

  6. snprintf() on Win32 with MSVC6 by pkplex · · Score: 1

    Something that irritates me about using MSVC:

    #include
    void main(void){
    char buff[1024];
    snprintf(buff, 1024, "Test"); /* wont compile */
    _snprintf(buff, 1024, "Test"); /* will compile */
    return;
    }

    GRR!! Very annoying when trying to write code for both win32 and *nix.

    1. Re:snprintf() on Win32 with MSVC6 by 0x0d0a · · Score: 1

      Just have a header file that checks what compiler it's running under (probably #idef __MSVC__ or something like that) and if so, #defines snprintf to _snprintf.

    2. Re:snprintf() on Win32 with MSVC6 by TheRaven64 · · Score: 1
      And you can't just pop
      #ifdef _MSVC
      # define snprintf(x,y,z) _snprintf(x,y,z)
      #endif
      or similar in a standard header?
      --
      I am TheRaven on Soylent News
    3. Re:snprintf() on Win32 with MSVC6 by Electrum · · Score: 1

      GRR!! Very annoying when trying to write code for both win32 and *nix.

      So fix it...

      #ifdef WIN32
      #define snprintf _snprintf
      #endif


      Or get the free Borland compiler.

    4. Re:snprintf() on Win32 with MSVC6 by pkplex · · Score: 1

      Oh :D Thanks guys, That is very usefull information :)

    5. Re:snprintf() on Win32 with MSVC6 by Vej · · Score: 1

      Here is the free ANSI compliant borland compiler: http://community.borland.com/article/0,1410,20633, 00.html

      Also, this seems to be the reasoning for the odd va_arg macros and these functions defined in stdio.h

      Purpose:
      * This file defines the structures, values, macros, and functions
      * used by the level 2 I/O ("standard I/O") routines.
      * [ANSI/System V]

    6. Re:snprintf() on Win32 with MSVC6 by adri · · Score: 1

      Or, to be portable AND clear:

      #define BUFSIZE 1024
      main()
      {
      char buf[BUFSIZE];

      snprintf(buf, BUFSIZE-1, "foo");
      }

  7. Re:Because the functions don't spec. a buffer leng by cpeterso · · Score: 1


    Exactly, the bugs are in libc, not the calling programs. OpenBSD should remove all dangerous libc functions, like sprintf() and strcpy(). Then unsafe user programs will not even compile. If OpenBSD is really, really serious about security, they could then disallow these unsafe programming practices. Source code ported to the new restrictive OpenBSD libc would benefit on other platforms, too.

    If forever removing strcpy() and friends is just too radical, OpenBSD could use a transitional approach by moving the function prototypes of the unsafe libc functions to a non-standard header file. Call is something like unsafe.h, deprecated.h, or i_like_buffer_overflows.h.

  8. Re:Because the functions don't spec. a buffer leng by realdpk · · Score: 1

    At least functions that call that stuff could produce a warning to stderr, like other functions are known to do.

    Would be pretty neat.

  9. Re:Because the functions don't spec. a buffer leng by almeida · · Score: 1

    The problem with removing strcat and strcpy is that you can use them safely if you put some thought into what you're doing. There's no reason to ban them from libc just because some programmers don't check their buffer sizes. Removing the old string functions would make a lot of code not work on OpenBSD.

    Also, the OpenBSD team isn't going to touch most of the third party code in the tree: "We are obviously not doing this to some parts of the tree which we borrow from other projects. In particular, the gnu part of the tree might remain largely dirty." So they'd kill a lot of functionality by banning them. They did make the changes in OpenSSL and Apache and it sounds like they've had some luck convincing the BIND and Sendmail guys to move to strl. It's obvious OpenBSD knows you can't change the world, but they can change their neighborhood and set a good example.

    What they might end up doing in the future is adding a compile time warning similar to the one generated by mktemp that suggests using mkstemp instead. That's probably the best approach.

  10. Spring Cleaning by FunnyPolynomial · · Score: 1
    Some of you might have noticed that we have been doing a 'string cleaning'. (I should probably apologize for this northern-hemisphere specific pun).

    Groan. Umm, we do spring cleaning in the southern hemisphere too ya know! Our spring follows winter (the cold one) and preceeds summer (the hot one) as well!! The only difference is the time of year.
    --
    // todo: implement sig
  11. This doesn't mean much... by MiniGhost · · Score: 1

    Some people on here are asking what the obsd group has been up to over the past few years. Some are claiming that they have spent a couple years rewriting packet filters, and are just now checking for unsafe string handling. This is not true. The source tree was/and is continuously being audited. strcat/strcpy/sprintf can be dangerous functions, BUT BUT BUT BUT, if they are used with proper condition checking code before they are called (as the obsd src does), then they are no more dangerous than strncat, etc. i.e.

    char buf[10];
    if(strlen(ptr) < sizeof(buf) - 1){
    strcpy(buf, ptr);
    }

    This example uses an unsafe function, but in a relatively secure way.. (assuming strlen(...) works properly :-)

  12. Re:shame on you slashdot editors by Pharmboy · · Score: 1

    Why couldn't a link have been given to the place that first aired this story?

    You obviously do not understand how /. works. Try reading the FAQ before opening your yap.

    Also, by removing down-moderating this post, you are proving my point that you censor what isn't convenient or fits your own purposes.

    Once again, read the FAQ. Nothing is censored. Set your threshold to -1 and you can read all the trolls, like this one.

    You are clueless.

    --
    Tequila: It's not just for breakfast anymore!