Slashdot Mirror
Analyzing Binaries For Security Problems
Matt writes "At the last talk at BlackHat in Las Vegas, Greg Hoglund demonstrated a product for sale by his new company that analyzes binaries for security vulnerabilities. He showed the analysis of several commercial products, the results of which were shockingly insecure. This product should help end the debate of closed source or open source applications being more or less secure."
Then again, it's not like virus scanners don't do the same thing.
Isn't it kind of strange how they make such big claims but present no actual evidence?
Is that, provided you have the ability, then you don't have to sit around and wait for someone else to fix the problems in the programs you use...
Still, politics aside, perhaps with more applications like this freely available, perhaps more bugs will actually be fixed - rather than relying on security through obscurity - sitting tight and hoping no-one notices...
Leave me alone! - I can dream can't I ??
You can get the slides of his presentation here:
h -us-03-hoglund.pdf
http://www.blackhat.com/presentations/bh-usa-03/b
I'd like to know exactly how it does this, considering how much of a mess compiled/optimised c++ code can look at an assembler level. It's also unlikely to be any use on a semi-compiled runtime, such as those used by Visual Basic, .NET etc as the only 'code' is the runtime, the actual program is held in a data section.
I can't imagine this program to work very well - finding buffer overflows and other possible security vulnerabilities can be an immensely hard task when you actually _do_ have access to the source code. Also, the available compilers produce quite different assembly for the same code. This just all sounds a little bit too good to be true...
Homepage
$ file /usr/lib/jed/bin/w32/w32shell.exe
/usr/lib/jed/bin/w32/w32shell.exe: MS Windows PE 32-bit Intel 80386 console executable not relocatable
And voila!
Judging from the url, they don't have a lot of faith in open source software.
I just put my boss's Windows 2003 Server CD under a microscope to examine the binaries.. Started zooming in.. and then SNAP. The bitch cracked into 2. I'll put gentoo on the server now and just tell him that a security cracker broke his shit.
-B
If this can be used to detect for example buffer overflows than does n't it also help speed up a crackers turn around rate?
I mean instead of trying to find flaws instruction by instruction with some debugger, simply specify all exe and dll's in your %winroot% directory press start and wait for the report and then manually inspect hilighted areas.
So this analyses binaries and will find all issues where the code will halt and will exceed its resource requests, thus eliminating the need for testing...
I call Snake Oil.
For those who don't know about the Halting Problem or Busy Beaver Problem then you should really know about what computers can or cannot do.
I dare say these people have some basic pattern matching, but this is NOT a reason to stop testing.
An Eye for an Eye will make the whole world blind - Gandhi
Which isn't to say that this product is useless, it's entirely possible to have useful approximations or rules of thumb for checking programs out. Heck, that's how people mostly do it, and automating what people do is fair enough.
-WolfWithoutAClause
"Gravity is only a theory, not a fact!"So actually you will end up with a report that cannot mention if you are safe or not, and no way to change the application if you think you are.
Snake oil. Very good against any kind of bugs, esp security bug whatever those may be.
This space is intentionally staring blankly at you
Lets look at the quote on the web page, shall we?
"The alternatives are to laboriously test software or meticulously review source code line by line. But these options are so time consuming and expensive that few companies will do it." (emphasis added)
So how exactly, as the article submitter says will this "help end the debate of closed source or open source applications being more or less secure"? The product page already says that few companies have the time or money to check source code, and how many others do? Sure, it's great to have the source, but when you install apache do you check every single line for buffer offerflows? Of course not. You rely on others doing it, and you rely on others doing it correctly. That may well be a mistake, are you sure someone else will check every revision line by line?
So, frankly, this product contributes nothing to open or closed source arguements, it's simply a nice tool to automate some reviews.
(as an aside, it appears that bugscaninc have made their choice over open and closed source,
Server: Microsoft-IIS/5.0
X-Powered-By: ASP.NET
The webpage says "report is created for each program identifying the specific locations of potential security vulnerabilities"
All programmers know that high level languages create very large binary files. A small program that prints few lines written in Visual Basic, might take hundreds of kilobytes space. Hundreds of kilobytes might mean even millions of lines of assembly code.
Let's take an example. The bugscan reports that there are bugs on lines 24.234, 93.234, 134.834, 342.234, 534.444, 767.835 and 822.511 out of 1.023.890 lines. The BugScan might even report that those lines are from abcd.dll, efgh.dll, ijkl.dll and aaaa.dll. Do you now feel reliefed? No, I didn't think so either. I mean that BugScan might be very useful on low level languages, but when there are ten layers of different libraries between your code and the machine code, I bet the usefulness is not that high.
"This product should help end the debate of closed source or open source applications being more or less secure"
how so? who's to say *this* tool is an official measure of security? its *a* measure. and how would you actually do the comparison? that statement just doesn't make sense.
Looks like a lot of hot air.
The PDF presentation tells us things that we know already (buffer overflow, race conditions, whatever).
Two screenshots show debuggers and disassemblers. Another screenshot shows the "analysis results" of the "tool": "wsprintf: This function is insecure, use another function." Even this info is useless, because wsprintf is insecure only if it is used the wrong way, and I bet the "tool" doesn't check that. Besides, everyone uses std::string these days (or at least should do so).
It's also worth to note that about every University in the world has one or more groups working on topics like "automatic code verification", "code path analysis" and other things. This stuff is nowhere rocket science, but there's a lot to happen until it will go usable by the mainstream of developers.
but complex ones? I imagine what this software does is it scans the binary for things like instances of strcpy calls instead of explicit strncpy calls. Given that the software is likely not executed, how would it be able to catch more complex bugs? How can it find all instances of user interaction which could modify a variable where that variable is used as a parameter in strncpy for example?
Dollars to do[ugh]nuts says that even with a program that gets a clean bill of health, there are still countless bugs undiscovered.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
A friend asked me to help her install an operating system on her brand spanking new PC. I have installed many operating systems - Debian, Slackware, Mandrake and Red Hat among them - and thought I knew a bit about the process. Boy was I in for a surprise!
..... 'cause that's exactly what Richard Stallman and Linus Torvalds got famous for doing!
.rpm, .deb or .tar.gz files on the CD. I've analysed it thoroughly and I found no sign.
The OS she wanted me to install was Windows XP Home Edition. I have never bothered with Microsoft software in the past, not since Bill Gates got all pissy-arsey about people making copies of "his" BASIC interpreter at the Homebrew Computer Club. Grow up guys! You liken your ideas to your babies, but babies eventually grow up, leave home and learn to survive without you! Well, Gates was basically saying that if people didn't pay for their software, programmers would go out of business because nobody would want to create software unless they got paid for it. Right
So I have never bothered with MS stuff, never having felt the need. But I figured, it could not be too difficult to install it, could it?
Windows XP comes on just one CD. First installation attempt sort of worked, but it was a bit flakey and it was a bit slow. And the desktop is just downright annoying - both in terms of colour sceme and general UI. It's a bit like KDE, but not quite. Only one desktop, for crying out loud! And it's slow and crash-prone. Just like Mandrake where you get a really bloaty stock kernel {drivers for god knows what compiled into it just in case anybody ever needs them}. So I figured, first thing we should do was maybe recompile the kernel. Never recompiled a kernel in Windows, never even run the damn thing. Never even likely to now.
Could we find the Kernel Configurator? Could we hell! And the command prompt was useless. It seems to be based on the old DOS command line. And it doesn't understand make menuconfig.
The kernel configurator was not the only thing we could not find. There didn't seem to be any Packages either. You know, stuff like KWord, KSpread and Kate. MySQL, Apache and a scripting language like PHP, Perl or Python. And some simple games. Just the basics. There is something called Internet Explorer, which is a bit like a cut-down Konqueror, but it's nasty to use.
So I'm guessing that the missing configurator probably is part of a Kernel Source Devel package which is not installed by default. In fact, almost no packages seem to be installed by default. And there are no
In the end, I installed Slackware 9 and configured it to look as much like the Live CD as I could manage, but obviously not running everything as root. I can only suppose those missing packages are on another CD which we weren't sent for some reason or another. I mean, she has paid good money for the software, so she is entitled to get it! And the source code. Especially the source code! After all, if we can't check out that source, we have no way to be sure what we're running. It could be sending every keystroke to Microsoft, for all we know!
Anyway, my friend is well chuffed with Slack so I suggested to take the XP CD back to the shop and get a refund. But of course, that might be difficult seeing as she doesn't seem to have the full set. We'll keep you posted as this story develops.
Security problems are often inteoperation issues. You can make sure a program is bug free, but this will not guarantee that your program is not going to fail if the rest of the pieces are not functionning properly. To analyze the interconnections, Open Source is required.
What does this have to do with open source vs. closed? Sure, in theory, every single person who downloads an open source program will review the code themselves to make sure there are no buffer overruns. If they find any, they will of course report them back to the maintainer, who will then fix the bug.
In practice, this doesn't really happen.
As an open source developer, I can assure you that very few people are interested in reviewing other people's code for free. I'm sure the bigger projects, like Apache and Linux, manage to get a good amount of code review -- but then, big closed source projects usually do ample code review, too. As for little open source projects, like the ones I run, you're lucky if people even take a peek at the source. Really, no one is interested. I do not believe that open source projects are any more (or any less) likely to have security issues than typical closed source ones (Microsoft aside).
As long as people are using C, there will always be buffer overruns. C is just that kind of language -- it makes it so amazingly difficult to do simple things (like allocate space for a character string) that programmers naturally take shortcuts (giving the string a static length) without taking the proper precautions (bounds checking). We can't make programmers not be lazy, so the only real solution is to move on to a better language.
I realise that this particular software may not actually decompile or disassemble anything, but this presents a very good reason for making reverse engineering of any software legal in any country: if I'm not allowed to make my own private analysis of a piece of proprietary software out there, how am I to know what it's going to do to my computer? How can I know that it isn't going to take liberties and do damage (such as installing backdoors) on my systems?
To be fair, many software packages I see for Windows machines these days do take advantage of this fact, such as by giving users adverts, invading their privacy, and withholding information to them about what their computer is doing. (One example is Freeserve, a UK ISP: some of their dialling software refuses to tell you what numbers your computer is dialling out to. This can be got round, but it's the principle of the thing...). For the past few years, I've refused to run any software on my desktop machine where source code is not made available, for that reason. If they are prepared to reveal to me what they're going to do to my computer, then I'm not prepared to run their software.
Here's another question: if I have a copy of this software on a machine in a country where reverse engineering is allowed, but then I shell in to that machine (via ssh, vnc, or some other means which will allow me to control that machine remotely) from a country where reverse engineering is not allowed, and then carry out the reverse engineering over that link, is that illegal?
# bugscan bugscan
Segmentation Fault
Hehe
Slashdot Sig. version 0.1alpha. Use at your own risk.
Does anybody have any idea how many binaries are protected nowadays, wich encryption, obfuscation and/or compression?
If a program uses any kind of serial entry, CD check or other kind of 'protection' scheme, you can be sure the makes have run an obfuscation program like 'PEcrypt' on it.
Even then, I don't see this program unpacking unprotected executables that have been packed with UPX or one of the other dozens of PE compressors.
Simply put, this program will have VERY limited uses for normal consumers. The only one who could use it would be the firm who made the program in the first place, before obfuscation/protection/compression, but why would they? They have the source code. A source-code checking program would be MUCH more effective.
I suspect that this product will flag a lot of false positives. After reading the white paper, I believe that the following code would be considered "insecure."
#include <stdlib.h>
#include <string.h>
char *duplicate(const char *input)
{
size_t len;
char *out;
out = NULL;
if (input != NULL) {
len = strlen(input);
out = malloc(len + 1);
if (out != NULL) {
strcpy(out, input);
}
}
return out;
}
Note the use of the "evil" function strcpy().
Once before, while working at a client site, I was installing a 3rd party application. Well, in setting it up and looking for any security holes, I found a pretty large one. Apparently, the client application talks to a MSSQL server using a single account (which happens to have dbo access). Not only did it use a single account for everyone, but the username and password were stored as cleartext in the executable itself! Now granted, not likely that an end user would look there to find this information, but if someone did, and the client did happen to know someone breached the security, the only way to block the intrusion was to shut down the entire system. With the username and password hard coded into the executable, there was no way to change it witout having the vendor make the change and send out a new executable.
Just goes to prove that MS programmers are a dime a dozen, but most of them are worth that too!
It is just a bunch of simple IDA pro plugins and it will give you a false sense of security.
Halvar has published is own open source version called BugScam on sourceforge
they can't even likely tell what code is going to execute, so that severely restricts their options.
.Net
.Net code on installation if you like. then the binary on the client machine is native code, compiled down to machine language on install (instead of execution), and optimized for their particular system (processor optimizations, api optimizations, etc).
odds are they are just scanning for loops that copy until they find a null at the end of a string. (searching for resulting patterns from compiled strcpy as opposed to strncpy).
as most exploits are buffer overflows, this would theoretically catch all of them. it would also catch all sorts of potential buffer overflows that would never be possible given the level of user input (since it's not running the code, or disassembling, it can't know).
but this is why i made my own string object wrapper that stores the bound of a string (and a regexp to define allowed chars) - and then overload the cpy functions to prevent a string from ever copying a single byte more to itself than it should, and always makes sure it's nicely null terminated. but that's just responsible coding.
and it's easy enough to get a compiled binary from
there's an option to 'finalize'
// "Can't clowns and pirates just -try- to get along?"
From the PDF:
Use this logon to scan any binary free for blackhat attendees for the next 60 days...
http://www.hbgary.com/freeblackhat/
Now the people who sound like they know what they're talking about can actually try it out and prove it ;-)
The facts have a liberal bias. --The Daily Show
that sounds misleading. the white paper states that "for example, using strncpy " is a good security practice"
even though strncpy and strncat are actually used incorrectly MUCH MORE OFTEN than strcpy.
Let me explain. People that use strcpy tend to use malloc()ed memory because they
know how it works, and that they have to supply a certain size before they copy in it.
However, almost nobody knows how strncpy works. (as for strncat, i don't recall seeing it correctly used)
i wouldn't call that "safe", i see most strncpy uses as "oh well there's probably an off-by-one there". (i'm not pushing for strcpy() use, it's horrible, i'm pushing for strlcpy() use, with which you know you understand the API, and you can detect truncation easily. google for the paper, and the stupid gnulibc objections)
A near as I could tell, for almost any executable you gave it, it reported there was a bug. The exception is that if you dropped its own executable on itself (even a renamed copy), it reported no bug. That seems pretty accurate to me.
With the username and password hard coded into the executable, there was no way to change it witout having the vendor make the change and send out a new executable.
if it was in cleartext couldn't you just edit the executable, so long as the new username/password was the same length you'd be set.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
The Golden Rule Of Programming:
Never check for an error condition you don't know how to handle.
I mean, what use is this? If you do not have the source, you may use this tool to check for potential security vulnerabilities. The result will leave you with a binary which you cannot change because you don't have the source, and with a list of potential vulnerabilities, which you can't validate without a great deal more of work which you would need to create working exploits. Failure to produce an exploit does not prove that there is no vulnerability, though.
And if you happen to have the source, what use is this tool? There are better tools to find this class of errors on source level.
Kristian
I don't see why this has to be a separate physical device, aside from being able to analyze programs without taking up your CPU time. Why not just sell it as a program?
If you HAVE the source code, use a source code analyzer like my flawfinder tool (or Viega's RATS tool). Source code analyzers can immediately identify where the problem is, and several are freely available. And has been noted elsewhere, the problem with binary analyzers is that they may show where some possible problems are, but it's very difficult to actually FIX the binary without the source code. That doesn't mean this is a useless product; if nothing else, if you're planning to use a proprietary program, a tool like this one might help you begin to understand your risks.
- David A. Wheeler (see my Secure Programming HOWTO)
I have seen strncpy() abused so many times it makes me sick. Like you, I prefer strlcpy(), although I have no idea about the politics behind its adoption in GNU -- I just link in -lgen (the xpg4 lib) under Solaris and code away. I usually have -lgen linked anyhow for strecpy() and such. The Apache Runtime (apr.apache.org) has apr_cpystrn() which is fine, too.
..where a is an array of chars and b is a "known good" ASCIZ char *.
I have actually seen code like this during a code review:
strncpy(a, b, strlen(b));
What the hell is the point of that?
{
char *a = malloc(10);
strncpy(a, b, sizeof(a));
}
At least that won't overrun, but lord help the guy who tries to put five characters into b.
From the same programmer, I have even seen this:
{
static char a[10];
if (a)
strncpy(a, b, sizeof(a));
}
Okay, the strncpy won't overflow, but what's the point of checking if a is NULL or not?
I like to to this:
{
strncpy(a, b, sizeof(a));
a[sizeof(a) - 1] = (char)0;
}
I suppose we should point out for the neophytes that strncpy() doesn't write the trailing NUL if it fills the buffer. So the next time you read the string, you're screwed; you have no idea how long it is.
Do daemons dream of electric sleep()?
I think you'd really want to do at least static analysis per this tool as well as dynamic analysis of the executable to get much confidence from results on a binary with no source. (Rational Purify seems to do a good job of locating dynamic buffer overwrites, etc.).
The static analysis may catch some code paths that aren't typically executed, while the dynamic analysis can catch problems not evident from the source code. You still can't prove an executable is safe, but you can show where it is clearly unsafe. Putting both of these functions into a single product might make more sense.