Slashdot Mirror
US DHS Testing FOSS Security
Stony Stevenson alerts us to a US Department of Homeland Security program in which subcontractors have been examining FOSS source code for security vulnerabilities. InformationWeek.com takes a glass-half-empty approach to reporting the story, saying that for FOSS code on average 1 line in 1000 contains a security bug. From the article: 'A total of 7,826 open source project defects have been fixed through the Homeland Security review, or one every two hours since it was launched in 2006 ...' ZDNet Australia prefers to emphasize those FOSS projects that fixed every reported bug, thus achieving a clean bill of health according to DHS. These include PHP, Perl, Python, Postfix, and Samba.
Now if they would do the same to Microsoft. Oh yeah...
I stopped reading after they called it "The PHP."
"Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth
Do they mean fixed or fixed?
You can't ever say that proprietary software is secure, because there's no way to prove it. With Open Source, you can come a lot closer to proving that it is secure, because you can employ every security test that exists.
The fact that a coverity scanner bug is reported doesn't mean it's an exploitable security flaw.
Bruce
Bruce Perens.
A third party came in, identified bugs, and they're being fixed...
Meanwhile, any third party can't peruse the code of the non-open software alternatives and can't find the bugs...
Congrats OSS!
(PS-- anyone know- do they count the same bug appearing multiple times as one bug or many?)
Just thought of this: Make it stipulation of GPL that if you publically report bugs or bug counts in GPL software, that you must also produce a detailed account of how to reproduce the bug, and you must provide that report to the maintainer of the current source (who you got it from, or the root source as listed in the code). Possibly a two-week window between notification (and acknowledgement) and publication. In a way, firms are profiting from the GPL software, but I can guess they aren't reporting all these issues. Sure you could fix stuff in private - but to publically announce it, you should have to allow everyone the oppotunity to fix it.
meh
I guess this benefits open source software since any bug fix is a good thing, but why on earth would the department of homeland security be studying software. shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?
Samba was not listed as having a clean bill of health, there were bugs that have yet to be verified.
I think the bug rate for alot of these projects (Linux, Samba, etc) remarkable
Make SELinux enforcing again!
Uh.. from the article, the software is called "Prevent Software Quality System"... Wow, I can't think of a bigger misnomer for something that should help improve software quality. I sure don't want to prevent software quality in my own products.
I thought that the Bush Administration could do nothing good by Slashdot standards?
http://scan.coverity.com/
Although I understand what you're trying to say, it does seem a little irrelevant.
I'm a software security engineer. I can look at source code and tell you if it has some bugs in it that I would consider relevant to security. If I can't find any, I might tell you that it is more secure than if I could... but that's doesn't mean it is secure. I'll never tell you it is secure, because testing simply can't give you that. I can do this on proprietary software or I can do this on Open Source software.. the only difference is that, with the Open Source software, I don't need permission from someone to do the testing and other people don't need permission to check my work.
Does this mean that more people will check the Open Source software for security flaws? Not necessarily. It completely depends on whether or not someone has an interest in the security of that particular bit of software. Even assuming a similar level of interest in the security of comparable proprietary and Open Source software, there's no guarantee that those who have an interest in testing the Open Source software for security flaws will report back the findings. They may simply decide that the Open Source software is too insecure for their use and go with the proprietary solution - assuming they can have it similarly tested by a trusted third party.
All in all, the assumption that Open Source software is more secure than proprietary software is most likely true, but there's no hard data.. because the stats on the insecurity of proprietary software are guarded secrets - and that's probably the best reason to assume that proprietary software is less secure.
How we know is more important than what we know.
According to McAfee recently (http://yro.slashdot.org/article.pl?sid=08/01/05/0215201) and Microsoft et al, having your code exposed lets the bad guys exploit it's vulnerabilities. Of course if or when a weakness is taken advantage of, it would likely be fixed vary quickly through the FOSS community, instead of on the first Tuesday of every month like as in Microsoft's business model.
I checked out the Coverity website and saw on the list of projects the aalib ASCII art library which according to the history hasn't been updated for something like 7 years.
Damn we better protect ourselves from Terrists hiding their WMD's in ASCI art
I am Slashdot. Are you Slashdot as well?
I'm not from USA, and I'm wondering if the Department of Homeland Security don't also have the code of some proprietary software? To get a certification for homeland security, you don't have to give your code to the government?
Insightful!
DHS is a dysfunctional mess for the most part when it comes to INFOSEC/IA. They negative for the sake of negativity approach does not surprise me in the lest. If it's any comfort, DoD takes FOSS quite seriously and makes use of many great FOSS tools and platforms. It really is a cultural difference. Those in the DoD that get the job done are prone to use 'the best tool for the job'. FOSS is a gimme in many (and an ever increasing number of) cases.
The article did not seem to give any data on false positives. A story here has Coverity claiming a 10% false positive rate. But there is no independent confirmation. It would also be interesting to know how hard it is to prove a false positive vs. how hard to fix a true positive. In other words, it it worth Coverity's time to further reduce the false positive rate.
This seems like a genuinely useful activity for DHS, certainly more valuable than x-raying my shoes and confiscating my saline solution.
What I'm listening to now on Pandora...
It is somewhat sad that copyright (the legal muscle in GPL) only covers the writing of software. The testing and verification - which is often as important, if not more important has no legal protection. It is the testing, rather than the coding, that really adds the value in OSS.
Engineering is the art of compromise.
Indeed. FTFA:
One can only speculate about the, er, source of their discomfort.... 8^)
1 per 1000 lines is even more impressive as an average across all 180 FOSS applications tested. Most impressive of all are the highlights:
Even some of those with more bugs have at least responded well:
And my favourite 'backslider' of all, OpenVPN, has yet to fix 100% of the bugs found during this exercise. Of course, that's only 1 bug in over 69,000 lines of code....
These results should be viewed as excellent, by and large. This doesn't mean all this software is bug-free, just that there aren't a lot of easily preventable bugs in the code base. Most encouraging, though, is how fast they got addressed and fixed by the healthier FOSS projects.
Crumb's Corollary: Never bring a knife to a bun fight.
Comment removed based on user account deletion
Most people only read the summary.
From TFA:
The popular MySQL open source database was not included in the scans for reasons that were not immediately evident.
Any suggestions as to why MySQL has no results? I'm stumped and wondering why one whole corner of a LAMP foundation was left unchecked.
Does this mean that more people will check the Open Source software for security flaws? Not necessarily.
It sure is nice to be able to do it if/when you feel like it, though!!
i have tested and evaluated that particular tool as well as several competitors; they all had high false positive rates.
Does anyone want to say thanks to the US government for this service?
c) It is not enforceable in most jurisdictions. In the US, and I assume most of the "free world", you can't prevent someone from talking about your products publicly. You can have them sign an NDA, but that doesn't work for publicly available software. McAfee tried something like this some time ago, stipulating in the EULA that you can't benchmark their software. It got shot down in court.
I haven't seen one in years but doesn't Microsoft's ULA have a clause that you can't publish a review of the software without having MS's approval? I think a year or so ago there was an article on /. about it.
FalconShould there be a Law?
I submit that people who are only looking for security flaws don't have a motivation to develop a deep understanding of the software. People who are out to modify the software do. And thus there are not just more eyes, but better eyes with Free Software.
There is a class of mathematically provable software languages, and you might be able to say with surety that programs in them are secure. For the languages we usually use, you can only say that you have tested them in the ways you know of. And only a person with access to the source can say that. If you want an independent asessment, Open Source software won't stop one from happening, and won't hinder what can be said with NDAs. That's why I think it's more secure.
Bruce
Bruce Perens.
I guess this benefits open source software since any bug fix is a good thing, but why on earth would the department of homeland security be studying software. shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?
Such as hurricanes?
FalconShould there be a Law?
Yes they said that, but you don't really believe it, do you? If so, just look up "security by obscurity" and read about it. To give you a clue, the unavailability of source has not prevented 100,000 Windows viruses.
Bruce Perens.
I don't really see how it's irrelevant - if a "security defect" exists but cannot be exploited (i.e. if there's a buffer overflow bug but it deals with internal data or data that's already been thoroughly sanitized), it does not present the same risk as a bug that may be easily exploited, for example in the input sanitizing code. It's not really clear how many of these bugs are of each type, and I think it's significant that the phrase "security defect" was chosen instead of "security hole" or some other phrase that is more commonly used for known significant risks.
Of course, "cannot be exploited" is relative - no matter what you do, there's always a real possibility that someone's going to come up with a creative way to get their data in the wrong place at the wrong time and break all your nice sanitizing code and layers you've erected over that heavily protected buffer overflow, so the real resolution is of course to fix it. Still, I think it's an important distinction, especially when dealing with statistics.
There's an old saying that says pretty much whatever you want it to.
DHS, certainly more valuable than x-raying my shoes and confiscating my saline solution.
What would be more valuable would be to get rid of DHS.
FalconShould there be a Law?
Although I generally agree with the belief that FOSS probably yields better security, I think FOSS has a different characteristic of vulnerability than closed-source software. Specifically, the "ease of exploiting" a vulnerability is increased along with the ease of modification of the software. The most understanding of the system that's out there, the easier it is to take advantage of a vulnerability. I realize that "security through obscurity" is not something you want to depend on, but it is a real effect that has to be considered if you're going to compare the likelihood of actually being attacked versus simply having a vulnerability.
For example, MacOS and Windows had a similar number of critical security patches last year. However, there were dozens of Windows viruses and hundreds of thousands of compromised machines, and zero MacOS viruses. Thus, while a certain measure of vulnerability is comparable, the likelihood of actually being attacked is infinitely higher with Windows.
E pluribus unum
I think your logic is a bit confused. The fact that viruses can be created without reading the source code does not prove that there's no value in keeping the code secret. It's like arguing that there's no point in locking your door because 100,000 houses with locks were broken into.
No-one was debating Bruce's last point about Coverity returning many false positives.
As for the use of terminology, excuse me for using an accurate term like "defect" instead of a more popular colloquialism like "hole".
How we know is more important than what we know.
In many cases they can see the code, albeit under pretty restrictive NDAs.
Which is why I consider open source more secure, anyone can find a hole and anyone, well programmers at least, can fix the hole. With closed source code a review can be restricted from informing users of said code the problems it has.
FalconShould there be a Law?
Make the licenses as restrictive as you please, but at least give people the opportunity to know what they are using. Like listing ingredients on processed food, it's good to know that I'm not consuming something that could possibly do me harm (or be beneficial).
There is a level of comfort in dealing with openness. It seems like that's why so many politicians and business leaders are not trusted; because they hide behind PR vetted canned answers (security through obscurity if you will), rather than being articulate and just admitting outright when things aren't working the way they should.
Oh man... Bruce Perens. What a pleasure. I couldn't have said it better myself.
(Actually, I was going to make fun of proprietary software for the general idea of having source unavailable).
More to the point though, I received a lecture on this in a Software Architecture course a couple years ago and it struck a nerve. Even if you never need to review 99.9% of the code you run, it is nice to be able to look through the 0.1% that might be helpful for you to gain a better understanding of what is going on. And that is only possible with Open Source.
Cheers.
Support the 30 Hour Work Week!!!
Well put. I wrote about this topic in my recent security concepts ebook:
http://www.subspacefield.org/security/security_concepts.html#tth_sEc24.5
A more apt analogy would be: There's no point in locking your door using a limp spaghetti noodle because a limp noodle makes a completely ineffective lock.
Actually, I read that as "We won't tell you how many bugs there are, our customers would not like it". They could well be inflating the reliability of proprietary software for their customers sake.
Bruce Perens.
So in other words, this thing started in 2006. So if Big Daddy Gubment had not come by with what's essentially a bailout of FOSS, it would STILL be a buggy mess.
Kind of hilarious, how no matter how much of an insecure, buggy, crappy mess FOSS proves to be, they still whine about Microsoft.
Guess it's easier to point the finger than it is to get your own house in order.
Bruce
Bruce Perens.
Not only did the article say much like its commercial counterpart, but most of the numbers it shows are actually good for open source software.
For instance, most of the projects discussed had less than 1 bug for 1000 lines of code. For instance, the Linux kernel had .127 bugs per 1000 lines, and that on over 3 million lines of code.
Also, the article talks about key projects, such as the glibc (which is basically used by everything on a Linux system) that already fixed all the issues.
Even something huge and complex as Firefox has already fixed half of the issues, and is showing progress on the rest of them (by the fact that some were already verified).
Overall, I didn't get the half glass empty tone that the summary is implying. And what I found strange is that even the comments on the site itself, and many of them on /. itself, are also taking the pessimistic view.
I thought that this news are great for open source software. Shows that it has less security issues than average, that the issues are fixed quickly, and still that some programs are certified by a company for use in security related departments such as the DHS. What could be better than that?
Not too many people would be interested in breaking into a lock on a door (smashing a Window to get into the house is most generally used by non-government intruders).
The greatest value in keeping code secret is making sure it cannot be easily re-produced, and thus subverting other individuals or companies from using it without authorization. It's much like music and DRM: in the end it is the licenses which are enforced in the court of law, and not so much the code itself (the "locks" if you will), that will be able to protect the intellectual property.
Yes there may be value in keeping code secret, but I would argue that the value is minimal compared to the benefits of keeping it open.
First off, prevent is not strictly a security flaw static-analysis checker. It is a static-analysis checker that checks for all sorts of defects. Some of which are directly related to security. Second, I have used prevent extensively over the past year and have found it to be an invaluable tool. It has a pretty low false positive rate and fixing the defects it finds means your code is better. On the code I work on, I find that we have a much lower defect count. But we also have pretty mature code and we really do attempt to make it as bullet proof as possible. But we still have defects.
My experience is with the C/C++ version of tool. We have also been evaluating the java version of the tool and it is good. But some of the free alternatives like findbugs are still better. I would use findbugs w/ prevent for java if I wanted good coverage.
"For example, MacOS and Windows had a similar number of critical security patches last year."
Willing to stipulate for the purpose of this discussion.
However, there were dozens of Windows viruses and hundreds of thousands of compromised machines, and zero MacOS viruses.
Likewise willing to stipulate.
Thus, while a certain measure of vulnerability is comparable, the likelihood of actually being attacked is infinitely highder with Windows.
I would suggest this doesn't necessarily follow. It could be. It could also be that while both fixed the same number of holes, the percentage of holes fixed was different. It could be that x holes represented 85% of the mac holes, whereas the same exact number x was only 13% of the available windows holes.
Not saying one or the other interpretation is true. Just that the facts don't necessarily lead to the conclusion posited.
Anyone can buy a re-key kit for Schlage locks at the Home Depot. Upon opening the cylinder of the lock with that kit, you will discover that (this is approximate, I don't have the lock in front of me) there are 5 pins, and 5 possible levels per pin, and that the minimum number of possible key patterns might thus be 5 ^ 5 or 3125. Which is enough that nobody's carrying all of the possible keys around and will have time to go through them at my front door.
The re-key kit comes with a set of two identical new keys that do not use the same pin length twice, and thus its number of possible patterns might be 5 * 4 * 3 * 2 * 1 or only 120. Uh-oh! Better not base the keys for my house on the master from the re-key kit! And shame on them for not saying this on the box.
See the benefit of being able to examine how they work?
FYI, yes I know about lock-picking, there's an alarm too.Bruce
Bruce Perens.
Analogies have their limits, so we shouldn't try to take it too far.
Even those who historically have critized "security through obscurity" never suggested that publishing their design or secrets would lead to better security, but rather that you can't assume your that your design can't be cracked.
Of course, the preferred approach is "security through design" which has nothing to do with correcting bugs. The latter could be called "security through maintenence". Thus while we might argue about whether closed or open source produces better design, examining source code for bugs can't compensate for a design that is insecure.
I see you're trying to imply that Windows is insecure, but I don't see what that has to do with the issue of security through obscurity.
If it's this easy to find security bugs, why aren't there tons of people doing this for every project? You mean you just set up a piece of software, let it run a scan and you've done your security testing?! SERIOUSLY?! Jesus. Seriously, why wouldn't the linux kernel maintainers, for example, run the scans on every release, then??
You're wrong about that. For example, NIST, a US government standards agency, is calling for proposals for a new cryptographic algorithm for government use. Their specification requires that it be publicly disclosed (and royalty free, too). This is so that they don't pick a weak algorithm. They want any known or theoretical problems to be pointed out to them. Most certainly NSA participates in building that sort of specification.
Bruce
Bruce Perens.
"Fact is anybody can dis-assemble a lock. And of course people can dis-assemble code"
There are probably few people who have even read every line of the Linux kernel - imagine trying to dis-assemble Vista looking for vulnerabilities.
"The greatest value in keeping code secret is making sure it cannot be easily re-produced, and thus subverting other individuals or companies from using it without authorization."
Perhaps, but your statement says nothing about security issues.
"Yes there may be value in keeping code secret, but I would argue that the value is minimal compared to the benefits of keeping it open."
Well, there are arguments for keeping code secret and for making it open. My point is that there's little evidence that keeping the code open improves the security of the code. Nobody can reasonably argue that not having the source code makes it easier to create exploits.
Just has to do with coding methodology. strcpy is insecure, strncpy is more so. strncpy(src, dst, sizeof(dst)) is more secure than strncpy(src, dst, size_of_dst). Those are easy to fix security bugs. Other security bugs are harder to find as you have to trace the myriad of states the app can be in during mem writes.
I said historically. I've worked on military crypto systems as recently as 10 years ago and the details were classified. I wouldn't assume that every government agency takes the same approach. Even the individual branches of the military often go their own way.
In this post-911 period I've seen a trend toward more secrecy rather than less. For example, the documents that described the military's UHF DAMA waveforms used to be freely available on the Internet, but they aren't now.
It's barely a speed bump for the evil hackers who feed garbage into programs to crash them, then poke around in a debugger to find where they broke, then write some machine code to take advantage of the bugs. Thinking that lack of access to source code is anything like a "lock" is just self delusion.
That's much easier to read.....
Boffoonery - downloadable Comedy Benefit for Bletchley Park
You're reading too much into my analogy (or perhaps it was flawed). I didn't mean to suggest that STO is a like a "lock". I was just saying that the fact that Windows has been exploited doesn't prove that STO has no security value.
There are numerous refutations to your "never suggested that publishing their design or secrets would lead to better security". Many experts have said precisely that.
An IT Security article on full disclosure states that as early as the middle of the 19th century locksmith Alfred C. Hobbes thought full disclosure was important to clear up the rash of lock picking people were experiencing. It goes on to discuss exactly why full disclosure works so well.
David Wagner says in an article on security: "Today, many security companies are strongly resisting this, and I think they will need to learn to accept and embrace public scrutiny as a natural and necessary part of security systems." -- David Wagner and Ian Goldberg are the ones who cracked the security of the SSL layer in Netscape 4.
IEEE article abstract stating that full source code access can have "real benefits for security", although that's not automatic and it has to be done correctly.
Bruce Schneier -- yes, THAT Bruce Schneier -- has an article on his blog that starts "Full disclosure -- the practice of making the details of security vulnerabilities public -- is a damned good idea. Public scrutiny is the only reliable way to improve security, while secrecy only makes us less secure."
Is that enough or do I need to go to the second page of this Google search?
BTW, DJB thinks that both full disclosure and isolation of trusted components are absolutely vital. He's the guy who won the right for Americans to export cryptography technology in court against the Department of Justice. He also found a timing attack against OpenSSL's AES cipher and his Unix Security Holes class of 16 students turned up 91 previously unknown holes in one semester.
As for "Security by design", that helps. However, with many programs being written in languages which allow null pointers, stack overflow, buffer overflow, and array overflow the design can be as secure as you want and the program can still be crashed. In some cases arbitrary code can still be executed. Address randomization, NX bits, run-time bounds checking, and automatic memory management can go a long way. Sanitation of inputs, static analysis, time padding, and more still have to be considered in some cases.
The tests Coverity is running are an example of static analysis. If there's a C routine that can be coerced into smashing the stack or overflowing a buffer in the heap, that can often be automatically caught and reported. Memory leaks often can be, too. They're probably also able to do at least rudimentary checks for sanitizing input values.
Of course, language choice is a design decision.
Thank you DHS for the contribution to FOSS!
We get all the bug fixes, and it will become that much more robust.
Too bad that Windows will never get this kind of review.
It probibly has a few less bugs per line,
but not much hope of getting those fixed.
On second thought, Mr Allen, I challenge you to compare!
I am willing to bet that FOSS software,
just because of its nature of peer review,
and from my experence of reading ALen Cox's work on the Kernel,
that it has less bugs than Windows.
As well, open-source software makes it easier to find built-in vulnerabilities (like the Jap proxy software was found to have a secret back door for the German police by examining it's source).
That should be so obvious as to not require stating, by McAfee or anyone else. How large that risk really is can be debated, but its existence is as certain as the sun rising tommorow.
Fixing it some uncertain time after it has been exploitied is fine by the [relatively] sloppy standards of the FOSS community. But neither having it fixed the first Tuesday nor some uncertain time later is of much consolation to the guy who suffers business or data loss.
Less encouraging is that they existed in the first place - doubly so since all the software you list is more-or-less 'mature'.
It's also a performance decision and a pragmatic decision in the fact that many languages don't bootstrap on bare hardware. Let me know when you have an OS kernel running on the Core 2 Duo written in Java, Python, Erlang, Eiffel, Haskell, or Ruby. ;-)
Yeah, lots of software gets written in assembly, C or C++ that probably should be in something else. No, nothing else is able to take their place for everything just yet.
Nobody can reasonably argue that having the source makes it harder to find where exploits might target.
If you believe security through obscurity is ineffective, I'd like you to hand over all your encryption keys and passwords: after all, there's no need to keep them obsecure! Eventually, at some point, all of security comes down to obscurity. Security IS the concept that you are hiding (preventing access to) some sort of 'secret'.
Something cannot be secure without obscurity.
It's like the physics concept: Observation is Interaction.
"It is possible to commit no errors and still lose. That is not a weakness. That is life." -Peak Performance
Just has to do with coding methodology. strcpy is insecure, strncpy is more so. strncpy(src, dst, sizeof(dst)) is more secure than strncpy(src, dst, size_of_dst). Those are easy to fix security bugs. Other security bugs are harder to find as you have to trace the myriad of states the app can be in during mem writes.
/* We'll just fit as much of the translated error as possible into this buffer */
strcpy is NOT insecure. It can be used insecurely.
But congratulations, you've just turned what could have been a borderline ok strcpy(src, dst) (ought to have been criticized at code review as the names of the variables are confusing) bit of code into (probably) a crash and definitely a buffer overrun if sizeof dst is larger than sizeof src.
I have lost count of the number of bugs I've had to fix after someone changed a perfectly good strcpy into strncpy. A common mistake is:
strcpy(dst, src);
becomes
strncpy(dst, src, sizeof dst);
and then you get a bug because only the first four characters of src appear in dst followed by garbage.
Of course, then it gets changed to
strncpy(dst, src, strlen(src));
because the original programmer did know what they were doing and the buffer was big enough.
Eventually we get to the brilliant:
strncpy(dst, src, strlen(src)+1);
Fantastic! What an improvement! And yes, it really does happen in what was once good production code because some idiot has heard that "strcpy is insecure".
Another one I've seen is:
dst = malloc(1000000);
strcpy(dst, "MESSAGE");
gets changed to
dst = malloc(1000000);
strncpy(dst, "MESSAGE", 1000000);
Yup, instead of writing 8 bytes, we'll write one million bytes because strcpy is insecure, but we won't fix the missing check for NULL. (there's a fairly good argument for not checking the return from malloc in much production code - if malloc actually fails then you're already so far up shit creek without a paddle that it's probably impossible to recover gracefully anyway. Obviously different considerations will apply if you're controlling a nuclear power plant than if you're writing a game)
strncpy is NOT a replacement for strcpy with a length parameter. Unfortunately strncpy has a very bad name, it should be called something like meminit_from_str() as strncpy ALWAYS writes n bytes and doesn't always write a null terminator. (I've also had to fix bugs where someone has replaced a correct use of strncpy with a version that guarantees to write the null)
strncat is a possibly safer replacement for strcat. However, the length parameter is so tricky to get right that I've seen cases where someone originally wrote strcat safely, that got changed to strncat "because it's safer" and then a bit later another change was made that caused a crash because the original change to strncat got the length parameter wrong.
extern char error_msg[][40];
char error[64];
strcpy(error, "ERROR");
strcat(error, error_msg[e]);
becomes
strncpy(error, "ERROR:", sizeof error);
strncat(error, error_msg[e], sizeof error - 6);
becomes
strncpy(error, get_translation("ERROR:", lang), sizeof error);
strncat(error, translated_error_msg(e, lang), sizeof error - strlen(error));
of course, even more common is to miss the -6 or strlen(error) completely than to remember the extra -1 that is required on the length parameter.
(The man pages are IMO, confusing for strncat as they usually say something along the lines of "appends at most n characters")
Tim.
God said, "div D = rho, div B = 0, curl E = -@B/@t, curl H = J + @D/@t," and there was light.
This is because the security problems with PHP aren't bugs, they designed it that way.
But hey, don't take my word for it.. go have a chat with your friend Theo de Raadt.. he'll give you the skinny on how terrible the majority of C programmers are when it comes to security issues. And don't get him started on the so called "safe" languages.
How we know is more important than what we know.
slightly offtopic: KDE has 4.7MLoc, and Gnome only 0.4MLoc. I thought they were filling the same niche, so how come there is such a difference in line count?
Ouch... Talk about throwing to the wolves. However, if you want to have a well-informed (albeit somewhat lacking in social graces) person to comment on the state of security in general, he'd probably be quite a good choice. As to the state of security sense/awareness in programmers, he'd probably be one of the best
Coz eternity my friend, is a long *ing time.
Great, great post. For an alternative to strncpy() etc,
see <http://www.courtesan.com/todd/papers/strlcpy.html>.
slashdottagsshorterthanhaikunewartform
But it's not a first time scan. Amada was checked long ago, and FreeBSD has been running a Coverity server since Jan 2006.
http://www.linuxtoday.com/developer/2006031800826OSCYDV
http://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/coverity.html
Worst of all, these articles haven't disclosed the classes of software issues detected. I'm sure huge classes of deadlocks and other system-wide issues go undetected. Even if the point of Coverity is to conduct system-wide analysis, I'd still say large classes escape.
I'm sure it can't detect that a Linux device driver sends the wrong byte to the wrong register of the hardware it supervises.
I think from the DHS perspective, they want to close as many bugs as possible that their adversaries could find by mechanical means. Finding deep bugs is real work, and wouldn't support a multi-vector concerted attack without massive preparation of the kind that HUMINT can usually detect.
[ .. Today's slashdot is linking to an interesting article on Information Week, the analyzed took the results of an DHS (Department of Homeland Security) Scan about some used FOSS-Software. ]
[ Today's slashdot is linking to an interesting article on Information Week, the analyzed took the results of an DHS (Department of Homeland Security) Scan about some used FOSS-Software. .. ]
http://blog.coldtobi.de/blog-website/2008/01/09/not-all-defects-are-security-bugs
While the numbers for the Linux kernel look pretty good, there's a bit more to this story, I think.
IIRC, Coverity is the commercialization of the "Stanford Checker" static analysis tool. By most accounts, it's a pretty nifty tool. Back when it was still a research project, some of the folks working on it would run it against different parts of the kernel and post the bugs it found to LKML. It was a mutually beneficial relationship--the kernel people fixed a lot of bugs, and the Stanford folks got analysis on the false positives which they then used to improve their tool.
I don't know if the Stanford Checker was used on other FOSS projects.
In the case of the kernel, at least, I would expect that some of the low defect rate can be attributed to that previous experience. I imagine the tool has improved over time, and I know there's been a lot of code added to/changed in the kernel, but still, the past history must help those numbers out significantly.
The solution to the security problem is to use The Google on the Internets, which is just a series of tubes, and then all will be secure!
There's a bit of difference between a closed source system that is trivially reverse engineered and a properly managed encryption key that would take millions of times longer than the age of the universe to decipher.
Most locks are absurdly pickable and highly susceptible to sledgehammers. They are barely a speed bump to someone intent on going through a door. Thinking they make you safer is largely self delusion. (the most reliable property they have is that someone wanting to gain rapid entry will have to make some noise to do so)
Nerd rage is the funniest rage.
I wish I could find the link but I remember being very impressed by an article about a code shop that did the flight controls for the space shuttle. There was some amazing dedication to quality going on there.
And that's exactly my point. The term is completely meaningless. It's an attempt to obscure the issue- that is, that security IS obscurity, and without obscurity you cannot have security.
Whining about 'security through obscurity' is fallacious in the extreme.
Now, some methods of obscurity are indeed better than others. But an Arquebus is arguably inferior to a crossbow; that does not mean that gunpowder weapons are useless.
"It is possible to commit no errors and still lose. That is not a weakness. That is life." -Peak Performance
> Something cannot be secure without obscurity.
Obscurity is not the same as keeping a secret. Obscurity means you _do_ give out the information, just in a really hard to understand way.
Real security on the other hand uses a real secret you do not give away in any way.
All the projects listed in InformationWeek, where they included numbers have lower bug per line counts than the scanners expected. Generally one FIFTH of what they expected or lower.
What does that say about popular FOSS projects versus commercial stuff?
It'd be interesting to see numbers on much less popular projects. And closed source products, of course.
I thought that one of the major arguments *against* checking malloc's return value is that the most common OSes lie about how much memory they have available and will gladly let a program think a malloc has succeeded and then terminate the program when it tries to use the memory if the OS is running low on memory.
It's not meaningless the way security professionals use it. You'd just like to redefine it to be meaningless with your own private definition. Have fun.
KDE: 4712273/(1554+25+65) => 1 fault per 2866 LOC
GNOME: 430809/(357+5+214) => 1 fault per 747 LOC
Wow. I did not expect GNOME to have 4 times more possible faults, maybe it's because of their preferred language C instead of C++?
Possibly, although the definition is probably academic; as I said before, interaction and observation are really the same thing.
The only way security is not obscurity is if the 'secret' does not actually exist.
But even so, it's far less clear here.
For example, say you have a lock (an analog for encryption) and do not give me the key. Obscurity here provides you with protection: with enough time examining the lock I can deduce the key.
On the other hand, you can refer to the example of Enigma; Allied cryptographers were able to break the encryption and so had access to the obscured messages. However, they could not act upon the contents of the message without revealing that they had, in fact, decrypted them (barring some incidents which provided plausible deniability, admittedly).
The result is that while it is possible to keep a 'secret' without obscuring its contents, doing so is essentially meaningless, as there is no way to act upon its contents without revealing it in some form- at which point the focus drops back to obscurability.
Which is why the dichotomy between 'secure by design' and 'security through obscurity' is so fallacious. Any security system designed to keep intruders out requires some mechanism of doing so, and by design, requires that mechanism to be presented to the intruder. The mechanism is hiding the message, and so is itself a mechanism of obscurity.
A security system (doesn't have to rely on obscurity, as seen above, but reasonably will be), and it can also be secure by design. The two concepts are hardly contradictory, and the latter is not a negative attribute even if it exists.
Badly implemented security through obscurity is bad. But then, so are badly-implemented firearms.
"It is possible to commit no errors and still lose. That is not a weakness. That is life." -Peak Performance
"There is a class of mathematically provable software languages, and you might be able to say with surety that programs in them are secure."
I don't think this is the case. The problem is that most people think "secure" is a binary principle - it's there or its not. But in reality "secure" deals as much with expected interactions as it does with any hard principles. A mathematically provable software language will only prove that the software meets a certain specification. The problem is that the specification may itself contain bugs or be insecure. Also, it may be perfectly secure in one context, and perfectly insecure in another (for instance, if they assumed incorrectly which other parts of the system could be trusted).
Engineering and the Ultimate
In addition, there is the question of "exploited by whom". For instance, let's say that there is a bug which allows one Postgres user to do queries on the table of another Postgres user. Should it be fixed? Sure. Is it a big problem? Not unless you're on a shared virtual hosting platform. And even then, its somewhat mitigated because the only people who have access to it are paying customers (i.e. their identity is known).
A "security bug" does not mean "hax0rs are going to own you", but it may mean something like, "if someone who you trusted enough to give permissions to your box were going to try to screw you, they could do it in this way".
So, even if it is exploitable doesn't mean that it is necessarily exploitable in a way that would leave people vulnerable to strangers.
Engineering and the Ultimate
I once wrote interrupt handlers and read "legacy binary files" in a dialect of Modula-2, as 2 realms that "only C can do well," and I didn't jump through terribly awkward or unsafe steps to do so. The problem was in that word, "dialect." While the language could be good and useful, the original definition was not, and apparently its inventor had insufficient interest in making it so. Hence only dialects became useful, and of course no 2 were the same.
Beyond that, there's the "common idiot" mindset that dislikes strong typing, even if there is a structured way to break it when necessary, predeclaration of variables, interfaces, and stuff like that that we'd generally be better of with if we had.
As a result, C and C++ are pretty much it, and programming languages that could help us do better, or been improved to do so, are practically dead on the shelf.
The living have better things to do than to continue hating the dead.
Indeed, you can never be completely certain that code is free of security holes - you can prove one exists, but you can't prove one doesn't exist. At least not using an automated scanner. So the ZDNet article is nonsense: 'its first list of open-source projects that have been certified as free of security defects'.
-- Ed Avis ed@membled.com
As an aggregate they know enough that they produce the vast majority of security bug reports. Only a tiny minority of those reports come from "experts" performing security reviews. If we have to rely on experts, we're not going to have much security anywhere.
Theo, unfortunately, has contempt for the programming abilities of most people. It is a valid point, however, that most C programmers can learn more than they know now about how to write secure software.
Bruce Perens.
I haven't tried this, and indeed there isn't much real work going on in provable software languages these days. But I think that it would be possible to set theoretical constraints on a program such that it serves data and does not allow it to be modified. There might be a good Ph.D. paper in it for someone.
Bruce Perens.
Microsoft does open up its source code to the government for review. They refer to the program as the Government Security Program.
Also see their Shared Source Initiative.
I'm so much of a rabid MS hater that I'm writing this post from Firefox 3b2 on Windows XP. Get real.
;-)
;-), and they obviously have advantages over many smaller OSS projects.
Where did you pull the 1% of OSS users being programmers from? Your ass? You didn't even cite your own ass? How rude!
Yeah, there aren't enough world-class programmers to go around the millions of OSS projects out there, or even the most popular hundred thousand of them. Maybe not the ten thousand most popular. Yet over half the patches for the Linux kernel come from people other than the core development team.
In fact, the top submitter of changesets into Linux 2.6.20 only accounted for 4.8% of them. The top 20 contributors accounted for 28% of changesets. Similar numbers pop up by number of lines added. Linus only personally signed off on 13% of the changes in 2.6.20 so there's a good spread there, too.
The people developing the Linux kernel aren't just weekend coders in their parents' basements. Red Hat, IBM, Novell, Intel, Oracle, Google, University of Aberdeen, HP, Nokia, SGI, Astaro, MIPS Technologies, MontaVista, and Broadcom were among the top 20 sources of changesets. Stats of 7.7% for "no employer" and 25% for "unknown" appear, along with a few lesser-known companies. Add Sony to the list of employers of contributors by lines of code. Put Freescale in the list for the versions in the year in which versions from 2.6.16 to 2.6.20 were developed.
In all, 65% of the changes to the Linux kernel for version 2.6.20 was from corporate development. Over 1,900 people had patches make it into the 2.6.20 version of the kernel alone.
All these statistics on who develops Linux can be found at LWN.net's article called "Who Wrote 2.6.20?".
How many companies write and vet the code at Microsoft? Yes, I'm sure there are a bunch of dedicated people at Microsoft, and they do a pretty good job at making a usable OS. They're getting better about security. It's my opinion that Vista's kind of a mess particularly because they're having trouble designing for both usability and security from the ground up. They'll improve on that, too. I don't hate Microsoft's developers (maybe their marketing and legal departments
However, the biggest OSS projects really do have a lot of people who are highly skilled professional programmers writing their code. They also have an advantage of being able to attack issues most important to their varied employers using skills and development methods different from those at other corporate contributors.
It's not a black or white issue. Microsoft's got pros and cons, and so does their software. OSS has pros and cons. I have two PCs at this desk. One's XP Pro and one's Linux. I use both every day I'm in the office. I also use Linux servers and I have a Mac at another desk. At home I have XP, Linux, Solaris, Mac, and OS/2 (the OS/2 is for fun). My wife's PC has XP on it, but she can use the Linux box when she needs to. She's not an admin level user, but she can fix some issues on Windows just from having used it so much for so long.
To bash MS when they really screw something up isn't to be a "rabid MS hater". To praise them when they do something well isn't to be an MS fan. The same's true of OSS projects. Most people want their software to meet their needs and don't root for one "team" or another. Most people who do prefer a particular project are still willing to give other projects their due respect. There are very vocal fanatics in every camp, but just because they're loud and quicker to spout doesn't mean they're actually that numerous.
Depends on the OS, and whether or not strict memory accounting is enabled. Even with non-strict memory accounting there are cases where malloc() will fail.
If you _really_ want to make sure that you have the memory available, malloc() it, verify the return code, then mlock() the address range. The truly hardcore will use mlockall() and pre-allocate all memory including their stack and signal stack, writing to each page of memory at startup to ensure it gets faulted in. This is rarely necessary, but useful in certain circumstances.
Of the 236 defects, 228 have been corrected, said Maxwell in an interview.
Proof that waterboarding developers is a valid part of any QA methodology.
If you haven't made a developer cry, you've wasted a day.
Nobody has. The question is whether having a potentially larger number of people who might examine the code, outweighs the disadvantage of making it easier for crackers to exploit it.
This is a truly insightful comment. Software security isn't just a matter of verifying low-level code correctness. Though that's a step which can be usefully automated, what could possibly motivate the deeper analysis of what the software is supposed to do? To even identify the edge cases which exist at higher levels of abstraction requires an intimate knowledge of the software as implemented, its architectural intent, and the requirements which drove that solution.
Who is there in a position to develop that kind of knowledge? Typically, someone who comes along with a reason to adapt the software to new requirements, or whose taste for whatever reason runs to a different solution architecture, or who thinks that the design needs refactoring.
Of these three, proprietary software tends only to put resources into the first, whereas we know that merely adding new features typically results in reduced security when addressed in isolation. It takes a certain kind of altruism to care about the other two.
Software development is intrinsically hard. More secure development is harder. In terms of human resources, it makes sense that deep security improvements are going to come from people who are strongly motivated to know the software.
Security isn't a treatment that can be performed after the fact. It has to be an intrinsic part of the design process. So I agree, though I'll word the claim even more strongly. People who are only looking for security flaws by definition lack the fundamental insight that security is mostly about design. Implementation is just the tip of the iceberg.
Parity: What to do when the weekend comes.
I haven't tried this, and indeed there isn't much real work going on in provable software languages these days. But I think that it would be possible to set theoretical constraints on a program such that it serves data and does not allow it to be modified. There might be a good Ph.D. paper in it for someone.
It's possible to prove almost anything about the programs and operating systems, from type safety and runtime guarantees to any arbitrary set of predicates you want the system to satisfy. That assumes perfect hardware, so at most a security guarantee must be probabilistic, but can be made arbitrarily close to 100% secure by using redundancy and potentially cryptography in hardware to ensure either correct results or the triggering of an error condition.
For some current examples of work in this area you might look at George Necula's work on proof carrying code, which looks pretty interesting. They also have a Java and I think even a subset of C compiler that can output proofs of type safety. I haven't tried them though.
Another big challenge is to figure out how much the security model should cover. Type safety, privilege separation and file permissions are pretty obvious things to include, but what about network security, probabilistic assumptions about cryptographic security, or information control polices like the original Common Criteria required?. It would be useful for normal users to have some sort of classification they could assign to their personal information that the OS could use to keep most processes away from it. I've been interested in capability systems for quite a while too, since they often match a provable security model and programming language a little better than the typical ACL and privilege approaches.
It does if those people actually turn out. It doesn't otherwise. They can't help fix the code if they can't see the code though. OSS is a security opportunity, and not one that's necessarily fulfilled.
There are advantages to having the code open besides security, too. Code reuse, code clarity, project longevity, and more can be helped by having more than one small team working on code.
There are two situations in which OSS helps most. One is the small team that opens their code because even a few outside helpers is a significant increase in programmers. Many projects, including the Linux kernel, start out this way.
The other is when a project becomes popular enough or gets close enough to meeting enough people's needs that large groups of programmers and major commercial and noncommercial organizations start working on the code. Here each additional programmer is statistically less significant to the overall project numbers, but the total number of people is bigger than you'd expect a development team at even a large company to contain. This is where Linux is now, with almost 2,000 developers in a fairly recent version. Sure, they're not all working on the Linux kernel full-time, but that's still a lot of eyeballs on the code.
Lots of commercial software isn't written by Microsoft, SCO, IBM, Novell, CA, Adobe, Apple, and the other big corporate software houses. Lots of it, too, is written by small teams in small companies or by in-house people supporting some other industry. Most of it is, in fact. While there are many types of software development targets, there are vanishingly few companies that have hundreds of programmers working on a single system of software. Most of the projects in the world that get hundreds of programmers involved are written, in fact, as OSS or in one of just a handful of companies.
The biggest reason OSS is so important is not necessarily because it presents a challenge to companies like Microsoft, Apple, and Adobe although OS and office suite alternatives are always nice. It's because if companies with 2 or 20 programmers who have the start of something good can have just one bug fixed by each of another 2 or 20 programmers and another company can pick up that program and develop it further without redoing the work of those 2 or 20 programmers, those things are really significant at that scale.
How we know is more important than what we know.
Well, one place to look is the Mozilla Foundation security announcements. Both security companies and individuals participate. Where the report comes from a security company, they are attributed.
Bruce Perens.
I'm so glad the bad guys haven't found any vulnerabilities in their unexposed code, yet. The number of distinct virus definitions are evidently "just in case."
All 19 hijackers were known terrorists 09-10-2001. Lack of FBI intelligence does not justify warrantless wiretaps..
(and, -- yeah, you can just make that loop into a subroutine).
Free Software: Like love, it grows best when given away.
The following was sent to Charles Babcock at Information week in reply to an article entitled:
Open Source Code Contains Security Holes
As a developer and administrator of the Firebird Project I completely reject the statement you made in the above article.
"The somewhat moribund Firebird project, for example, is listed with 195 identified defects, of which it has verified zero and fixed zero. The active Firefox browser project, on the other hand,
has fixed 370 bugs, verified 56 and faces another 246 to verify and fix."
The Firebird project is in fact incredibly active - perhaps a look at this chart on our bug tracker might give you a clue.
http://tinyurl.com/yt5pgl
Firstly the Firebird project reviewed the Coverity results almost immediately they were published and found that the report isn't actually related to the Firebird engine. This URL shows our appropriate comments from the 7th March 2006:
http://www.firebirdnews.org/?p=180
Also more comments from Claudio on the 26th March 2006:
http://www.firebirdnews.org/?p=243
Secondly in a more detailed reply to the actual "PR" issue raised by David Maxwell, open source strategist for Coverity. If you had asked about this before printing the article you could have put some facts straight.
Nearly all of the 195 identified defects are in fact actually within an external piece of code we use for character sets and collation sequences ICU
http://www-306.ibm.com/software/globalization/icu/index.jsp
"The International Component for Unicode (ICU) is a mature, portable set of C/C++ and Java libraries for Unicode support, software internationalization (I18N) and globalization (G11N),
giving applications the same results on all platforms."
A open source project maintained by IBM. I will admit that we are using an older version of ICU (3.0) than is currently available and we will be upgrading to a newer version in the near future.
But this is not something that is a trivial exercise, as it means that any database using a different version of ICU would be incompatible with the version we ship. We plan to upgrade ICU
in Firebird version 2.5
Other defects reported are one in
usr/include/c++/4.0.2/i386-redhat-linux/bits/gthr-default.h
Not our problem either....
And there are four defects in firebird2/src/gpre/pretty.cpp a piece of old code used with a pre-compiler (gpre) to make BLR look good. BLR (Binary Language Representation),
Firebird's internal compiled language. This doesn't affect the Firebird server at all.
I would like you to print a correction or at least acknowledge the innacuracy of the article as regards Firebird.
Regards
Paul Beach
developer http://flamerobin.org
Ah, to hell with it all, let's just party!
Unless you're saying that all open source contributers are themselves the party. Now that's just dirty. Not my thing, you see. I guess, well, maybe if they really are first rate.
Hi Bruce! Good to see you are still around! Have a very good New Year, my friend, and give 'em hell!
Excellent points. However, many times the vulnerabilities are found via the exploits that take advantage of them, then subsequently fixed on all affected operating system. Said exploits are very often on Windows (and in particular Windows XP). That sways me towards the argument that my assertion is probably correct anyway.
My guess is the exploits tend to be on Windows XP because it has the fewest layers of security and is the most popular OS in the world. I don't know the answer to the question, "What if linux or MacOS were the most popular?" I suspect it wouldn't be as bad because it's a lot harder to inject the right code in the right place at the right authentication level in those OSes, but the fact that the iPhone was jailbroken so fast (it runs MacOS X 10.5 and is jailbroken via a buffer vulnerability) leads me to believe there would at least be more than zero viruses.
E pluribus unum
Third party reviews are valuable, but as an experienced engineer, I have experienced doubts about code when bugs found during development seemed to disappear without being "fixed". There is a tendency to forget about mysteries that go away even though they may have been indicative of potential trouble that would only manifest under rare conditions. Professionals try to take the time to understand these wobbles to make sure they are fixed, but I have never read code notes written by developers that include admissions of possible unstable code when it doesn't seem to be causing trouble. Aside from auditors making a list of things they suspect by reading the code, bugs can be based on bugs in hardware, or mismatched versions of sub components and so on. For us to have the perfect world, we would have to have the code reviewed, but also the developers would have to write journals and admit their doubts and observations about unsolved mysteries. HHDL commented about how science is based on observation and our ability to observe computers is limited.