Slashdot Mirror
LAMP Lights the OSS Security Way
Kevin Young wrote to mention a ZDNet article which goes into some detail on new results from a Department of Homeland security initiative. It's called the 'Open Source Hardening Project', and (funded to the tune of $1.24 Million) the goals of the initiative are to use a commercial tool for source code analysis to buck up the security base of many OSS projects. LAMP (the conglomeration of Linux, Apache, MySQL, and PHP/Perl/Python) was a 'winner' in the eyes of the project. From the article: "In the analysis, more than 17.5 million lines of code from 32 open-source projects were scanned. On average, 0.434 bugs per 1,000 lines of code were found, Coverity said. The LAMP stack, however, 'showed significantly better software quality," with an average of 0.29 defects per 1,000 lines of code, the technology company said.'"
This is old news:
'The tyrant will always find pretext for his tyranny.' - Aesop's Fables
http://it.slashdot.org/article.pl?sid=06/03/04/145 3245
Maybe I've been reading too much politics news lately, but I'm just waiting for Microsoft to come out with a statement that people capable of evaluating Perl, PHP, and Python are biased in favor LAMP solutions.
I need to do something about my cynicism.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
I'm so sick of everyone making their software depend on MySQL. If you're software is any good it should be able to run on more then one DB, at least Postgres.
To me, MySQL is like the MS Access of the Open Source world.
autopr0n is like, down and stuff.
Why not release the results of all the bugs? All those OSS projects will then have 0.00% bugs!
After all, that stuff's running most of the Internet.
Well, once you read this snippet from the article, they'll have enough ammo:
"There is one caveat: PHP, the popular programming language, is the only component in the LAMP stack that has a higher bug density than the baseline, Coverity said."
I assume he means the baseline of 0.434 bugs/1000 lines, and that if they removed PHP from the LAMP stack, that average bug count would go down even further.
How can one ever count the defects/bugs per line?
And why count them, and then not remove them?
And one huge defect is better than more than one small ones?
Sounds like a crappy research to me, time to RTFA.
If Microsoft was mass, stupidity would be gravity.
What are you talking about? We ship appliances with postgres and they don't need vaccuming.
autopr0n is like, down and stuff.
As part of the government-funded effort, Stanford and Coverity have built a system that does daily scans of the code contributed to popular open-source projects. The resulting database of bugs is accessible to developers, allowing them to get the details they need to fix the flaws, Coverity said.
Just an FYI...AMANDA had the highest amount of bugs at 1.214 Defects / KLOC and OpenVPN the lowest at 0.100 Defects / KLOC.
He who knows best knows how little he knows. - Thomas Jefferson
your post is also a dupe.
And it won't be long before Linux-zealots will start preemptively bashing Solaris to distract form the screaming shortcomings of their toy-OS. In fact, it will start in t 0.
Also from the article: The lowest was the XMMS audio player, with 0.051 defects per 1,000 lines of code.
Being someone who has used Amanda for many years and also XMMS, I find it hard to believe. Amanda has few problems (unless its the tape drive itself) and XMMS crashes sometimes when you just push a button in the "wrong way".
I think there can be a big difference between actual number of bugs and the perceived number of bugs. This almost makes counts like this useless for actually comparing software.
I have to say, I'm suprised and impressed... a $1.2M grant to harden open source software? Thanks all seeing orwellian eyeball. I don't recall slashdot posting anything about the original grant but here's a link from the posted article to another about the funding.
The data is meant to help secure open-source software, which is increasingly used in critical systems, analysts said. Programmers working on the Linux operating system, Apache Web server, BIND Internet infrastructure software and Firefox browser, for example, will be able to fix security vulnerabilities flagged by the system before their code becomes part of a released application or operating system.
We always knew Comcast was corrupt, here's the proof: http://tech.slashdot.org/comments.pl?sid=1909890&cid=34545432
If an automated system can detect bugs in code, why can't it fix them automatically too?
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
If you were really pro open source, rather than anti-Microsoft, you'd probably not care.
Seriously, the "at least it's not Microsoft" argument shouldn't impress anybody. The desire to put out a superior product, period, should be motivation enough to undertake something along these lines.
I don't trust Oracle
Honestly, I don't trust MySQL either. Every since they started going more commercial, there have been indications that eventually MySQL will be more closed up than open. But that's just speculation. So I've been slowly switching my stuff to use Postgresql. The only problem I have with postgresql is that it doesn't handle user administration as well. Other than that, its awesome.
I noticed that on Coverity's demo page, there's no mention of MySQL. If they *do* scan MySQL for bugs, why not have it on the front page?
Installed the Bubblemon yet?
Researchers at clandestine research labs in bases hidden deep in the Russian Alps have attempted to analyze portions of the leaked Internet Information Server (IIS) and Windows Vista code for similar flaws.
The findings were remarkable. They found 4,669 flaws, but since they didn't have the source code it resulted in a divide-by-zero error when they calculated the statistics on their Excel spreadsheet. The error triggered an unheard-of lockup on their Windows XP desktop.
On a positive note, recovering from the error alerted them to the presence of 43 strains of the MyDoom virus, 257 instances of Alexis spyware, and a bootleg copy of "Making of the Britney Spears Sonogram".
sigs, as if you care.
Of course this system from Coverity only checks for static source code bugs, and not run-time errors or semantic errors. And even then, not even necessarily real bugs, but "possible" bugs. Sounds more like FxCop than anything else. The real meat of this evaluation was probably done on the Symantec side, which was probably a laundry list of tests for known vulnerabilities.
No, you missed my point. I'd just love to see a *fair* comparison w.r.t. the number of bugs per KLOC.
As for "the desire to put out a superior product", what does that have to do with Microsoft?
RHCE; are you certified? Karma: ambiguous.
FTF(linked)A:
The project, while generally welcomed, has come in for some criticism from the open-source community. The bug database should help make open-source software more secure, but in a roundabout way, said Ben Laurie, a director of the Apache Foundation who is also involved with OpenSSL. A more direct way would be to provide the code analysis tools to the open-source developers themselves, he said. "It is regrettable that DHS has decided once more to ensure that private enterprise profits from the funding, while the open-source developers are left to beg for the scraps from the table," he said. "Why does the DHS think it is worthwhile to pay for bugs to be found, but has made no provision to pay for them to be fixed?"
I new I'd find a little truth in there somewhere!
The LAMP stack when broken down consists of:
Linux & Apache - rock solid stable releases.
MySql - Okay, getting better with each release.
P - This is the kicker. Perl, Python, PHP, and more so lately even that R one Ruby & Rails.
We are living in interesting times when we have so much choice... much like the Chinese curse. I do not see as how you can evaluate all of these platforms together in a general fashion. Where is the skew or bias in this study?
Someone on IRC recently was critical of a small website I put together in 2000. It was written in plain html, using frames *gasp*. Many people today do not realize how far web development has come since then.
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
What I mean is, why live for such comparisons? Does it have to be about beating Microsoft, or using them as a bar to jump over?
Even if it is, would you consider this an objective metric? Everybody knows that the kloc is, at best, an informal estimate of effort. Perhaps the Microsoft code does in 5 lines what the Open Source code does in 150. There are no bugs in those 5 lines, but 5 in the 150. The 150 line implementation implements an algorithm that runs in poly time, but the 5 lines run in exponential, what's the better code?
What about KDE? icewm? XFce? Blackbox? KDE at the very minimum, I would think, was a significant oversight. And why FreeBSD? 'L' is for 'Linux'. It's not 'BAMP'. But so long as they were gonna test BSD, why not OpenBSD, the one that can't speak 3 sentences without yelling mentioning how secure it is? It's awesome to see the government do something like this, but I just have to wonder what their justification was for some of the things they picked.
Unpleasantries.
Parent: +1.33 (Sarcastic yet funny).
Did the article say what kinds of things the automated tool flagged as bugs?
*sigh* back to work...
Security is not a feature, security is design. This ultimely means that security should provide good default values, knowledge about how to prevent buffer underruns/overruns and most importantly knowledge how to use a system. This means that security only will need tools to help a system architect and developer to confront him with his limits of his human brain and have a well documented yet very simple concise system and low speed development cycles.
Open source is great because of the many eyes, knowledge sharing and having nothing to do with corporate tradeoffs (the users have the largest voice. But it stinks in the fact that any noob can make programs which are badly designed and are a serious risk to security, however someone may learn faster form the mindsharing in the open source world. To have a well concise system so much more is needed than just some bugfixes. OSS is just a proof that closed source coorporate software is not good with security, but it isn't proof of sound security.
Most interesting is OpenBSD with it's oustanding default values, it's very own high profile malloc which prevents coders for lot of buffer underrunes/overruns, outperforming other malloc implementations. It has a very high quality of manpages and if you want to do something then you have to RTFM. That's what security should be, other than some less known bugs. I would even suggest that it would be better in the name of security that people would use program derivation (which is a very concise way to do formal verification). PIE and all other solutions maybe look practical, but they don't solve the lacking attention for "secure by design".
Cliff Claven
K.E.G. Party Chairman
Founding Leader of: Koncerned for Egalitarin Governance
"The company did not give details on the scope of the flaws it found." After all that work reviewing a rather massive amount of code, are they not going to publish detailed results, or at least contact developers? They have their data for the study now. WTF?
a few regular committers to openbsd work at coverity
search the archives. it's explained
vodka, straight up, thank you!
"LAMP Lights the Way"?! Was Slashdot acquired by C|Net?
For the love of all that's holy, please drop the hackish high-school-newsletter headlines.
What is there available for this kind of analysis that doesn't cost money to use?
Maybe they've measured in a specific way?
The whole database concept is just fundamentally wrong.
Do we have any metrics to compare this to Commercial software quality? I know thats a bit hard to answer, but I'm curious what this same tool has found when used on commercial code.
Maybe someone works for a company that used the tool on their code? Or some results have been published somewhere?
Big ones, small ones, some as big as yer 'ead!
Give 'em a twist, a flick o' the wrist...
It could be one of its library's. XMMS source code doesn't give you the player (or at least not one that will do anything) if you used gentoo or LFS you would know this.
Oh and bugs != programming errors or design flaws. Even if you elimenate all the bugs you could still have a program that blows up your cat when you try to save a file. It will just do it without any bugs getting in the way. Wich is a good thing. Unless your the cat.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Do not use complex queries then. If your DB is on the same host as your web/application server there should be no big performance penalty in doing series of "smaller" queries. As an added bonus - they well may be easier to debug. And do not name your tables "order" or your fields "desc" (though I really hate prefixes/suffixes).
Add more linefeeds!!! ;)
-Billco, Fnarg.com
17500000 lines of code, 0.434 bugs per 1000 lines, that makes 17500*0.434=7595 bugs, so where are the bugreports?
#
#\ @ ? Colonize Mars
#
Perhaps the Microsoft code does in 5 lines what the Open Source code does in 150.
;-)
I didn't know MS used Perl.
(unix tools excepted)
--Phillip
Can you say BIRTH TAX
My, that was a yummy potato!
First of all, just because people desperately need a stupid acronym for everything, they call pretty much any non java unix web development "LAMP". So there's nothing wrong with testing other free unixes, webservers, databases and languages. Second, a couple of the OpenBSD developers work at coverity. They have tested openbsd and fixed the issues found. It just isn't cool enough for the people who use acronyms like "LAMP" to care about.
I hope they report all the bugs they found in the source code with their automatic tools. Wow, once they fix all those bugs, there won't be any left, because obviously the tools found all the bugs that exist in the code, automatically, and didn't identify anything as a bug that wasn't one.
Is have DB spesific code for creating the tables, auto increments, etc. and then use standard queries.
autopr0n is like, down and stuff.
I think you're just a bad programmer trying to justify your bad programming practices. SPs are great if you have one big database that lots of applications use, as long as you don't put too much business logic in there such that trying to fix it for one program breaks it for all the others... and then you've got to debug them.
I mean they have their place. If you're writing code for a company that has a standard DB in place they can cut down on transaction time, but if you're writing a more generalized program for people who might want to use a DB they already have they're stupid.
autopr0n is like, down and stuff.
Spot on, as you can see on scan.coverity.com:
- PHP features 205 defects for 431,327locs, or 0.475 defects/kloc
- Perl has 91 defects for 431,327locs, or 0.19 defects/kloc
- Python is very slightly lower than perl (but with a noticeably smaller codebase) at 49 defects for 259,908locs or 0.189 defects/kloc
- Apache-httpd features 32 defects in 127,817 locs, or 0.25 defect/klock
MySQL isn't featured (Ruby is also a noticeable absent), but PostgreSQL stands at 296 defects for 815,748 locs, or 0.363 defects/kloc, and the lightweight SQLite has 16 defects for 60,722 locs or 0.263 defect/klock."The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
I don't know where you were in 2000, but i was using Apache Mysql PHP on *bsd in 2000, to run blog software off my cable modem. LAMP was already pretty functional in 2000, so I don't know why you'd be using static html unless you were just a n00b then ;)
https://www.gnu.org/philosophy/free-sw.html
Congratulation you're an idiot?
The goal of Coverity's tool is not to estimate the quality of an algorithm, but the quality of the code e.g. memory leaks and stuff.
"The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
Oh nog... lets rewrite all the DNS root servers to use LAMP~!
That would be the coolest thing EVA!!!
Until Google gets around to buying them all and porting them to a simple but usable AJAX format!
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
Use the best tool for the job. DB driven sites and huge PDF downloads are a bane for posting simple static content.
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
Thanks. I know. That was my point. Congratulations, you're an idiot who didn't manage to catch that, and you even pointed it out my posting in a forum about it.
The point was that errors per KLOC is a rather naive measure of success to begin with, just as KLOCs are a poor measure of output to begin with.
Next time you decide to call me an idiot, say something smart.
Relax. Some of us are so cynical, we regard your statement as a ray of Pollyanna sunshine.
They keep on making the same mistake.
I18N == Intergalacticization
Perhaps the Microsoft code does in 5 lines what the Open Source code does in 150.
.NET is a lot better, though...)
I take it you've never actually done any Win32 programming.
(I hear
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
You know what the difference is? The kernel and about five programs each - the rest is the same GNU software you find all over the place. When I tried Belenix 0.3, it was very much like an early Ubuntu. - only slower (and I never would have thought that *anything* could have gone slower than Ubuntu!). As I hear it, the next release of Belenix is the one where it's starting to shine, and then there's Nexenta...
Well, I know a way where you can leave PHP in that stack and still make the bugs/KLOC figure go down REALLY fast. All I need is for someone to check in this simple fix:
[BTW, I suggest using an optimization setting for the compiler so that redundant code is removed.]It has long been known that bugs/KLOC is a convenient, but not necessarily informative, statistic. A few seconds with google might prove enlightening:
it was a pain to figure out how to set up
It was a pain to properly configure (never did to my satisfaction, not much good documenation on editing the users file).
It was a pain to get programs that support it to work with it (eWiki) and then not all the features worked when I did (embedding images).
And then if I wanted to do clustering or distributed DBs and such down the road, it was turning out to be a pain finding information about that also.
I liked what I saw, the DB features make sense, and looks great but there's just not enough documentation support to keep me interested.
"Enjoy what you're doing! If it becomes drudgery, you're doing it wrong!" - Jim Butterfield
And PHP is the Visual Basic of the open source world.
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
It's not really free software now that I'm being forced to pay for it.
The site in question was a personal site with pictures from my wedding.
It was mostly for family members who did not get to attend.
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
It's perfectly fine to subset SQL, IMO.
The fact that so many databases do subset SQL99, however, is perhaps an indication that we would benefit from a well defined "SQL Light" subset.