Slashdot Mirror


US Government Studies Open Source Quality

anadgouda writes "US Department of Homeland Security has released a report on open source quality in an effort to study the security of open source. 31 popular open source packages were studied as part of this effort. From the article: 'Coverity's report, Stacking up the LAMP stack: a study of open source quality, was produced as part of a $1.24m, three-year DHS Science and Technology Directorate effort to evaluate and improve the security of open source.'"

39 of 165 comments (clear)

  1. So, by Eightyford · · Score: 4, Interesting

    So, does anyone have the numbers as to how much of the government uses open source? Is it mostly an applications thing (OpenOffice) right now, or are Linux and the BSDs much in use?

    1. Re:So, by jtev · · Score: 2, Funny

      It's called an explitive. Yes, realy, the word is an actual part of speach, though common usage has twisted it into meaning words that are unacceptable for publication. An explitive is a word that adds flavor to written or spoken communication that does not alter the meaning of that communication. Thank you for playing the grammar game, but please, understand what you're criticizing before you play again.

      --
      That which is done from love exists beyond good and evil
    2. Re:So, by egypt_jimbob · · Score: 4, Interesting

      Speaking as a student about to graduate and go into Federal Civil Service as a penetration tester, I can tell you that all of the agencies with which I have interviewed use mostly Linux. Well, all of the agencies that are actually good at what I want to do.

      So far it's been mostly gentoo from what i've seen, but there are some places that have to use RedHat because their management said it has to have 'support.'

      Bear in mind, however, that the places i'm interviewing are hardcore hacker groups, so this may be (and probably is) completely off the norm.

      --
      I am a leaf on the wind. Watch how I soar.
  2. Evaluate and Improve by Jeremy.DeGroot · · Score: 5, Insightful
    I think it's great that the government is backing this kind of study, and I think the the high marks a lot of packages received will really be a boon to the OSS movement. I think the part of TFA that excites me the most though, is this:
    Coverity evaluated 15m lines of open source code with Stamford University's Computer Science Department. The report has identified bugs that can corrupt a machine's memory space, memory leaks, buffer overruns and crashes. Coverity said it would now engage with open source developers to improve code, and identify potential reasons for why some projects have more bugs than others.
    If they're going to take their comments back to the communities that develop the software, then this could give the development communities a lot to work on and improve, and that could give us some greatly improved software in a year or two's time. I think work like this is the real strength of Open Source, and I hope to see more of it in the future.
  3. So they submitted Bugs, Right? by BigBuckHunter · · Score: 5, Interesting

    This study would be extremely valuable if they had submitted BZilla bugs for each and every defect they encountered. It's hard to tell from the article whether they did or not. One thing that I have learned from running ~arch in Gentoo is that if you don't submit bugs, things aren't going to get fixed.

    BBH

    1. Re:So they submitted Bugs, Right? by Too+many+errors,+bai · · Score: 4, Funny

      If these packages are used within the government, the security holes discovered are probably kept secret. National security and all that.

  4. Fan of Linux, not of Homeland Security by toddbu · · Score: 3, Informative

    I feel very conflicted by this report. On the one hand, I'm happy to see a report that favors open source. On the other hand, in the wake of the Katrina political fallout, it's difficult to say whether this report helps or hurts. The last thing LAMP needs right now is to get caught up in Brown/Chertoff/GWB affair. The only thing worse would be to have the UAE issue a similar report. :-)

    --
    If you don't want crime to pay, let the government run it.
    1. Re:Fan of Linux, not of Homeland Security by Saeed+al-Sahaf · · Score: 2, Insightful

      There is no relationship between this study and Katrina. The disaster people work in a different office, down the hall. Would you like me to transfer you? Hold on....

      --
      "Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
    2. Re:Fan of Linux, not of Homeland Security by toddbu · · Score: 3, Interesting
      The branches of government like the GAO, NASA, the President's Office of Management and Budget are all known for this. Not everything that goes on in washington has to do with politics.

      You can't really be that naive, can you? Take the OMB for example. There's a big debate going on about whether OMB should use static scoring or dynamic scoring. It doesn't really matter which one you prefer, but I can tell you that in the current political climate it makes a *huge* difference. Democrats prefer static modeling because then they can argue against tax cuts. Republicans favor dynamic modeling to support a "trickle down" effect. But the idea that somehow OMB is neutral is ignoring reality. Even if they don't intend to favor one party or another, the fact is that there is no action that they can take that won't benefit one group or another.

      Interesting that you should mention NASA. Their very existence depends on the support of the aerospace community and the regions of the country that benefit from NASA centers. They are very good at using their influence to get what they want. Even if you could claim that they don't favor one political party over another, they are still very skilled at using political influence to their advantage.

      --
      If you don't want crime to pay, let the government run it.
  5. Their findings are as follows by Mancat · · Score: 4, Funny

    Open-source software is a serious threat to this country. These terrorist schemes, or "development projects," as the terrorists refer to them, are designed to rot away the core values of our great nation that we hold so dearly. One in particular, known as "Linux," is especially suspect. It is "developed" by terrorists worldwide, many of which are communists, and many of which do not even support our great commander in chief! It is apalling! How can we trust the security of our nation to these rogue "developers?" Surely they may have hidden devices in their programs, hidden in elaborate matrices of computer programming, that when activated by the terrorists, will disable the software and send them all of our secret data! It can only be expected.

    The terrorists are cunning, they are secretive, and they will destroy us if they have their way. This world-wide "open source" terrorist movement must be deconstructed and eliminated. There is no other way to protect our Great Nation! We say to you, as the purveyors of truth and all that is good, avoid this "open source" and its proponents like the plague! They wish to destroy everything we hold dear. You, my good American, are the first line of defense. Report users of "open source" to the authorities. Gather any information on them that you can. You may even consider running their dastardly "software packages" in your own free time, so that you may come to know your enemy - for knowledge is the greatest tool that we have in this fight.

    Stand proud, my fellow Americans, and beware this new emerging beast. It will surely be the end of us all if we do not take action now.

    Quoted from President George W. Bush's State of the Nation Address, January 2007.

    --
    hello dear sirs my name is jamesh i are india (bihar) can u guide me install red had linux 9?
  6. Where's the report? by boa13 · · Score: 4, Insightful

    One would expect that being about open-source and all, and with a purpose of helping open-source developers improve the quality of their code, they would publish the report on a governement website somewhere. C'mon, where's the link?

  7. stanford will keep the database public... by hihihihi · · Score: 5, Informative

    the report have a better coverage on this page: http://www.eweek.com/article2/0,1895,1909946,00.as p

    from this TFA:
    "Anti-virus vendor Symantec Corp. is providing guidance as to where security gaps might be in certain open-source projects."

    PS:i am not sure if it has been published on /. or not

    --
    everyone downmodding this post will be prosecuted for reading my post without first buying a license!!!
  8. Meaningless categorization by sreekotay · · Score: 4, Insightful

    I've always thought it VERY odd to think about "Open Source" as a thing.

    It'd be like saying: We studied the quality of software compiled with the Watcom 10.0 C++ compiler. "Open source" cuts across so many levels of skill and projects. You can pretty find projects that support (or destroy) whatever thesis you'd like to put forward

    Even more, somebody pays for the development of the software, one way or another.

    This artlice (from ONLamp) http://www.onlamp.com/pub/a/onlamp/2005/07/21/soft ware_pricing.html really puts into better perspective. Basically, it says ALL software can be deconstructed to being about the service (at least so long as the technology curve continues, in practice, to limit its lifespan).

    --
    graphicallyspeaking

    1. Re:Meaningless categorization by Night+Goat · · Score: 2, Insightful

      It's a lot more difficult to study the bugs in closed source code and get a bugs per thousand lines of code metric out of it. That is probably why they're doing the testing on OSS.

  9. MOD PARENT DOWN by Anonymous Coward · · Score: 4, Funny

    The parent is wasting valuable time on Slashdot that should be spent finalizing his Independent Study project for the College of Wooster. He has precious little time left.

    1. Re:MOD PARENT DOWN by Jeremy.DeGroot · · Score: 2, Interesting

      If this came from who I think it did, your IS ain't in any better shape than mine, buddy. :-p

  10. Compare with... by Anonymous Coward · · Score: 2, Interesting

    ...New Zealand's recent analysis of open source, which focuses on legal issues.

    1. Re:Compare with... by cyber-vandal · · Score: 2, Insightful

      25 There is a risk that open source software contains functional defects, or breaches a third party's intellectual property rights (e.g. where it contains code misappropriated from proprietary software or functionality in breach of a patent). The absence of warranties and indemnities in most open source licences means the licensee bears this risk. This can be contrasted with the protection usually available under commercial software licences.

      That made me laugh.

  11. Yes by jascat · · Score: 5, Interesting

    While not used on every desktop, I know of a lot of F/OSS being used everyday in the military. It would be stupid to not use it. Why would companies like Redhat and Novell spend money on getting their software certified to run on classified systems if it wasn't going to get used? While we may be selling out to Microsoft a lot, there are times when those of us who know better manage to convince the decision makers of the right tool for the job. In some cases, it's a MS product, in others, it's something else.

  12. RTFA by Night+Goat · · Score: 3, Interesting

    From the article, which I'm SURE you read:

    Coverity evaluated 15m lines of open source code with Stamford University's Computer Science Department. The report has identified bugs that can corrupt a machine's memory space, memory leaks, buffer overruns and crashes. Coverity said it would now engage with open source developers to improve code, and identify potential reasons for why some projects have more bugs than others.

    Yes, the folks who ran the tests plan to submit their findings to the developers to help squash bugs.

  13. Open Source Software: Opportunities and Challenges by Old+Duck · · Score: 5, Informative

    An interesting study was done by the U.S. Military (the Airforce, I believe) concerning Open Source and it's place in the department of defense, though it is written in such a way to be useful to non-military personnel and applications. It is a similar, yet IMHO, a more interesting read than the parent.

    The report can be found as a PDF at http://www.stsc.hill.af.mil/crosstalk/2005/01/0501 Tuma.pdf

    --
    There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.
  14. What is normal? by CAPSLOCK2000 · · Score: 2, Insightful
    FTA:

    LAMP "showed significantly better software quality" above the report's baseline with an average of .32 defects per 1,000 lines of code, according to Coverity.
    The average for open source projects analyzed is .42 per 1,000 lines.


    Does anyone have any factual data on what is "normal" (accepting all the problems of counting lines and bugs in the first place). I've seen estimates range from 2 to a 100 per 1000 lines.
  15. Re:OSS Security depends on bugs being fixed by J.+Random+Luser · · Score: 3, Insightful
    Support. I can make Linux work for me and my company but not every company can. Where is the Linux Geek Squad? Yea all those scan-disk, defrag, run adaware and scan for virus "techies" give me the creeps but they seem to fill a need. Where can the mythical grandmother go to get a DVD installed in her Linux box or find out how to fix Thunderbird if the mail folder blows up? I will not even go into the poor state of some documentation for open source programs.

    Reminds me of when as a noob, I reported an error in a man page to a project mailing list, hoping somebody close to the project might pick it up and fix it. Nah, the response was: OK, write yourself a new man page.

    That attitude still pervades most OSS projects. The result is open source is regarded as by geeks for geeks, and IMHO this, more than any perceived security risks, will keep it off the desktop for a long time yet. Sure, I see quite a few specialist applications coming thru now packaged for MacOS-X. Here's an example (names obscured to protect the ignorant): a multimedia application, gui built on GTK, equal to commercial products of several hundred dollars, well worthy of the suggested paypal donation. But it requires access to the Hardware Abstraction Layer, which is provided by a different oss project, whose raw binaries will do what's needed from the command line, but no gui interface yet, unless you build it, in Qt.

    Security problems in OSS are multiplied by forking, and geekishness for its own sake.
  16. Re:OSS Security depends on people admitting a bug by JulesLt · · Score: 2, Insightful

    It's that good old 'total cost of ownership' - for the two categories you identified the answer is 'lower', but for many people lacking in IT skills it is a more complex calculation - especially in places where their IT support is already contracted out. O/S actually needs to come in and compete in these environments, rather than expecting them to become IT literate.

    Advocates need to consider the many places in their lives where they purchase things rather than make or maintain them themselves - for many people without interest in technology, software is in that category - we live in a society where people pay a premium for ready-made meals, despite the repeated message they could save money by making their own.

    --
    'Capitalists of the world, unite! Oh ... you have' (League Against Tedium)
  17. Wow by ROOK*CA · · Score: 2, Funny

    Three years, $1.24 Million, and what do we got .....

    The envelope please ...

    "LAMP "showed significantly better software quality" above the report's baseline with an average of .32 defects per 1,000 lines of code, according to Coverity. The average for open source projects analyzed is .42 per 1,000 lines."

    Wow, LAMP is a pretty damn high quality stack after all....gee thanks Captain Obvious, we didn't really need those tax dollars for anything anyways. :)

  18. Re:OSS Security depends on people admitting a bug by killjoe · · Score: 3, Insightful

    Well the expected FUD mobile shows up again.

    I especially love the "Windows XP and office 2003 just worked" line. That's a rich one. Anybody who has actually worked with those technologies knows how much effort it takes to make them "just work".

    I do think you have point about the incompatibilities of the office formats with other software. It's a well known fact that MS products use office formats to undermine other software. I think that people are finally wising up to this and pushing for ODF. Even MS has tried to make the default office format XML based so I think this problem will go away very soon.

    What's interesting to me is how different office 12 looks from office 2003 (who the fuck came up with that versioning scheme?). It will be much easier to re-train employees from office 2003 to open office (which looks very similar) then to retrain employees to migrate from 2003 to 12. Office 12 looks and acts radically different then what people are used to.

    --
    evil is as evil does
  19. superb! by macsox · · Score: 4, Funny

    if there is one group of people i trust to be able to accurately identify a quality product, it's the government.

  20. Re:money? by BeanThere · · Score: 4, Insightful

    And I wonder how many more millions they can now save by using OSS, now that they know they can be more confident in its quality? Have you ever heard of the word "investment"?

  21. Same Old Math Error by oldCoder · · Score: 2, Interesting
    These guys just can't think straight:
    LAMP "showed significantly better software quality" above the report's baseline with an average of .32 defects per 1,000 lines of code, according to Coverity. The average for open source projects analyzed is .42 per 1,000 lines.
    So if LAMP open-source is simply more verbose than other kinds of open source, the number of bugs per line of code can go down? How about just adding a million lines of bug-free but totally bogus code to your project -- and completely winning the race of "Defects per 1,000 lines of code"?

    If I remember correctly Coverity has been discussed on slashdot previously and they used the same diseased statistical thinking back then, too.

    --

    I18N == Intergalacticization
    1. Re:Same Old Math Error by tricorn · · Score: 2, Interesting

      The real problem isn't bogus stats caused by line inflation. The real problem is that it only finds certain types of bugs. If a bug causes an incorrect result or improper behavior, but doesn't cause a memory leak or the like that crashes the program or system, then it isn't being found. It also isn't finding STUPID code - code that works but is ridiculously convoluted, slow, difficult to modify, redundant (writing 5000 lines of code to do some string manipulation and parsing that could be done just as easily and efficiently using a RE library, or use lex, or some other straightforward solution - I've seen code that re-implemented several of the standard library string routines, and to add insult to injury, did it poorly and with a memory leak - at least these guys would have found the memory leak, but their solution would probably be to fix the leak, not toss the whole routine). C++ programmers seem to do this kind of thing particularly often, although many "object oriented" programmers can screw things up in multiple languages with equal facility.

  22. Yes. by Anonymous Coward · · Score: 2, Informative

    I'm involved in one of the F/OSS projects that Coverity analyzed; and yes, they were co-operative with the dev team in sharing their insights.

  23. Re:OSS Security depends on people admitting a bug by tech_guru5182 · · Score: 2, Interesting

    Actually, it appears to be a switch back to the old versioning scheme.

    Also, I agree with the comment about the FUD mobile appearing.

    I have no problems finding a local community college with Linux classes. I actually took one a few years ago as part of my associate's degree. You may want to try searching for UNIX instead, as Colleges usually keep old names around. The class I took was actually called UNIX Concepts, but was actually taught on Red Hat Linux.

    See
    EET 175 Network Operating Systems
    EET 208 UNIX Concepts
    at <a href="https://www.owens.edu/cgi-bin/courses.pl">Ow ens Community College</a>

    --
    BAN BPL! Keep the radio spectrum free fro
  24. Re:OSS Security depends on people admitting a bug by 0x0000 · · Score: 3, Interesting
    software is NOT always the best solution for every problem, especially when it comes to security.

    .... you say this [the above], the procede to make an argument based solely on funcitonality and support of software packages available. Do you have anything to back up your initial statement there, that non-Open software is somehow better for applications that require "security" (a vague term at best, in this context, I think - are you talking security against networked crackers, automated worm attacks, attempts to de-crypt encrypted data ... )? I'm not trying to "flame" you, but you don't support the your statement at all in your post, and I honestly can't think of an instance where proprietary or closed source software is "more secure" than F/OSS...

    I can not take a course on Linux at my local Community College.

    You should move to where there's a better community college - I think it may even be safe to use the word "most" when describing how many schools there are across the country now that are teaching Linux, FreeBSD, or both. Are you saying your school doesn't offer it, or that you can't take it for some other reason?

    As a sidelight, note that many schools that have recieved endowments from M$ (thru one channel or another) have magickally dropped the course-work they once had that didn't require the purchase (at a student discount, of course) of M$ products - if that's what's going on at your school, you might want to address it with your administration - after all, when you're paying for an education, they're defrauding you if they don't give you what you pay for - regardless of what M$ is paying them (under the table) not to teach you....

    out in to the real world as system administrators and/or programmers, they will have a better chance to find a job if they know Windows and Linux

    Not sure just what sector of the real world you're talking about, here, but *I* won't hire you if you don't understand operating systems generally (we're talking critical embedded systems here - the stuff that's going to outlive the users who are thinking they need a "new" obsolete PC), and have some skill with anything that can be called one. "Platform Independance" and "Language Independant" aren't just test questions in the Real World outside Microsoft Applications Land - a rich and profitable land to be sure, but nothing grows there so all [brain] food must be imported, and life expectancy is pretty short generally due to contaminated memepools, rarified atmospheres, and the mind numbing depressions induced by the incredibly bleak cyberscapes...)

    Anyway - all that said, I do agree with you about support for F/OSS - it is overall diffcult to access, often hard to understand, and generally just unusable for those who are not already to some degree techinical initiates. And that does need to change. Imo.

    --
    "The Internet is made of cats."
  25. SE-Linux by Anonymous Coward · · Score: 2, Interesting

    Why no mention of SE-Linux?
    One agency study.
    1.5 million dollars spent.

    How much did the NSA spend developing SE-Linux?
    Must have cost more than 1.5 million. And that is now at the core of Linux.

    Yes many in the US Government are aware that Open Source software rocks.

    Impeach the Liar

  26. Stamford University? You mean Stanford. by Morganth · · Score: 2, Interesting

    I know that there is a Stamford University, and everyone always jokes that it's for people who want to pretend they went to Stanford, but, this just makes things really confusing. The Register article says Coverity used a verifier from Stamford University, when really the program came from Stanford. In fact, AFAIK, UCONN-Stamford doesn't even have a CS department.

  27. Where's the Beef? by PhYrE2k2 · · Score: 2, Insightful

    To quote the Wendy's commercial, "Where's the Beef?".

    No seriously! Where's this article? I'd imagine three years and 1.25 million dollars would produce a hefty article. I'd love to give it a read! "US Department of Homeland Security has released a report on open source quality"- so where's the release?

    It cites one or two figures, and throws around lots of buzz-words, but there's no comparison? No information? No study of reliability? Nothing at all.

    PS: As a side-note, if they 'studied' 15 million lines of code over three years, and were able to identify defects, shouldn't we be seeing a nice patchset coming from Coverity sometime soon... Think about it. It's easy to tell someone else to fix it, but a good part of OSS is giving back.

    --

    when you see the word 'Linux', drink!
    1. Re:Where's the Beef? by sl4sh13 · · Score: 2, Funny

      The report must be closed source!

  28. Re:OSS Security depends on bugs being fixed by MarkByers · · Score: 2, Insightful

    Reminds me of when as a noob, I reported an error in a man page to a project mailing list, hoping somebody close to the project might pick it up and fix it. Nah, the response was: OK, write yourself a new man page.

    What project was it? Is it anything we care about?

    How about linking to your 'bug report' so that we can see this supposed reply?

    That attitude still pervades most OSS projects.

    What OSS projects are you referring to? Not all OSS projects are equal. You are generalising.

    What evidence do you have of most OSS projects having a bad attitude?

    --
    I'll probably be modded down for this...
  29. Navy Replaced Sun with Yellow Dog Linux ... by AHumbleOpinion · · Score: 4, Interesting

    The US Navy replaced Sun with Yellow Dog Linux, originally on Apple hardware and now on some other PowerPC based hardware, for sonar processing on subs.