Slashdot Mirror


US Government Studies Open Source Quality

anadgouda writes "US Department of Homeland Security has released a report on open source quality in an effort to study the security of open source. 31 popular open source packages were studied as part of this effort. From the article: 'Coverity's report, Stacking up the LAMP stack: a study of open source quality, was produced as part of a $1.24m, three-year DHS Science and Technology Directorate effort to evaluate and improve the security of open source.'"

14 of 165 comments (clear)

  1. Evaluate and Improve by Jeremy.DeGroot · · Score: 5, Insightful
    I think it's great that the government is backing this kind of study, and I think the the high marks a lot of packages received will really be a boon to the OSS movement. I think the part of TFA that excites me the most though, is this:
    Coverity evaluated 15m lines of open source code with Stamford University's Computer Science Department. The report has identified bugs that can corrupt a machine's memory space, memory leaks, buffer overruns and crashes. Coverity said it would now engage with open source developers to improve code, and identify potential reasons for why some projects have more bugs than others.
    If they're going to take their comments back to the communities that develop the software, then this could give the development communities a lot to work on and improve, and that could give us some greatly improved software in a year or two's time. I think work like this is the real strength of Open Source, and I hope to see more of it in the future.
  2. Where's the report? by boa13 · · Score: 4, Insightful

    One would expect that being about open-source and all, and with a purpose of helping open-source developers improve the quality of their code, they would publish the report on a governement website somewhere. C'mon, where's the link?

  3. Meaningless categorization by sreekotay · · Score: 4, Insightful

    I've always thought it VERY odd to think about "Open Source" as a thing.

    It'd be like saying: We studied the quality of software compiled with the Watcom 10.0 C++ compiler. "Open source" cuts across so many levels of skill and projects. You can pretty find projects that support (or destroy) whatever thesis you'd like to put forward

    Even more, somebody pays for the development of the software, one way or another.

    This artlice (from ONLamp) http://www.onlamp.com/pub/a/onlamp/2005/07/21/soft ware_pricing.html really puts into better perspective. Basically, it says ALL software can be deconstructed to being about the service (at least so long as the technology curve continues, in practice, to limit its lifespan).

    --
    graphicallyspeaking

    1. Re:Meaningless categorization by Night+Goat · · Score: 2, Insightful

      It's a lot more difficult to study the bugs in closed source code and get a bugs per thousand lines of code metric out of it. That is probably why they're doing the testing on OSS.

  4. Re:Fan of Linux, not of Homeland Security by Saeed+al-Sahaf · · Score: 2, Insightful

    There is no relationship between this study and Katrina. The disaster people work in a different office, down the hall. Would you like me to transfer you? Hold on....

    --
    "Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
  5. What is normal? by CAPSLOCK2000 · · Score: 2, Insightful
    FTA:

    LAMP "showed significantly better software quality" above the report's baseline with an average of .32 defects per 1,000 lines of code, according to Coverity.
    The average for open source projects analyzed is .42 per 1,000 lines.


    Does anyone have any factual data on what is "normal" (accepting all the problems of counting lines and bugs in the first place). I've seen estimates range from 2 to a 100 per 1000 lines.
  6. Re:OSS Security depends on bugs being fixed by J.+Random+Luser · · Score: 3, Insightful
    Support. I can make Linux work for me and my company but not every company can. Where is the Linux Geek Squad? Yea all those scan-disk, defrag, run adaware and scan for virus "techies" give me the creeps but they seem to fill a need. Where can the mythical grandmother go to get a DVD installed in her Linux box or find out how to fix Thunderbird if the mail folder blows up? I will not even go into the poor state of some documentation for open source programs.

    Reminds me of when as a noob, I reported an error in a man page to a project mailing list, hoping somebody close to the project might pick it up and fix it. Nah, the response was: OK, write yourself a new man page.

    That attitude still pervades most OSS projects. The result is open source is regarded as by geeks for geeks, and IMHO this, more than any perceived security risks, will keep it off the desktop for a long time yet. Sure, I see quite a few specialist applications coming thru now packaged for MacOS-X. Here's an example (names obscured to protect the ignorant): a multimedia application, gui built on GTK, equal to commercial products of several hundred dollars, well worthy of the suggested paypal donation. But it requires access to the Hardware Abstraction Layer, which is provided by a different oss project, whose raw binaries will do what's needed from the command line, but no gui interface yet, unless you build it, in Qt.

    Security problems in OSS are multiplied by forking, and geekishness for its own sake.
  7. Re:OSS Security depends on people admitting a bug by JulesLt · · Score: 2, Insightful

    It's that good old 'total cost of ownership' - for the two categories you identified the answer is 'lower', but for many people lacking in IT skills it is a more complex calculation - especially in places where their IT support is already contracted out. O/S actually needs to come in and compete in these environments, rather than expecting them to become IT literate.

    Advocates need to consider the many places in their lives where they purchase things rather than make or maintain them themselves - for many people without interest in technology, software is in that category - we live in a society where people pay a premium for ready-made meals, despite the repeated message they could save money by making their own.

    --
    'Capitalists of the world, unite! Oh ... you have' (League Against Tedium)
  8. Re:OSS Security depends on people admitting a bug by killjoe · · Score: 3, Insightful

    Well the expected FUD mobile shows up again.

    I especially love the "Windows XP and office 2003 just worked" line. That's a rich one. Anybody who has actually worked with those technologies knows how much effort it takes to make them "just work".

    I do think you have point about the incompatibilities of the office formats with other software. It's a well known fact that MS products use office formats to undermine other software. I think that people are finally wising up to this and pushing for ODF. Even MS has tried to make the default office format XML based so I think this problem will go away very soon.

    What's interesting to me is how different office 12 looks from office 2003 (who the fuck came up with that versioning scheme?). It will be much easier to re-train employees from office 2003 to open office (which looks very similar) then to retrain employees to migrate from 2003 to 12. Office 12 looks and acts radically different then what people are used to.

    --
    evil is as evil does
  9. Re:money? by BeanThere · · Score: 4, Insightful

    And I wonder how many more millions they can now save by using OSS, now that they know they can be more confident in its quality? Have you ever heard of the word "investment"?

  10. In what sense is the CBO a political animal? by Anonymous Coward · · Score: 1, Insightful

    Democrats prefer static modeling because then they can argue against tax cuts. Republicans favor dynamic modeling to support a "trickle down" effect.

    From the link you provide:

    "[I]instituting a formal policy for the use of dynamic scoring would require planning or estimating around future fiscal policy, taxpayers' future behavior, and future business cycles well beyond the window of time for which they can reliably be predicted .... [M]easures of macroeconomic feedback effects are very sensitive to assumptions that are subjective... Given the degree of uncertainty inherent in current methods of macroeconomic forecasting, true dynamic scoring would not allow the consistent and comparative cost estimates"

    "CBO and JCT do currently provide estimates to illustrate potential effects on the economy of significant tax proposals, at the request of Members of Congress, but such estimates are not official and only offered as supplemental information. Even opponents of dynamic scoring have encouraged this practice to continue in the same sort of advisory, rather qualitative (and not quantitative) manner because, as Kobes and Rohaly explain, they 'show how sensitive a proposal would be to various changes in these [macroeconomic] assumptions. However, producing an estimate in the form of a single revenue or cost number would be misleading.'"

    Upshot:

    1. The CBO uses static scoring for official estimates, and does so only for non-political reasons.

    2. Democrats perfer static scoring for official estimates, which sounds like a better method, but no doubt they do so only for political reasons.

    3. Republicans prefer dynamic scoring for official estimates, which sounds like an inferior method, and no doubt they do so for political reaons.

    So yes, there is a controversy, and the controversy is politically-motivated.

    But no, the CBO's decision in this matter is motivated only by sound accounting principles without regard for political ramifications.

  11. Where's the Beef? by PhYrE2k2 · · Score: 2, Insightful

    To quote the Wendy's commercial, "Where's the Beef?".

    No seriously! Where's this article? I'd imagine three years and 1.25 million dollars would produce a hefty article. I'd love to give it a read! "US Department of Homeland Security has released a report on open source quality"- so where's the release?

    It cites one or two figures, and throws around lots of buzz-words, but there's no comparison? No information? No study of reliability? Nothing at all.

    PS: As a side-note, if they 'studied' 15 million lines of code over three years, and were able to identify defects, shouldn't we be seeing a nice patchset coming from Coverity sometime soon... Think about it. It's easy to tell someone else to fix it, but a good part of OSS is giving back.

    --

    when you see the word 'Linux', drink!
  12. Re:Compare with... by cyber-vandal · · Score: 2, Insightful

    25 There is a risk that open source software contains functional defects, or breaches a third party's intellectual property rights (e.g. where it contains code misappropriated from proprietary software or functionality in breach of a patent). The absence of warranties and indemnities in most open source licences means the licensee bears this risk. This can be contrasted with the protection usually available under commercial software licences.

    That made me laugh.

  13. Re:OSS Security depends on bugs being fixed by MarkByers · · Score: 2, Insightful

    Reminds me of when as a noob, I reported an error in a man page to a project mailing list, hoping somebody close to the project might pick it up and fix it. Nah, the response was: OK, write yourself a new man page.

    What project was it? Is it anything we care about?

    How about linking to your 'bug report' so that we can see this supposed reply?

    That attitude still pervades most OSS projects.

    What OSS projects are you referring to? Not all OSS projects are equal. You are generalising.

    What evidence do you have of most OSS projects having a bad attitude?

    --
    I'll probably be modded down for this...