Undocumented Open Source Code On the Rise

ruphus13 writes "According to security company Palamida, the use of open source code is growing rapidly within businesses. However, the lack of documentation and understanding of how the code works can increase the vulnerability and security risks the companies face. OStatic quotes Theresa Bui-Friday saying, 'In 2007, Palamida's Services team audited between 300M to 500M lines of code for F500 to venture-backed companies, across multiple industries. Of the code we reviewed, Palamida found that applications written within the last five years contain 50% or more open source code, by a line of code count. Of that 50% of open source code, 70% was undocumented. This is up from 30% in 2006.' How can businesses protect themselves and still draw on open source code effectively?"

Comments

Kind of makes the underhanded code contest

[the_humeister]

hit closer to home perhaps? A quick glance at some of those code snippets and they can be easily missed. Now place them in large applications with thousands upon thousands of lines of code and who knows how long it'll take to find them.

Re:Statistics ...

[tcopeland]

> 70% of most code is just common-everyday stuff that doesn't
> NEED to be documented in the sense that comments are completely wasteful.

So true! Rather than this code:

# Finds the most recent orders for the passed in person
def get_rec(p)
    # blah blah
end

I'd much rather see an intention-revealing method name (hat tip Marcel Molina):

def find_recent_orders_for(person)
    # blah blah
end


I'm still not really sure what documentation is really useful - maybe a few diagrams plus some use case descriptions that go through the code, maybe? I'm not sure. I guess it depends on the project - it is a widely used library? Is it an internal department app to track the coffee fund? etc.

My experience with open source code has been that the large projects have decent docs... I was just reading through some of the PostgreSQL docs on backups this weekend and they're quite good.

Meh, I'll save y'all reading all of this

[sticks_us]

The money shot:

Again, open source is not any more risky than any other kind of code. What is risky is not documenting your use of 3rd party code...here are some quick examples: OpenSSL, phpBB, xt:commerce

So what they're basically saying is that if you use OSS tools in your company, someone should probably be keeping track of them and patching them as needed.

Should this not hold for *all* software you've deployed? Few programs are immune to eventual obsolescence (including ongoing bugs and security problems), so if you think you're safe just because you're running a bought-and-paid-for solution that you've subsequently ignored, you're probably in trouble.

That being said, I wonder about this: ...applications written within the last five years contain 50% or more open source code, by a line of code count

I get the impression that we're not getting the full story here. If their code audit showed that 50% of software X was copypasta from sourceforge, that would be something (you probably have crappy developers plus possible legal hell if there were copyright infractions).

On the other hand, if they figured "hey, your hello world program uses library Y, which is 2 million lines that we don't think is documented properly," then the "application" does not *contain* 50% or more open source code, but rather *references* a certain amount of open source code, which is probably a meaningless statistic.

Re:Not just for security

[tacocat]

I would be interested to know what languages you have used.

I have found Perl to be very well documented, even though it appears to be on a decline or leveled off on the number of developers and active projects.

Meanwhile, I have looked into use Rails and found it a great example of shitty code practices. I've stated this very case to the development community and they pretty much debunked my statements as one belonging to an inexperienced developer unwilling to "go the distance".

I hope this might be slightly helpful in getting people like the Rails community to either understand that they really do need documentation or get companies to throw aside Rails as POS software that is so lacking in documentation that it's a greater burden to have it than to use the alternatives.

There is an excellent case where if you have a highly experienced and knowledgeable developer then you maybe don't care. But if you have to replace this developer with one less knowledgeable or want to expand your development team, you suffer a huge start up cost of trying to bring someone up to speed at your expense.

Specifically, the Rails plug-ins are documented with over simplified tutorials that aren't even available for free and so you have to make an extra effort to find the documentation for the software that you download since they aren't in the same location. Restful Authentication is one example in particular.

Add to that the documentation in Ruby DBI. There isn't any. The documentation says to see Perl DBI for documentation. Considering this is a reference to a different language with different syntax and some of the Perl methods aren't possible in Ruby and likewise Ruby DBI has methods that aren't available in Perl. WTF? This is documentation.

Re:Not just for security

[MikeFM]

Any software product without good documentation is difficult to use. Proprietary programs are potentially a lot worse than open source because you don't have an easy option for figuring out the hard bits for yourself.

My example would be Activant Eclipse (formally Intuit) - the ERP software the company I work for uses. It's expensive, performs poorly (even on the expensive IBM mini that is required), is buggy, is largely undocumented, is hard/expensive to customize and is completely required to keep our business running. We pay for an expensive support contract and support is still often a joke - complicated issues are usually figured out by our staff because their support staff isn't able to do much. A horrible painful experience.