Open Source Code Maintainability Analyzed

gManZboy writes "Four computer scientists have done a formal analysis of five Open Source software projects to determine how being "Open Source" contributes to or inhibits source code maintainability. While they admit further research is needed, they conclude that open source is no magic bullet on this particular issue, and argue that Open Source software development should strive for even greater code maintainability." From the article: "The disadvantages of OSS development include absence of complete documentation or technical support. Moreover, there is strong evidence that projects with clear and widely accepted specifications, such as operating systems and system applications, are well suited for the OSS development model. However, it is still questionable whether systems like ERP could be developed successfully as OSS projects. "

Comments

I tried to read it but...

[Anonymous Coward]

by Ioannis Samoladas, Ioannis Stamelos, Lefteris Angelis, Apostolos Oikonomou ...it was all Greek to me.

Was this really a surprise?

[StateOfTheUnion]

Was this really a surprise? Did anyone think that open osurce software is as a general rule well documented or documented as well as many commercial projects that have project management (for better or worse) and technical writers on staff to do internal as well as external documentation?

Re:Was this really a surprise?

[arkanes]

In my experience, OSS is no more or less well documented than commercial, in house code. Some OSS projects have great docs. Most don't. Most in house software is poorly documented. The number of companies that actually have technical writers on staff for internal software is very, very small. Certainly I've never worked for one.

In fact, I'd go so far as to extend this to software in general. Even when the comments can really matter, like API docs for libraries, the documentation sucks as often as not. I see no advantage to OSS here, but I don't see a disadvantage either.

Re:Was this really a surprise?

[Anonymous Coward]

O.k., perhaps its not a surprise, but in the end the community needs to do away with the 'more eyes make better software' myth in order to move forward. In that sense, it is good that 'professionals' are now pointing out that some of the software out there is actually quite bad and that it is _not_ generally acceptable to not maintain documentation and uphold good project hygiene.

Here's a nice experiment for you:

1. Select a random project, preferably one that's slightly buggy during ordinary use.
2. Subscribe to project's mailing list.
3. Politely inquire if the project has any kind of automated test suite.
4. Observe stumped reaction.
5. Kindly explain the absolute necessity of such a system in any non-trivial app.
6. Go down in flames.

That attitude needs to change.

Re:Was this really a surprise?

[Feztaa]

I'll probably get flamed for this, but at least with OSS, if there's no documentation, you can at least read the code to see what it does. I know, that's no substitute for proper documentation, but with closed source, if there's no documentation, you're just fucked, plain and simple. In OSS, if there's no documentation, you can read the code.

Re:Was this really a surprise?

[Rei]

Well, the importance of a test suite rises dramatically with the complexity of the project. The difficulty of making a test suite increases with the amount of hardware that you need to implement it. When I think of "big" open source projects that aren't very hardware dependant - for example, ITK (the Insight Toolkit), they tend to have nice test suites. Naturally, the little ones don't, but little projects of most things don't have test suites.

I agree, though, that automated test suites are underused. Also, not enough programmers (OSS or otherwise) seem to understand the importance of refactoring.

A message to coders: People, if your function is more than 10 lines long, you should start to consider splitting it. If it's more than 100 lines long, you're probably doing something wrong. If you have the same code written with slight modifications two or more different ways, you're probably doing something wrong. Use templates rather than repeating code if your language supports them. If you ever feel "this should probably be commented more", don't comment it - split it up into functions and let the functions be their own comments (if you have to, comment the functions as well). Use const as much as physically possible (in supporting languages). Use array objects that clean themselves up instead of arrays allocated on-the-fly whenever physically possible. If you find certain variables being used often together, group them into an object. If you find a set of functions operating on an object and only that object, make them member functions. Etc.

Just doing basic refactoring can make code far more organized and readable.

At least...

[BJZQ8]

At least with Open Source Software you CAN maintain it if necessary. With closed source, there is no way to make any changes to old software...and much too often, the companies that make some of the obscure CAD stuff (my field, once) are out of business. At least having it open makes it possible to change something...even if you don't.

Re:At least...

[Anonymous Coward]

As a software engineer, I do like to point out something to you. "At least with open source you CAN maintain it if necessary" is not true! Maintenance takes up a good 60+% of a software cycle. Doing maintenance requires good documentation, fixing a bug is a minor issue, especially when it's easy to track down. Understanding design decisions, understanding architecture so that you can extend the software is a big challenge.

Without appropriate documentations, you end up doing what has been done all over again, studying the software to understand how it works, which can be taxing. Go look at somewhat complex OSS projects, try hacking gcc to spit out a different binary format without reading any documentation. Try understanding postgres without documentation. GUI applications like a CAD system are even harder to make sense out of. If you are actually talented enough, the sheer effort you will poor into understanding the system, you might as well spend it designing from the ground up.

Most people are not hackers, If they were, why would they need source code? crackers don't need source code to add functionality to any system, it's a matter of patching the object code, having a section of the code jump to your own code and return. But it's ugly, having source code makes it a little bit prettier but not much.

Documentation is the key! .segmond

Re:At least...

[charvolant]

At least with Open Source Software you CAN maintain it if necessary.

Sort of ... kind of ...

There comes a point where, particularly without design documentation, the bar is raised so high that the effort involved in maintaining something is more than that involved in moving to a new product. There's a scaling problem here. What works with small, simple direct programs doesn't work with large, complex or indirect programs.

And some OSS code is simply completely undocumented, not even a comment -- apart from the licence. Something I discovered wandering through the XFree86 XKB code.

See http://firstmonday.org/issues/issue4_12/bezroukov/ index.html for a discussion some of the weaknesses of the open source model when it comes to program comprehension.

This assumes commercial software is any better

[Augusto]

And it's often not.

Many of us have and are working in the "real world" out there, and I've been less than impressed with most documentation on large products.

Not to mention design documents, which end up being dead documents that are outdated as soon as the first line of code is written. To many corporations, there's no big incentive to spend so much money on these types of activities when you can have people just churning out code and finishing the darned product in the end.

I'm not saying commericial development is any worse, but I can't say it's any better for sure either.

You don't want to know what goes into sausage

[melted]

I've worked on a major product in CRM market, and let me tell you, don't want to know what goes into sausage. If you knew, you wouldn't touch this code with a 10 foot pole much less bet your company on it.

I'm sure it's the same with ERP. It's just a huge polished turd, but because you don't have the source code you don't know it's a turd. You only see the polish.

Open Source ERP

[stiller]

However, it is still questionable whether systems like ERP could be developed successfully as OSS projects.

I could be mistaken, but isn't Compiere an established OSS ERP implementation?

I think the questin shouldn't be: 'Can software like ERP be developed as OSS?' But rather: 'Are there enough people in the OSS community interested enough to develop this kind of software without any form of financial support?' I think the answer has turned out to be 'no'. The same goes for things like (good) financial software, and anything that would require heaps of work, high precision and coordination, but no spectaculair result for the common man to brag about.

Re:Extremely Ridiculous Publishing

[Bozdune]

Right, and what the hell does "Enterprise Resource Planning" mean?

It used to mean the combination of MRP ("Material Requirements Planning") + Accounting. Then along came PeopleSoft and kinda changed it to HR + Accounting. Then along came Siebel and everyone scurried to make it MRP + HR + accounting + CRM (not quite there yet, though). Then they noticed Kronos and they all scurried to make it MRP + HR + Accounting + CRM + Time & Attendance. And failed, because Time & Attendance is a big pain in the butt. Heh. So they partnered with Kronos instead.

The march of "embrace and extend" continues. Next app up: Expense Reporting (say bye-bye to Concur, etc., that's an easy app). Already on deck: data warehousing (say bye-bye to Cognos, Business Objects, etc., say hello to SAP BW). Soon to come: business process automation (say bye-bye to Ariba, etc.)

And so on, if you believe the pundits.

"ERP" has become a meaningless acronym, an umbrella under which every business app known to man is rammed into the same stinking pile of multi-million dollar shit. At some point it will probably implode from its own weight, and we'll go right back to the "best of breed" interoperable software model.

But it will be a while yet. I suspect in the meantime there will be some Open Source alternatives. I sure hope so.

Re:GUI

[Phleg]

You're actually trying to claim that Winamp's design is good?

Winamp and other players which try to emulate the look and feel of a "new wave" stereo do nothing but piss me off. Stereo systems have the bad interfaces they do because of an inherent lack of physical space; something that's still a concern with computers, but much less of one.

Here's to more programs like Rhythmbox and iTunes which have the *important* controls accessible, allow for easy categorisation of songs, and use screen space nicely. All that without having to resort to 6pt fonts.

Re:maintainability index = bullshit

[pclminion]

Particularly amusing is a term of 50 * sin (sqrt(2.4 * perCM)) in the maintainability index

It's only amusing to people who don't bother to think about why it's there. It's actually a very insightful part of the metric.

First of all, perCM ranges from 0 to 1, not 0 to 100. Yes, that isn't explicitly stated, but it would be ridiculous otherwise. Second, try looking at the damn graph.

As I told somebody else, do it now. Don't pretend to do it, GRAPH the damn thing and look at it: sin(sqrt(2.4*x)) for x=0..1.

That graph makes it completely transparent what they're trying to accomplish with that part of the formula. First off, if comments are 0, the value is 0. Having no comments does not positively impact maintainability! Second, the function PEAKS at around 0.43. This represents an avgCM of 0.03, or 3%. Then, the function begins to go down again, but not as drastically as it rose.

What this is saying is that the benefit of comments has a maximum at around 3%. Having more comments than this tends to DECREASE the maintainability (and this is borne out by experience). However, having too many comments is better than having too few comments, so the function is skewed to the left side by the sqrt() function.

You see, every part of that expression makes total sense if you spend more than 2 nanoseconds thinking about it. Sheesh.

The language is very important

[doc modulo]

- C++ is more readable than assembler
- C# and Java are more readable than C++ ...
- At the end of this list are functional programming languages.

If you can read source more easily, then maintainability will be better.

This article will tell you why you should be interested in functional programming languages. If you're smart and open minded, you will be convinced.

The best functional languages are Haskell and Erlang (click "next" at the bottom of the page).

For example, with Java you prevent bugs by static typing variables, example:

int numberOfTries = 3;

If you later try to fill "numberOfTries" with a string, the compiler will warn you of a bug and you'll have prevented it.

With Haskell, you don't have to type int. Haskell will figure out the type for you, you get the benefit of preventing bugs with the convenience of not having to type variables.

The reason I chose Erlang is because with functional purely functional programming languages like Erlang, you can automatically multitask your program over several CPU's (or this will take minimal effort). Nice feature to have in the future because every CPU manufacturere is going multi-core chip now. Also, you can easily make a server that never goes down with Erlang because your server is automatically clustered. Just plonk down a couple networked PC's and if one dies, the server cluster will just keep on going (a bit slower) until you replaced the power supply of the broken PC.

There are tons of other advantages but, as I said, the above links will convince you if you're smart. Haskell is a bit more academic in nature, they're figuring out the best possible language and Erlang is more polished and ready to go. It was invented by Ericsson to create ultra reliable realtime servers.