Rating System for Open Source Software

prostoalex writes "Carnegie Mellon University, Intel and SpikeSource are launching a rating system for open source software, New York Times says. OpenBRR 'is being proposed as a new standard model for rating open source software. It is intended to enable the entire community (enterprise adopters and developers) to rate software in an open and standardized way.'"

Comments

FUD from the NYT

[Catamaran]

The author of the article talks as though evaluating software objectively were a problem unique to adopters of Open Source:

Free software, despite the price, can be confusing and costly for corporations to use. A few freely distributed programs, like the Linux operating system and the Apache Web server, have become well known, but most are still unproved.

A more simple and accurate statement would be, "Software can be confusing and costly".

Re:FUD from the NYT

[1000101]

"A more simple and accurate statement would be, "Software can be confusing and costly"."

The entire article is about open source software, not all software. His statement is valid.

As good as CMM?

[orthogonal]

Carnegie-Mellon these are the guys who love to quantify the unquantifiable.

Didn't they also give us the "Capability Maturity Model"? I've seen organizations race to get to CMM-3 or CMM-4, and it's all been a joke.

A bunch of highly paid consultants tell everyone a new way to count beans ("under CMM, we group the beans starting from the right, not the left....").

Promises are made about code auditing, but once the CMM level has been awarded (usually by highly paid consultants who just happen to work with the highly paid consults who "mentored" the company's CMM training), all tat's actually done is that the people doing the real work of writing software are regularly distracted by a clown with a check-list and a clipboard.

Carnegie-Mellon continues to have a fetish for quantifying and for creating check-lists, and middle management continues to have a fetish for anything that allows them to quantify (even spuriously), because it takes the risk and bother out of their jobs.

Middle Manager: "The WordPerfect Project only got a 3 on the Carnegie Mellon software score, but the Clippy Project got a 5! So, it's perfectly safe for me to decide that to disband the WordPerfect Project and devote its resources to the Clippy Project. (And if it turns out later that was a bad decision, they can't fire me, because I relied on hard numbers generated by a known process!"

Re:The problem with rating software...

[broller]

If a user can't figure out how to do X, then X might as well not even be implemented... it amounts to the same thing.

I absolutely agree if we change it to "If all users can't figure out how to do X" but saying "a user" leaves no room for a learning curve. I think it's completely valid to expect that some software will be written that is not necessarily meant for the novice user.

I can see how you might feel that way..

[msimm]

But the truth is propriety software is quite well reviewed (there's an entire industry who makes it their business to review and recommend commercial software, usually somewhat usefully).

Reading reviews of you're favorite Windows Antivirus software or researching an enterprise class database package will turn up a wealth of infomation (of course you still need to dig into it and make the final decision, but some things simply can't be helped:).

OSS software is comparably a total mess, with only certain major projects (and not surpisingly usually projects with some sort commercial support, i.e. apache, mysql, sendmail, etc, but the water gets pretty muddy quickly).

And aside from all those mainly concrete (maybe to you and I anyway) worries there are other concerns when reviewing OSS software for deployment in a business/production environment: support, boss appeal; someone has to sign off even if the software is free, that the software is mature/will meet or exceed your needs and that (if you decide to leave the company) its reasonably well supported (so someone that comes in and doesn't know the particular software has a reasonably good chance of configuring and maintaining it).

Those crazy business people.

Re:I can see how you might feel that way..

[Kludge]

But the truth is propriety software is quite well reviewed

The truth is that some of it is and some of it isn't. And a good number of the "reviews" are done by hacks who are getting paid by the developers for it.

Reading reviews of you're favorite Windows Antivirus software or researching an enterprise class database package will turn up a wealth of infomation ... OSS software is comparably a total mess, with only certain major projects

To the contrary. I can type the name of almost any OSS project into Google and get lots of posts by people who have actually used the software. A lack of hits indicates software that's probably not mature.

Or I can just search the software that I have installed on my computer. Suse has no problem picking out thousands of great software packages for me to use.

... support, ... supported ... maintaining

Support is way over rated. I have never ever gotten good support from any company that produced anything. The only company worth what I've seen them paid was a little 4-5 person Linux consulting/computing biz we contracted. One of the reasons they do so well is that they have the ability to grab and utilize whatever code does the job.

Re:I can see how you might feel that way..

[Tim C]

Support is way over rated. I have never ever gotten good support from any company that produced anything.

There are two aspects to support; actually getting support, so that your problems are fixed quickly and efficiently is one of them. The other, and often just as important to some people, is being able to point the finger at someone else.

It's arse-covering - being able to say "Yes, there's a problem, yes, I recommended the software, but it's not our fault, the support firm is dicking us about - we've paid them, they should fix it!" is a lot better than "Yes, there's a problem, yes, I recommended the software, we're doing our best - sorry"

Generally speaking, managers and clients don't want to hear that in the event of a problem there's a whole bunch of people on the internet who are bound to be able to help. They want to know that someone with the appropriate expertise is going to be dedicated to fixing the problem ASAP. Whether or not that's actually what happens is often beside the point. (Generally if it doesn't, it just means that next time you'll get support from someone else instead)