Slashdot Mirror
Ohloh Tracks Open Source Developers
eldavojohn writes "The startup company Ohloh has a database listing 70,000 developers working on 11,000 open source projects. Their aim is to 'rank' open source developers, which raises some interesting questions about exactly how useful this tracking company is. Questions like, 'Is there an accurate way beyond word of mouth to measure the importance and skill of a developer?' I found it slightly alarming that, to this site, the number of commits (with input from the number of kudos) tells how good a developer you are."
This reminds me of how academics are increasingly judged. It is more about how many papers and how many other people link to it rather than the quality of each paper's work or the note of the linking party. Accordingly, many authors inflate their 'impact' scores by splitting up papers and publishing nonadvancing science, no-one can blame them for this as many are trying to justify themselves to their departments or are still doing the postdoc merry-go-round looking for new jobs every 18months.
Disclaimer: I co-founded Ohloh. We let people 'claim' development contributions individually. So your Ohloh account can be related to as many (or as few ;-) nicknames on projects as you like.
If you look around, eg javawoman not happy thread it is a bit worrying that an ex cat herder from M$ is not only behind the wheel, but clearly has the same marketing speak to shut up unhappy fodder for the soon to be commercialised service: ohloh_goes_open_source
Software development metrics are not worthless. They are, however, seriously misunderstood. This is partly why we built Ohloh to focus on Open Source: it's the world's largest testbed of available software development metrics.
One challenge to interpreting development metrics is having a clue about what is 'normal'. Just knowing your FOOBAZ count is X doesn't help much. Once you can compare your FOOBAZ count to 100k other developers, it may begin to give you some helpful perspective. Of course, relying on a single metric is myopic - which is why we offer comment ratio, language breakdown etc...
Btw, I agree that human opinion plays a vital role. That's why we also enable people hand out 'Kudos' to their peers - to acknowledge human judgement as well. The Kudo scores are then evaluated using a PageRank-inspired algorithm across all open source contributors.
Exactly. There is much more to get out of the code repository. Check http://www.sourcekibitzer.org/ metrics. It is able to extract developers "know-how" score by crawling the repositories. Much better than simple Lines of Code.