Slashdot Mirror
The Unsung Heroes of Scientific Software (nature.com)
An anonymous reader sends this news from Nature: For researchers who code, academic norms for tracking the value of their work seem grossly unfair. They can spend hours contributing to software that underpins research, but if that work does not result in the authorship of a research paper and accompanying citations, there is little way to measure its impact. ... Enter Depsy, a free website launched in November 2015 that aims to "measure the value of software that powers science."
[Postdoc researcher Klaus] Schliep's profile on that site shows that he has contributed in part to seven software packages, and that he shares 34% of the credit for phangorn. Those packages have together received more than 2,600 downloads, have been cited in 89 open-access research papers and have been heavily recycled for use in other software — putting Schliep in the 99th percentile of all coders on the site by impact.
[Postdoc researcher Klaus] Schliep's profile on that site shows that he has contributed in part to seven software packages, and that he shares 34% of the credit for phangorn. Those packages have together received more than 2,600 downloads, have been cited in 89 open-access research papers and have been heavily recycled for use in other software — putting Schliep in the 99th percentile of all coders on the site by impact.
And yes, we have to fix this somehow, sooner or later. At least in my area (Computer Graphics), complex, cutting-edge research increasingly builds on highly specialised software stacks that are being maintained by researchers the community. Whose efforts usually are not appropriately rewarded. An example is meshlab: that thing is hugely useful for lots of people - but in retrospect, the main author has all but described developing it as a mistake. As it cost him too much time that he would have needed elsewhere in his career efforts.
A few guys are lucky, like Wenzel Jakob: he both wrote Mitsuba (the extremely useful research path tracer that everyone uses these days to build on), as well as a couple of high profile publications that set him up for an academic career. But in a lot of cases, even very good researchers only have the time and brainpower for one thing at a time: software *or* publications. We need both of them, but only reward one category. Bad move, systemically speaking.
You have a good point with regard to "where does it end". But you seem to have mis-understood what this is about: this is not about automatically getting credits or co-authorship whenever someone uses the software you wrote. That would indeed be strange.
Rather, the point they are making is that being the (co-)author of such a software should count as much as being the (co-)author a paper, with regard to getting tenure, or for general performance reviews within academia. Sort-of-similar metrics could be applied to this as they are applied to publications: how many people downloaded it (probably a fairly bad metric), how many papers mention using the software for experiments (probably better). The whole thing is a bit shaky, of course: if someone writes some fairly trivial piece of software that ends up being ubiquitous, they get lots of credit - while highly specialised software that took ages and lots of brainpower to develop scores low.
But this dilemma is of course equally true for publications themselves: citation counts and such are not very good metrics, either.
A lot of scientific software is extremely specific and designed to enable very specific types of research. There are many fields where there are 1 or 2 pieces of software exist and they where written inside the community.
They most certainly should get credit for writing that software and enabling very specific research since in many cases without the software the research would not even exist.
Computer modeling for biotech drug manufacturing is HARD!
Exactly this. Including the authors of Word would indeed be absurd. But the authors of software that is *directly pertinent* to the research in questions should get some sort of credit.
Of course, this has fuzzy borders: to stay with the graphics examples, meshlab is probably too general to get much credit these days (it is a MS Word of mesh processing, so to speak). Mitsuba, on the other hand, probably counts for a lot of research papers where it was used - for a number of tasks, no comparable open-source software exists.
Please stop the bullshit of heroes with superpower in science, as science never was about glorifying people and personality cults. Leave that to the entertainment industry.
The whole science star system is doing much more harm than good to the actual scientific outcomes.
People tend to optimize the metric that is used. If that metric is popularity, they'll do as much as they can to become popular, with no correlation whatsoever to the importance of their original field. Publishing crappy results on a new dataset so that everyone can beat you gets you more citations than providing insights to why some methods work and some don't. Publishing a shitty software that allows a million master student to make up wrong results for the master's thesis get you more download than writing a correct implementation of uncommon algorithms.
I went into science because I didn't give a shit about the smoke and mirrors that are so important in other fields. Most of my best technical students now are just disgusted by the "appearance prevails" mentality that is at the core of other disciplines. Please leave science as it ought to be: efficient but careless about the image.
Video of some good progressive thrash music