Slashdot Mirror
The Unsung Heroes of Scientific Software (nature.com)
An anonymous reader sends this news from Nature: For researchers who code, academic norms for tracking the value of their work seem grossly unfair. They can spend hours contributing to software that underpins research, but if that work does not result in the authorship of a research paper and accompanying citations, there is little way to measure its impact. ... Enter Depsy, a free website launched in November 2015 that aims to "measure the value of software that powers science."
[Postdoc researcher Klaus] Schliep's profile on that site shows that he has contributed in part to seven software packages, and that he shares 34% of the credit for phangorn. Those packages have together received more than 2,600 downloads, have been cited in 89 open-access research papers and have been heavily recycled for use in other software — putting Schliep in the 99th percentile of all coders on the site by impact.
[Postdoc researcher Klaus] Schliep's profile on that site shows that he has contributed in part to seven software packages, and that he shares 34% of the credit for phangorn. Those packages have together received more than 2,600 downloads, have been cited in 89 open-access research papers and have been heavily recycled for use in other software — putting Schliep in the 99th percentile of all coders on the site by impact.
And yes, we have to fix this somehow, sooner or later. At least in my area (Computer Graphics), complex, cutting-edge research increasingly builds on highly specialised software stacks that are being maintained by researchers the community. Whose efforts usually are not appropriately rewarded. An example is meshlab: that thing is hugely useful for lots of people - but in retrospect, the main author has all but described developing it as a mistake. As it cost him too much time that he would have needed elsewhere in his career efforts.
A few guys are lucky, like Wenzel Jakob: he both wrote Mitsuba (the extremely useful research path tracer that everyone uses these days to build on), as well as a couple of high profile publications that set him up for an academic career. But in a lot of cases, even very good researchers only have the time and brainpower for one thing at a time: software *or* publications. We need both of them, but only reward one category. Bad move, systemically speaking.
...of the software and then request citations as parts of the license agreement. It's not a guarantee, and yes some will fail to cite properly, but at a present a short (possibly even conference) paper with a citation request really is you're best bet to get some credit.
The difficult part is finding a journal that will accept a description-of-software type paper *and* has a decent ranking. I'm somewhat lucky to have such a journal in my field, but I know that other fields are not so fortunate.
It's crap. It combines the worst of CentOS / Red Hat Linux with the pitfalls of academically managed IT.
Fedora is much better, where bugs do, gasp, get fixed in a timely manner. If you can manage the shortish release cycle.
You know, I see this a lot opinion from youngsters who have never administered mission critical systems. For your basic web server & SQL database, Fedora is fine - but how many copies of Fedora would ever get installed on the ISS?
Sendmail is like emacs: A nice operating system, but missing an editor and a MTA.
You have a good point with regard to "where does it end". But you seem to have mis-understood what this is about: this is not about automatically getting credits or co-authorship whenever someone uses the software you wrote. That would indeed be strange.
Rather, the point they are making is that being the (co-)author of such a software should count as much as being the (co-)author a paper, with regard to getting tenure, or for general performance reviews within academia. Sort-of-similar metrics could be applied to this as they are applied to publications: how many people downloaded it (probably a fairly bad metric), how many papers mention using the software for experiments (probably better). The whole thing is a bit shaky, of course: if someone writes some fairly trivial piece of software that ends up being ubiquitous, they get lots of credit - while highly specialised software that took ages and lots of brainpower to develop scores low.
But this dilemma is of course equally true for publications themselves: citation counts and such are not very good metrics, either.
A lot of scientific software is extremely specific and designed to enable very specific types of research. There are many fields where there are 1 or 2 pieces of software exist and they where written inside the community.
They most certainly should get credit for writing that software and enabling very specific research since in many cases without the software the research would not even exist.
Computer modeling for biotech drug manufacturing is HARD!
Yes, he chose to do it, but it's a choice that for many people means the end of their academic career. There's a huge amount of great scientific software coming out of the universities, but it tends to count very little toward a new faculty's tenure review. If you don't get tenure; you're fired. The unique issue for software is that it takes a very long time to write well and drains resources for its maintenance. For tenure, you're typically evaluated on your ability to get grants, research papers, departmental service (conferences and research groups), and teaching in about that order, though, it varies. Notice that software isn't in there. Yes, you can get a publication out of a piece of scientific software, but you're really looking at a single paper, or 2-3 if you really milk it. Tenure track faculty need to be publishing 5-10 papers per year over the traditional 7 year probationary period.
Now, is this a cultural problem at universities? Absolutely. But those of us in this situation have every reason to gripe. It's not that we're not getting paid. It's that we're running the very real possibility that we're going to get fired after providing an extremely valuable service to the greater community.
Exactly this. Including the authors of Word would indeed be absurd. But the authors of software that is *directly pertinent* to the research in questions should get some sort of credit.
Of course, this has fuzzy borders: to stay with the graphics examples, meshlab is probably too general to get much credit these days (it is a MS Word of mesh processing, so to speak). Mitsuba, on the other hand, probably counts for a lot of research papers where it was used - for a number of tasks, no comparable open-source software exists.
Please stop the bullshit of heroes with superpower in science, as science never was about glorifying people and personality cults. Leave that to the entertainment industry.
The whole science star system is doing much more harm than good to the actual scientific outcomes.
People tend to optimize the metric that is used. If that metric is popularity, they'll do as much as they can to become popular, with no correlation whatsoever to the importance of their original field. Publishing crappy results on a new dataset so that everyone can beat you gets you more citations than providing insights to why some methods work and some don't. Publishing a shitty software that allows a million master student to make up wrong results for the master's thesis get you more download than writing a correct implementation of uncommon algorithms.
I went into science because I didn't give a shit about the smoke and mirrors that are so important in other fields. Most of my best technical students now are just disgusted by the "appearance prevails" mentality that is at the core of other disciplines. Please leave science as it ought to be: efficient but careless about the image.
Video of some good progressive thrash music
This is a horrible comparison. Shakespeare could have done his plays in any theater.
I am talking about custom written software that solves an EXTREMELY narrow niche and without it a lot of the research in the field would not even happen.
I
Computer modeling for biotech drug manufacturing is HARD!
Well the software I am talking about is the result of a lot of very difficult research that has taken many years to do. The people writing the software are doing the research.
There is no simple algorithm that a professor came up. It is a complex physical model and it is highly non-trivial to figure out which parts need to be modeled and why to get the correct physical behavior.
Others use the software to try and solve a specific problem that would have been impossible without a lot of very hard work in getting the technology to do it. That work should be given credit since in the academic community credit is all that really matters for your career long term. That is one reason I don't like academia and look forward to finishing and going back into corporate research.
Computer modeling for biotech drug manufacturing is HARD!
How do you get listed? My software project isn't listed. We have a few hundred users and we're nearing about 1000 citations. It's an open source project on GitHub.
How do we tell Depsy about it?
The Spice electrical circuit simulation software was developed in FORTRAN on several platforms (including VAX VMS) in the 1970s. I managed to compile it for Linux and Windows years ago, and I host the source and binaries on a laptop in my basement.
This specific version is in many circuits textbooks - newer versions are not compatible with the syntax of this release. I see a fair amount of traffic for it. I should probably spend some time on a nicer HTML5 download page.