Slashdot Mirror
Current Scientific Publishing Methods Problematic
A recent examination of current scientific publishing methods shows that they are problematic at best, treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers. "The authors then go on to discuss a variety of economic terms that they think apply to publishing, but the quality of the analogies varies quite a bit. It's easy to accept that the limited number of high-profile publishers act as an oligarchy and that they add value through branding. Some of the other links are significantly more tenuous. The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers — the scientists — can't accurately judge what's significant. "
As a scientist who has published work in a few journals, I know that the process is arcane and fraught with peril. There are publishers who have axes to grind and it sometimes keeps good information out of the scientific discourse. Of course, I can't offer a real solution because all peer-reviewed journals involve humans with all of our attendant weaknesses.
Is this serious, or just push back from economists who are upset that a number of papers and editorials have recently appeared in high profile scientific journals questioning the description of economics as science? Allegories, for example, are not scientific.
I'm not convinced that this is true. The only people (besides the library) that receive print publications receive them as a part of their professional membership fees (or as part of a training grant). Most scientists, myself included, simply rely on the email TOC (which we receive much sooner than the hard copy) or go to the websites directly. I suspect that most journals 'make their cash' from institutional subscriptions, professional fees (in the case of Blood), and/or publishing fees.
I'm a grad student in the natural sciences. Some other friends of mine and I started Labmeeting.com because we are so eager to help change the way science gets published.
The current system of peer review is inefficient, arbitrary, and hidden from public view. We definitely need something new, but, as we said in our talk at BioBarCamp a while back, change needs to be gradual enough to preserve consensus.
That's why we're starting by just trying to make research tools that are useful to scientists in their everyday professional lives.
That this article got published in PLOS Medicine is a data point saying that the publication model for PLOS Medicine is flawed. That's about it.
The authors don't bother to back up any of their assertions. If there is a winner effect, for example, the most prestigious journals should have the highest rate of publication of junk results, whereas lower-ranked journals should be more accurate. So, is this true? Did the authors bother to look, or even to think about and discuss it?
Also, does "overpayment" correspond to "poor quality science" or to "only slightly more cool than the rejected paper, on second thought"?
Now, it is more true in the medical sciences that positive results are published that claim to show p0.05, but are one of a dozen similar studies 11 of which have not shown an effect (i.e. overall there was no significant finding). But this recognition has nothing to do with bidding per se; it's not that the journals are picking the high tail of a distribution of value so much as that they're seeking statistical significance without controlling for the number of times that the study was done.
And as the summary says (which is actually better than the research article itself, IMO), there are a number of other problems.
There are certainly ways that one might seek to improve scientific publishing. But this seems almost entirely off target and/or ill-supported to me.
If its a reputable journal, the evaluators assume that the articles have been thoroughly peer reviewed , and the quality can be taken as read.
The big assessment exercises (such as the 5-yearly RAE in the UK which determines the research ranking of universities) have to "assess" a metric shedload of papers - so they're not going to spend too much time on each one!
Of course, the reliability of this assumption is legendary.
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
It's also more likely that someone will actually read your paper if it's in a big wide-circulation journal (e.g., Nature) instead of a hard-to-find low-circulation journal. This is particularly true for papers outside your own specialization where you won't necessarily have heard of them at a recent conference. The publication volume is just overwhelming — if you're going to stay current, you need someone else to filter most of the junk for you, and that's the service which selective journals like "Nature" (and review articles) ultimately provide.
-JS
Vanity of vanities, all is vanity...