Slashdot Mirror
Current Scientific Publishing Methods Problematic
A recent examination of current scientific publishing methods shows that they are problematic at best, treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers. "The authors then go on to discuss a variety of economic terms that they think apply to publishing, but the quality of the analogies varies quite a bit. It's easy to accept that the limited number of high-profile publishers act as an oligarchy and that they add value through branding. Some of the other links are significantly more tenuous. The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers — the scientists — can't accurately judge what's significant. "
What is wrong with the free market? When has it ever failed us?
publishers as bidders at an auction, authors as sellers, and the community at large as consumers
Hmmm, there's some term I'm thinking of that deals with people in the middle of the source and the destination that take money for acting as men in the middle when they're not doing anything or providing any service except being in the middle of the transaction. Also, it's beneficial to the sellers & the consumers to eliminate these people. I think they're called 'rich greedy bastards.'
Seriously, hosting a document for me to view doesn't cost $100/mo. so why are you trying to charge me that? I know it's primarily physics but if any other field wanted to pull their heads out of their asses, they would leave the journals to the professors and start up something like arxiv for the rest of humanity that can't afford an outrageous premium!
My work here is dung.
That's exactly why I gave up writing scientific papers and now rob gas stations.
Intron: the portion of DNA which expresses nothing useful.
There are things that you can run like a business, and there are things that you cannot. Without meaning to be political about it, look at what 8 years of running the country like a business with an MBA at the top has got us.
I have not read the article. If the summary is accurate reflection of the authors' point about this, then it is at once misguided, and foolhardy. The purpose of business in a modern capitalist economy is to produce goods at low prices that the consumers can afford, generate enough profit to please the shareholders and to set aside enough money to do research to develop the goods and services to increase these profits and consumer good down the road. Sure, businesses cannot be left alone to do what they wish and government regulations limit unchecked profit-mongering, but the primary purpose of businesses is to establish a market share and earn profit for the shareholders.
Contrast this with the purpose of scientific research. The purpose varies from gaining a more accurate understanding of physical, chemical and biological phenomena to leveraging these phenomena into processes and contraptions that improve the quality of human life (where you lie on this spectrum depends on how pure/fundamental or applied your area of research is). The only shareholders in this process are the authors of scientific work, and their reward varies from just scientific renown to funding for future research or even commercialization of the fruits of their research. However, to achieve the most progress, scientific research tends to be 'open source', in the sense that anyone capable of understanding, and with financial resources to buy access to the journals (if the work is not presented in the growing number of free journals online) can read not only what was done, but also how it was done (something commercial concerns never reveal).
Of course, scientific journals are often run like a business (at least successful and well-renowned ones), but to extend these ideas to the actual business of carrying out research is utterly misguided. The goals of business (from a businessman's pov) and science (from a scientist's pov) are very different. The authors might as well apply these ideas to conduct of a military for all the relevance it has.
There are a host of other objections to such treatment as well, but I will pause here as people know what they are.
Why don't they just start their own wiki? I would find a great deal of value in a wiki moderated by a team of reputable scientists that published their findings to the great peer review workflow.
No, authors don't get paid to appear in journals like Science and Nature. In fact, in most cases the author pays a fee for the space in the journal. It's a total racket.
The benefit to the author is that he can put the paper in his CV. The more big name journals you publish in, the more likely it is that you'll get grant funding and that all important tenure. It's publish or perish out there.
Give me Classic Slashdot or give me death!
As a scientist who has published work in a few journals, I know that the process is arcane and fraught with peril. There are publishers who have axes to grind and it sometimes keeps good information out of the scientific discourse. Of course, I can't offer a real solution because all peer-reviewed journals involve humans with all of our attendant weaknesses.
You don't make money from scientific publications. (On the contrary, you typically pay page charges.) You benefit because no one is going to give you a job or research funding if all you produce is a bunch of self-published manuscripts on your website.
What I'm listening to now on Pandora...
I looked at scientific journals, and I honestly can't see much of an incentive to appear there. I mean sure, you might get published and that's got some merit...
Some merit? In many academic institutions, number of papers published in respectable journals is the preferred metric of performance, and will affect your promotion and the status/funding of your institution.
YMMV depending on the level of enlightenment and subject area of your institution - there are, of course, other aspects which can and should count - but number of papers is the "gold standard" and the safe bet.
If this were a scientific paper, I'd back that up with some references (but my institution definitely doesn't recognise /. karma and mod points, so I can't be arsed).
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
Having a high publication helps a graduate student land a good post-doctoral opportunity. As a post-doc, you'll need a good publication record (Nature, Science, Cell) if you want to land a good faculty position at a top university (tenure track). A scientist that can semi-regularly publish in the top journals will have an easier time earning grants (without such, they wouldn't be able to run a lab). Without a good publication record, a junior faculty won't get tenure (the review is typically 5-7 years for the biological sciences post hire). Publications - no, make that publications in good journals - is everything.
From the scientist's perspective, if they have pure research, then, they can put it on a web site, such as the university web site or even their own, and just skip the b.s.
Any yahoo can post on a website. The reasoning behind scientific journals is that the science is peer reviewed before being accepted. While not everything published on Nature, Science or Cell is top quality work (politics does play a role), the signal to noise ratio is much higher than say, International Immunology. The science presented in the top journals usually has a much higher impact factor than the 'lower' journals; i.e A paper published in Nature Immunology or Nature Medicine typically has a much broader impact on the field than, say, a paper published in Journal of Immunology. That's not to say that the JI paper is worse than the Nat. Imm. or Nat. Med. paper - it's not. Just that the JI paper will likely be much more narrow in scope.
Robin Hood? Is that you?
No, he's just robbin' the 'hood. Easy mistake to make.
Is this serious, or just push back from economists who are upset that a number of papers and editorials have recently appeared in high profile scientific journals questioning the description of economics as science? Allegories, for example, are not scientific.
Information doesn't want anything.
YOU want information to be free.
I'm a grad student in the natural sciences. Some other friends of mine and I started Labmeeting.com because we are so eager to help change the way science gets published.
The current system of peer review is inefficient, arbitrary, and hidden from public view. We definitely need something new, but, as we said in our talk at BioBarCamp a while back, change needs to be gradual enough to preserve consensus.
That's why we're starting by just trying to make research tools that are useful to scientists in their everyday professional lives.
Of course we could all just publish stuff on the Internet.
Faster! Faster! Faster would be better!
Thanks, I thought it up after smoking two joints.
Due to the publish or perish mindset at universities, scientific authors must be prolific. Putt's Law (good book) lampoons this quite well, roughly akin to an Amway style ponzi scheme. Sharing new knowledge with the larger community is no longer among any of the major motives for cranking out papers. Frankly the system punishes those who would compile and distill the huge number of obtuse and often stupid articles into a useable form for us rank and file engineers. Such useful efforts are not "new and novel", despite what would be a great service for those of us doomed to wade through the stacks and stacks of crap papers written in acadamia-ese.
In my job, a hardware design engineer, I find that most of the modern papers are indecipherable and irrelevant at best. Only occasional gems make it through and actually apply to my day job (of designing state of the art T&M hardware). By contrast, the old journals from the 70's and 80's easily have a 10:1 better signal to noise ratio, despite their dated nature.
That this article got published in PLOS Medicine is a data point saying that the publication model for PLOS Medicine is flawed. That's about it.
The authors don't bother to back up any of their assertions. If there is a winner effect, for example, the most prestigious journals should have the highest rate of publication of junk results, whereas lower-ranked journals should be more accurate. So, is this true? Did the authors bother to look, or even to think about and discuss it?
Also, does "overpayment" correspond to "poor quality science" or to "only slightly more cool than the rejected paper, on second thought"?
Now, it is more true in the medical sciences that positive results are published that claim to show p0.05, but are one of a dozen similar studies 11 of which have not shown an effect (i.e. overall there was no significant finding). But this recognition has nothing to do with bidding per se; it's not that the journals are picking the high tail of a distribution of value so much as that they're seeking statistical significance without controlling for the number of times that the study was done.
And as the summary says (which is actually better than the research article itself, IMO), there are a number of other problems.
There are certainly ways that one might seek to improve scientific publishing. But this seems almost entirely off target and/or ill-supported to me.
I love Thomas Jefferson as much as the next American, but there are certain things you listen to him on and some things you don't. Civil liberties, the scope of government, certainly. The economy.... not so much.
Jefferson wanted us to farm our way to victory. Here's some primary source stuff on the subject for your edification/amusement.
Just because he's a founding father doesn't make him a visionary on everything. See also: slavery.
Ceci n'est pas une signature.
One of the major problems with scientific journals and the peer review process is that we have a positive bias for publication, in that you are much more likely to be published if your study has positive results than if it has - equally valid so as not to have everybody else keep doing the same thing and failing - negative results.
Half of getting into Science and Nature is politics, not science.
And just TRY to get something published about improved methodology in statistics for genetics studies ... hah! You have to publish in obscure journals or start your own self-publishing annual or biannual workshop and then attach it to a positive study to get it out there.
-- Tigger warning: This post may contain tiggers! --
If its a reputable journal, the evaluators assume that the articles have been thoroughly peer reviewed , and the quality can be taken as read.
The big assessment exercises (such as the 5-yearly RAE in the UK which determines the research ranking of universities) have to "assess" a metric shedload of papers - so they're not going to spend too much time on each one!
Of course, the reliability of this assumption is legendary.
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
It's also more likely that someone will actually read your paper if it's in a big wide-circulation journal (e.g., Nature) instead of a hard-to-find low-circulation journal. This is particularly true for papers outside your own specialization where you won't necessarily have heard of them at a recent conference. The publication volume is just overwhelming — if you're going to stay current, you need someone else to filter most of the junk for you, and that's the service which selective journals like "Nature" (and review articles) ultimately provide.
-JS
Vanity of vanities, all is vanity...
A softball question. One simple example of the failure of the market is the apparent inability of science publishers, particularly in the pharma area, to publish so-called negative results or to spin negative results as if they are postive. In epidemiology and in pharmacology, negative results are at least as important as postive ones ("first, do no harm"). Yet, the greater economic forces of pharmaceutical sales (and nutricutical sales, and outright woo sales) incent the supression, or simple failure to publish, of such findings in pernicious ways. Check out Ben Goldacre's site (and buy his book while you are there). Tucked away among various rants against, among other things, media coverage of medicine, you will find several discussions about this very phenomonon, and why it is so incredibly bad.