Slashdot Mirror
Current Scientific Publishing Methods Problematic
A recent examination of current scientific publishing methods shows that they are problematic at best, treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers. "The authors then go on to discuss a variety of economic terms that they think apply to publishing, but the quality of the analogies varies quite a bit. It's easy to accept that the limited number of high-profile publishers act as an oligarchy and that they add value through branding. Some of the other links are significantly more tenuous. The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers — the scientists — can't accurately judge what's significant. "
What is wrong with the free market? When has it ever failed us?
A recent examination of current scientific publishing methods show that it is problematic at best. Treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers.
Agreed. I think we need to switch this whole process to Vickrey Auctions. Then you can explain to the authors of the papers that they will receive $75 for their paper instead of $100 because whoever bid $100 was gaming the system. Why is it suddenly so popular to turn everything possible into an auction system with 75 different flavors of said auction system?
My work here is dung.
publishers as bidders at an auction, authors as sellers, and the community at large as consumers
Hmmm, there's some term I'm thinking of that deals with people in the middle of the source and the destination that take money for acting as men in the middle when they're not doing anything or providing any service except being in the middle of the transaction. Also, it's beneficial to the sellers & the consumers to eliminate these people. I think they're called 'rich greedy bastards.'
Seriously, hosting a document for me to view doesn't cost $100/mo. so why are you trying to charge me that? I know it's primarily physics but if any other field wanted to pull their heads out of their asses, they would leave the journals to the professors and start up something like arxiv for the rest of humanity that can't afford an outrageous premium!
My work here is dung.
That's exactly why I gave up writing scientific papers and now rob gas stations.
Intron: the portion of DNA which expresses nothing useful.
No, scientists do not get paid for the papers they write by the journals. Their reputation though is almost solely built upon their published work. Also, universities often give a bonus if a certain number of publications is written during a year. Same with government grants.
Think about this: Axe and Dove are actually the same company. Vincent L.B.
There are things that you can run like a business, and there are things that you cannot. Without meaning to be political about it, look at what 8 years of running the country like a business with an MBA at the top has got us.
I have not read the article. If the summary is accurate reflection of the authors' point about this, then it is at once misguided, and foolhardy. The purpose of business in a modern capitalist economy is to produce goods at low prices that the consumers can afford, generate enough profit to please the shareholders and to set aside enough money to do research to develop the goods and services to increase these profits and consumer good down the road. Sure, businesses cannot be left alone to do what they wish and government regulations limit unchecked profit-mongering, but the primary purpose of businesses is to establish a market share and earn profit for the shareholders.
Contrast this with the purpose of scientific research. The purpose varies from gaining a more accurate understanding of physical, chemical and biological phenomena to leveraging these phenomena into processes and contraptions that improve the quality of human life (where you lie on this spectrum depends on how pure/fundamental or applied your area of research is). The only shareholders in this process are the authors of scientific work, and their reward varies from just scientific renown to funding for future research or even commercialization of the fruits of their research. However, to achieve the most progress, scientific research tends to be 'open source', in the sense that anyone capable of understanding, and with financial resources to buy access to the journals (if the work is not presented in the growing number of free journals online) can read not only what was done, but also how it was done (something commercial concerns never reveal).
Of course, scientific journals are often run like a business (at least successful and well-renowned ones), but to extend these ideas to the actual business of carrying out research is utterly misguided. The goals of business (from a businessman's pov) and science (from a scientist's pov) are very different. The authors might as well apply these ideas to conduct of a military for all the relevance it has.
There are a host of other objections to such treatment as well, but I will pause here as people know what they are.
Why don't they just start their own wiki? I would find a great deal of value in a wiki moderated by a team of reputable scientists that published their findings to the great peer review workflow.
They could do more to help. At $15-25 a read most scientific papers are beyond the means of ordinary folks. Even if you are at a university or institute that subscribes to some journals the chances of getting what you want if you're casually browsing across disciplines (from which many great creative insights come) is slim.
I think the search engines contribute to this problem because the search algorithms do not take account of availability. Search for any specialist subject and you will see the first 10 pages dominated by links to the gatekeepers of knowledge who offer shallow abstracts and for-pay access. Somewhere after page 10 you will find an obscure link to the actual authors website where you can get the paper.
Google has the technology to rank availability above kissing the publishers asses. I can only assume they choose not to act in the best interests of the community.
No, authors don't get paid to appear in journals like Science and Nature. In fact, in most cases the author pays a fee for the space in the journal. It's a total racket.
The benefit to the author is that he can put the paper in his CV. The more big name journals you publish in, the more likely it is that you'll get grant funding and that all important tenure. It's publish or perish out there.
Give me Classic Slashdot or give me death!
As a scientist who has published work in a few journals, I know that the process is arcane and fraught with peril. There are publishers who have axes to grind and it sometimes keeps good information out of the scientific discourse. Of course, I can't offer a real solution because all peer-reviewed journals involve humans with all of our attendant weaknesses.
You don't make money from scientific publications. (On the contrary, you typically pay page charges.) You benefit because no one is going to give you a job or research funding if all you produce is a bunch of self-published manuscripts on your website.
What I'm listening to now on Pandora...
I looked at scientific journals, and I honestly can't see much of an incentive to appear there. I mean sure, you might get published and that's got some merit...
Some merit? In many academic institutions, number of papers published in respectable journals is the preferred metric of performance, and will affect your promotion and the status/funding of your institution.
YMMV depending on the level of enlightenment and subject area of your institution - there are, of course, other aspects which can and should count - but number of papers is the "gold standard" and the safe bet.
If this were a scientific paper, I'd back that up with some references (but my institution definitely doesn't recognise /. karma and mod points, so I can't be arsed).
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
Having a high publication helps a graduate student land a good post-doctoral opportunity. As a post-doc, you'll need a good publication record (Nature, Science, Cell) if you want to land a good faculty position at a top university (tenure track). A scientist that can semi-regularly publish in the top journals will have an easier time earning grants (without such, they wouldn't be able to run a lab). Without a good publication record, a junior faculty won't get tenure (the review is typically 5-7 years for the biological sciences post hire). Publications - no, make that publications in good journals - is everything.
From the scientist's perspective, if they have pure research, then, they can put it on a web site, such as the university web site or even their own, and just skip the b.s.
Any yahoo can post on a website. The reasoning behind scientific journals is that the science is peer reviewed before being accepted. While not everything published on Nature, Science or Cell is top quality work (politics does play a role), the signal to noise ratio is much higher than say, International Immunology. The science presented in the top journals usually has a much higher impact factor than the 'lower' journals; i.e A paper published in Nature Immunology or Nature Medicine typically has a much broader impact on the field than, say, a paper published in Journal of Immunology. That's not to say that the JI paper is worse than the Nat. Imm. or Nat. Med. paper - it's not. Just that the JI paper will likely be much more narrow in scope.
The authors then go on to discuss a variety of economic terms that they think apply to publishing
Which is probably the most problematic point.
While even in the current crisis "market models" are still hip, they don't give the answers to all questions. A scientific conclusion that starts with "if X were a market" must question, among all the other validations, the "if" part as well.
And while economy provides interesting theories that are helpful in many cases - just like evolution, it does not fit everywhere. So the very first thing that would've to be established is that the model fits.
In this case, I've not seen enough of that, so any conclusions drawn are meaningless until then.
Assorted stuff I do sometimes: Lemuria.org
Robin Hood? Is that you?
No, he's just robbin' the 'hood. Easy mistake to make.
Is this serious, or just push back from economists who are upset that a number of papers and editorials have recently appeared in high profile scientific journals questioning the description of economics as science? Allegories, for example, are not scientific.
Information doesn't want anything.
YOU want information to be free.
Instead of claiming that the whole system is broken, just fix the breakdowns in the current system.
The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers â" the scientists â" can't accurately judge what's significant.
Any given paper does suffer from 'uncertain valuation', but uncertain doesn't mean the consumers have no clue.
Consider the impact factor of the journal to which the paper was submitted, the reputation of the author, the actual evidence that has been presented in the paper, the fact that the paper has undergone peer review, and what impact the paper would have its claims were to have merit.
In combination, these factors allow better papers to tend to float to the surface. This is fairly typical of a market.
Most fields have decent market regulation built-in, in the form of peer review and independent verification of results.
Politically charged fields (global warming) tend toward unregulated market behavior. Papers are no longer selected based on scientific merit, but instead on hype/scare factor. This makes the value of a paper much more uncertain, and leads to a nasty failure mode (see the current world economy).
There are models that do not have such nasty failure modes (or at least have very different failure modes). Usually, these also fail to produce such good results in the common case.
I dislike the current scientific publishing system because the publishers tend to be paid by both the author and the consumer, and can generally force the author to relinquish copyright. However, the quality of the system seems to me to be far better than this article supposes.
-V-
Who can decide a priori? Nobody.
-Sartre
I'm a grad student in the natural sciences. Some other friends of mine and I started Labmeeting.com because we are so eager to help change the way science gets published.
The current system of peer review is inefficient, arbitrary, and hidden from public view. We definitely need something new, but, as we said in our talk at BioBarCamp a while back, change needs to be gradual enough to preserve consensus.
That's why we're starting by just trying to make research tools that are useful to scientists in their everyday professional lives.
Of course we could all just publish stuff on the Internet.
Faster! Faster! Faster would be better!
Thanks, I thought it up after smoking two joints.
Due to the publish or perish mindset at universities, scientific authors must be prolific. Putt's Law (good book) lampoons this quite well, roughly akin to an Amway style ponzi scheme. Sharing new knowledge with the larger community is no longer among any of the major motives for cranking out papers. Frankly the system punishes those who would compile and distill the huge number of obtuse and often stupid articles into a useable form for us rank and file engineers. Such useful efforts are not "new and novel", despite what would be a great service for those of us doomed to wade through the stacks and stacks of crap papers written in acadamia-ese.
In my job, a hardware design engineer, I find that most of the modern papers are indecipherable and irrelevant at best. Only occasional gems make it through and actually apply to my day job (of designing state of the art T&M hardware). By contrast, the old journals from the 70's and 80's easily have a 10:1 better signal to noise ratio, despite their dated nature.
That this article got published in PLOS Medicine is a data point saying that the publication model for PLOS Medicine is flawed. That's about it.
The authors don't bother to back up any of their assertions. If there is a winner effect, for example, the most prestigious journals should have the highest rate of publication of junk results, whereas lower-ranked journals should be more accurate. So, is this true? Did the authors bother to look, or even to think about and discuss it?
Also, does "overpayment" correspond to "poor quality science" or to "only slightly more cool than the rejected paper, on second thought"?
Now, it is more true in the medical sciences that positive results are published that claim to show p0.05, but are one of a dozen similar studies 11 of which have not shown an effect (i.e. overall there was no significant finding). But this recognition has nothing to do with bidding per se; it's not that the journals are picking the high tail of a distribution of value so much as that they're seeking statistical significance without controlling for the number of times that the study was done.
And as the summary says (which is actually better than the research article itself, IMO), there are a number of other problems.
There are certainly ways that one might seek to improve scientific publishing. But this seems almost entirely off target and/or ill-supported to me.
I love Thomas Jefferson as much as the next American, but there are certain things you listen to him on and some things you don't. Civil liberties, the scope of government, certainly. The economy.... not so much.
Jefferson wanted us to farm our way to victory. Here's some primary source stuff on the subject for your edification/amusement.
Just because he's a founding father doesn't make him a visionary on everything. See also: slavery.
Ceci n'est pas une signature.
One of the major problems with scientific journals and the peer review process is that we have a positive bias for publication, in that you are much more likely to be published if your study has positive results than if it has - equally valid so as not to have everybody else keep doing the same thing and failing - negative results.
Half of getting into Science and Nature is politics, not science.
And just TRY to get something published about improved methodology in statistics for genetics studies ... hah! You have to publish in obscure journals or start your own self-publishing annual or biannual workshop and then attach it to a positive study to get it out there.
-- Tigger warning: This post may contain tiggers! --
No, scientists do not get paid for the papers they write by the journals. Their reputation though is almost solely built upon their published work. Also, universities often give a bonus if a certain number of publications is written during a year. Same with government grants.
And you don't get tenure if you don't have a good publishing record for your papers, with yourself as a lead for a few, and co-author for some of your earlier work.
-- Tigger warning: This post may contain tiggers! --
If its a reputable journal, the evaluators assume that the articles have been thoroughly peer reviewed , and the quality can be taken as read.
The big assessment exercises (such as the 5-yearly RAE in the UK which determines the research ranking of universities) have to "assess" a metric shedload of papers - so they're not going to spend too much time on each one!
Of course, the reliability of this assumption is legendary.
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
It's also more likely that someone will actually read your paper if it's in a big wide-circulation journal (e.g., Nature) instead of a hard-to-find low-circulation journal. This is particularly true for papers outside your own specialization where you won't necessarily have heard of them at a recent conference. The publication volume is just overwhelming — if you're going to stay current, you need someone else to filter most of the junk for you, and that's the service which selective journals like "Nature" (and review articles) ultimately provide.
-JS
Vanity of vanities, all is vanity...
A softball question. One simple example of the failure of the market is the apparent inability of science publishers, particularly in the pharma area, to publish so-called negative results or to spin negative results as if they are postive. In epidemiology and in pharmacology, negative results are at least as important as postive ones ("first, do no harm"). Yet, the greater economic forces of pharmaceutical sales (and nutricutical sales, and outright woo sales) incent the supression, or simple failure to publish, of such findings in pernicious ways. Check out Ben Goldacre's site (and buy his book while you are there). Tucked away among various rants against, among other things, media coverage of medicine, you will find several discussions about this very phenomonon, and why it is so incredibly bad.
Wrong.
The adage expresses an Essentialist sentiment. Fire _reaches_ for the sky. Water _seeks_ the lowest level.
In this personification, "information WANTS to be free", there is a warning and deep truth (that any intelligence worker understands) - the tendancy of information to move, as to entropy, towards the greatest degree of freedom. In other words, human desires aside, whether you want it, or I want it, any restriction on the flow of information is imperfect and temporary at best.
"... treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers."
Not really, not at least in biomed papers.
With those the scientist actually *pays* the publisher to print the article. A paper, especially if it contains colour images (microscopic slides, colourful graphs etc) can set you back by several thousand dollars. That's why there's a little disclaimer under each article that states that since the author paid for the publication, the article legally is paid advertisement.
So no, the authors are not sellers. Only the publishers are the sellers, selling article space to the scientists, advertisement space to corporations and the end result back to scientists. The scientists who publish don't even get the complimentary free copy, although as an author, you can ask them to send you the PDF that you sent them in the first place; now this service is free. On the other hand, if you want the whole magazine on your bookshelf, fork out $200 for a single issue.
It is a wonderful business model:
- Get the article from scientist (free)
- Send it to other scientists for peer review (free)
- Accept it and charge the author (income)
- Sell the advertisement space, at least 50% of the mag (income)
- Print the mag (expense)
- Sell the mag *way* above production/distribution cost (income)
- Keep the copyright to every article so that the author can't republish it without paying you (possible income)
- Profit!!!
The Underpants Gnomes had no clue about business...
Plus there are further tricks - if you manage to bribe the execs of some research association so that membership in the assocation also means a compulsory subscription to your periodical, then the customer base is guaranteed. Of course it is usually sold as "we had to increase our membership fee, but now membership also includes a complimentary subscription to magazine X". This of course increases your readership, making your ranking higher among the sci mag list.
Ah, yes. Chances are that the money they take from the scientist is public money from some research grant. Yet the copyright to the article belongs to the publisher, a private organisation that actually has very little to do with scientific research and everything to do with making profit. That's clever! Even better, that when research grants are awarded, they check your publication record. If you published your articles on the Web, that is worth exactly nothing. Publishing a handful of articles in a couple of those high-ranking expensive publications significantly increases your chance to get a grant, so that you can publish some more...
Imagine if the **AA could find a way to implement this! Every musician and actor paying them to be included on a CD or in a film, plus selling ads even smack in the middle of a CD (music stops, someone screams about the advantages of washing powder X, music continues) and making it compulsory that members of any civil organisation buy a certain number of CDs or DVDs every month. Those Hollywood dudes know diddly squat about making profit.