Slashdot Mirror
Peer-Reviewed Research Over The Web
bhoman writes "The San Francisco Chronicle (sfgate.com) has an article today about Stanford biochemist Patrick O. Brown, who helped develop low-cost DNA microarrays for gene research. He is seeking $20M to start a foundation that would fund peer-review of research papers and then make them available for free over the web, thereby avoiding the high-cost of subscriptions common in existing research publications. Predictably, some publishers seem to be warning that their publishing model is hard to improve upon.
The article mentions that a previous effort by Brown and others, The Public Library of Science garnered the signatures of 30,000 supporters, but then implies that it basically failed, suggesting that academics need the journals more than vice versa.
Sounds like Brown's idea is exactly what the web is made for."
Good idea, but the article says it all:
"It's publish or perish," Stern said. "As long as we have promotion and tenure tied to publishing, change won't work."
sadly.
That would have been great.
Yes!! I heard many years ago about efforts to replace the completely broken journal/subscription of today with a peer-review/web-of-trust. Problem is, there's a market of sci. journals for a reason! Scientific excellence became a screwed notion immediately after the WWII years, when the Iron Courtain broke in two (reasonably equal halves) the older web of trust. Scientific excellence has now to be quantitatively defined, by number of articles published, especially. This is very wrong, of course.
:-/
Thus, we first need to change our perception of scientific excellence and _then_ put in place a peer-review mechanism. And the new perceptions needs a peer-review mechanism in order to be reformed properly. Hen or egg?
CiteSeer is great but mostly links to articles published elsewhere. A better example (physics, math, computer science, and nonlinear systems) is lanl.arxiv.org (also at xxx.lanl.gov). Does anyone know a good history of this database and how these disciplines have seemed to escape from published journals as being the only available source for published articles?
My next sig will be ready soon, but friends can beat the rush!
Not that Geometry and Topology is the only one, but this is a very good example.
Speaking as someone who just submitted an article to Science for review, I would have to say the primary hurdle that Brown will have to overcome is pedigree. Journals like Science and Nature have a history and an editorial board that ensure a rigorous review process that ultimately presents the best of the best and the most significant science to the scientific community. Publication of ones work in journals of this caliber are important to your career, and given their wide distribution can be critical in obtaining funding.
The implications of this are far more than simple "peer mating" and "copy editing" as one other poster suggested. Granted, there is nothing that can keep an online journal from eventually becoming the place to publish, but it will take time and a commitment to excellence that will have to be maintained for scientists to become comfortable in submitting their hard earned results to. Publication of observational science will not cut it. The implication of this is that since most scientists view Science and Nature (among a select few) as the pre-eminent journals, they will be concerned about submitting the most significant scientific results to a new online journal. Typically from what I have seen, when one gets rejected from the more prestigious journals, you start moving down your ladder of preference until somebody accepts your article. Of course results targeted for specific journals with a readership that would be interested in your results always matters and this is where online journals stand the best chance of making it as opposed to large pre-eminent general interest scientific journals such as Science and Nature.
Visit Jonesblog and say hello.
The whole point behind science, its entire reason to exist, is to provide us with a predictive explanation of the world around us. It needs the "many eyes" approach more than just about any other human endeavor, because the entire point is to model the real world and you can't do that without a lot of observation.
Of course, science has also proven to be useful, and that's been something of an anathema to it. The reason is that things which are useful are things which people (corporations in particular) want to capitalize on in an exclusive way. It seems to me that there was a time when everyone recognized the truth that public disclosure and widespread collaboration is necessary for science to advance.
That no longer seems to be the case from where I sit. Today, corporations fund a great deal of research at the university level, and there is a great deal of pressure from both corporations and from the universities themselves to keep ongoing research under wraps as much as possible, in order to maximize the chances not just of publishing but also of getting patents on the results (which are probably then transferred to the corporations that funded the research).
Those people in the scientific community that I've spoken to believe, to a man, in collaboration with their peers in order to further science. They're held up by the people that fund their research.
How does this relate to publishing on the web? Well, publishing on the web removes a lot of the exclusivity that currently exists, so there will naturally be opposition to it from those who benefit from the control they have over scientific publishing right now. And my cynical mind tells me that there's a good chance that those who fund research exert an additional level of control through the current publishers (it would make sense, right?). It's my hope that research over the web will help in reducing the amount of exclusivity that seems to exist currently in the scientific community. But then, that's probably wishful thinking.
As long as that level of exclusivity exists, our understanding of the universe won't advance as quickly as it might otherwise. Perhaps things have always been this way and I'm just pining for better days that have never existed. But if there's even a chance that publishing on the web will improve the amount of collaboration and peer review, I think it's worth doing.
But this proposal doesn't do much to help with that, because it still concentrates the power of peer review and publishing into the hands of a few. What prevents researchers from collaborating with each other, getting peer review from each other, and publishing on the web directly, instead of going through middlemen like they do now? Seems to me that they're being held up by those that fund the research. And unfortunately, this proposal wouldn't change that.
Yes, it's a step in the right direction, and the current scientific publishers need some competition. But it shouldn't be seen as the end goal.
Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
Anyway, I wish Brown all the best success, but as others have mentioned, it's a somewhat harder problem than it first seems (at least he's asking for $20 million, which is somewhat realistic for handling real peer review for a substantial number of articles - 10's of thousands at least).
What's behind this nebulous "peer review" concept, at least for us, is a complex and historically based system of checks and balances involving communications between authors, editors, and (anonymous and non-anonymous) reviewers; we're essentially a legal/court system for scientific articles. There's a lot of information-related issues in there, and information technology helps a lot (that's the part I'm involved in). But fundamentally, at least the way we do it, there needs to be a paid, responsible human being reading most communications and monitoring the process, and as far as we've been able to work out, you can't get the cost under about $500 or so per article.
Now, just distributing the papers can be done essentially for free (to as many people as would want to read for about $1-5 per article, for hardware, software, disk, network, etc.) which is what the famous physics e-print archive does so well. Of course it doesn't cost publishers any more than that to distribute articles online either - the costs are in the review part (and whatever copyediting they do), not in distribution.
You'll hear about journals now that are essentially free - this is almost always for one of two reasons:
Given the $500/article cost, the other question is does science really need this level of peer review, or can it get by with less? Well, we've already seen a couple of instances of scientific fraud that slipped by in physics in the last few months even with the current level of review - is skimping really a good idea? And is the $500 minimal cost or even $1000-$2000 typical cost per article now all that bad, compared to the typical $50,000-$100,000 research grant that generally funded such research?
Yet another proposed solution has been to publish fewer papers in those journals that receive the full peer-review treatment. Unless authors miraculously constrain themselves somehow, the only way that would save us money would be to reject a lot of things without review (because the costs are in the review process itself) - but then you've thrown out the whole "peer" process you're using to determine what's published!
So, maybe Brown has found a way through this morass - but the scientific system has a complex, little studied dynamic in which peer review as it currently stands plays an important role... if we really can't afford it (the old way) any more, we're headed into some uncharted waters...
Energy: time to change the picture.
The article doesn't mention who would be the peer review board in the online journal system.
/.
I think Brown could learn a lot from the open discussion forum used by
Anyone could "publish" an article. People would receive alerts when an article was published in a topic area of their interest. Readers would be able to rate the article on several points, and would be able to add commentary, notes, etc.
Commentary, ratings, etc., could be sorted according to the evaluators' verified academic credentials (maybe I only care about what Harvard academics think of article X on particle physics, but someone else may be interested in what the general public, or for that matter 8th graders think of article X).
Any new system would have to preserve the aspect of the status quo that generally dictates that unless the big shots in your field think you are onto something, you don't get recognition.
Amazing magic tricks
I am one of the few graduate students who published in a journal as an undergrad, and I think you should know what goes into the publishing process (for IEEE publications, which covers almost all respected peer-reviewed computing journals).
Traditionally, you first have to have the article published in a conference, which requires for you to
1) Write the article
2) Go to the conference and present your work
Submission to a conference usually happens approximately six months before the actual conference. You get acceptance about a month after submission (or you get rejection). Most conferences have an acceptance rate of 50% or worse, meaning that they turn away HALF of the applicants.
The process of selection is done by assigning reviews to major professors in the field who are not submitting to the conference. These professors sometimes pass the review work along to some of their best grad students (this happened several times in a lab that I worked in).
After you are accepted, you send the final version, which includes any changes you may have made to the rough draft, and then go to the conference.
The next step is a journal article. This usually includes some additional fleshing out of the article. Most conference procedings are between 4 and 8 pages; journal articles can be as long as you can get it to be. You want it longer, because the longer it is, the more likely that the people reading it will understand and want to use your idea, because you can explain it upteen ways and provide numerous examples of why your [whatever it is] works well, which always leads to good things for a journal article writer's career.
That often takes a while as well. Once you submit the journal article, you get a preliminary acceptance contingent upon making changes after three months of review. Of course, once again acceptance is less than 50%, usually, but if you publish in a conference first, your chances are significantly higher than if you don't. You have an incredibly powerful idea to make it without a conference first.
The reviewers in this case are required to make a very careful inspection of the article to ensure that
1) the theories presented are useful
2) the theories make sense
3) the paper is written well enough to be readable
Reviewers are also required to find ALL spelling and grammer mistakes, and they have to understand the methods presented within the paper well enough to make a summary of the journal article. Also, reviewers are the same as before - experts in the field (college professors) who are not submitting to the journal at the time of the review.
These reviewers give you a report, accepting contingent upon meeting their requirements (or defending why you can't).
You then have to submit again and your article is once again edited for approximately 6 months.
If you REALLY rush, this entire process takes one year, however realistically, it usually takes two. (Yeah, I started the game as a junior in college).
Now, I don't care if we do this online. IEEE has a research engine called IEEE Xplore, which is often purchased by research institutions such as Universities. It has the whole database of IEEE publications within it.
But I don't know how to get a much better peer-review process than this; its pretty darn strict. So professors can't just whip of papers like nobody's business - they really have to put some work into it. If they have a lot of papers, it means they've done a lot of stuff that at least six other experts (for each paper - sometimes more) believe to have merit.
Mod me down and I will become more powerful than you can possibly imagine!
Just last week I received a box of photocopies of an article I published in one of Elsevier's journals in March. Apart from the fact that its nearly half a year late, what the hell should I do with 25 crappy photocopies of laser printer output of one of my own articles? We do have printers and copiers at our office.
.... Elsevier. Natuarally their own journals are on this list.
The whole process from beginning to end is so obsolete. I initiated contact with the journal editor more than a year ago by sending him a pdf of my article. He mailed back to thank me for my interest and asked me to send him three doublespaced paper copies to his office in the US (BTW reading doublespaced copies sucks IMHO). I did this, then I heard nothing for a long time. Finally I got a request to review a paper for the journal (this is quite common, most reviewers are also submitters). Finally after about half a year the paper was conditionally accepted (Yay!). This required an editing round and another submission of three paper copies. And several months later I was notified that my paper was accepted.
I submitted a final version (by paper and electronically). That was the last I heard from them (a letter/email would have been nice) until I received the box full of photocopies. By monitoring the site I found out which in which issue of the journal my article was to be published.
The editor of this journal is probably receiving a small fee for his efforts, which mostly consist of allocating reviewers to papers and putting stamps on envelopes. The actual technical editing is done by a bunch of latex monkeys provided by Elsevier. All communication is done by snail mail, communicating by email confuses both editors and elsevier staff (even though it would save loads of time).
The worst thing of all is that their journal is far too expensive for individuals to subscribe to. Hence the only subscriptions go to university libraries who mostly store packs of unread dead trees in their archives. In my country, a significant portion of government research funds is used for this purpose (i.e. money intended for fundamental research is flowing directly to the pockets of publishers) which I think is outrageous. I'm pretty sure the situation is the same elsewhere.
Now back to the role of the publisher. The publisher wastes everybodies time with a stupid editing process and by producing dead trees nobody reads anyway. It pays the editor a small fee and thats it. Apart from wasting everybodies time and funding the editor they do not actually contribute anything else. It is the editor who handles the peer review (100% volunteers as far as I know), it is the authors who deliver the content (100% volunteers). Taking the publisher out of the loop would save enormous amounts of money. Public funds could be used to fund editors and electronic hosting of journals for a fraction of the money currently flowing to publishers. This would not hurt the peer review process since it already depends on volunteers anyway.
I have no other choice than to either comply with this obsolete process or pursue another career. The productivity of my university is measured in terms of number of articles published. One of the parties involved in annually creating a list of acceptable journals and a nr. of publications per dutch university is
Jilles