Peer-Reviewed Research Over The Web

bhoman writes "The San Francisco Chronicle (sfgate.com) has an article today about Stanford biochemist Patrick O. Brown, who helped develop low-cost DNA microarrays for gene research. He is seeking $20M to start a foundation that would fund peer-review of research papers and then make them available for free over the web, thereby avoiding the high-cost of subscriptions common in existing research publications. Predictably, some publishers seem to be warning that their publishing model is hard to improve upon. The article mentions that a previous effort by Brown and others, The Public Library of Science garnered the signatures of 30,000 supporters, but then implies that it basically failed, suggesting that academics need the journals more than vice versa. Sounds like Brown's idea is exactly what the web is made for."

Comments

Won't Work

[Egoine]

Good idea, but the article says it all:

"It's publish or perish," Stern said. "As long as we have promotion and tenure tied to publishing, change won't work."

sadly.

That would have been great.

Changing the publishing system

[CTib]

Yes!! I heard many years ago about efforts to replace the completely broken journal/subscription of today with a peer-review/web-of-trust. Problem is, there's a market of sci. journals for a reason! Scientific excellence became a screwed notion immediately after the WWII years, when the Iron Courtain broke in two (reasonably equal halves) the older web of trust. Scientific excellence has now to be quantitatively defined, by number of articles published, especially. This is very wrong, of course.

Thus, we first need to change our perception of scientific excellence and _then_ put in place a peer-review mechanism. And the new perceptions needs a peer-review mechanism in order to be reformed properly. Hen or egg? :-/

Pedigree

[BWJones]

Speaking as someone who just submitted an article to Science for review, I would have to say the primary hurdle that Brown will have to overcome is pedigree. Journals like Science and Nature have a history and an editorial board that ensure a rigorous review process that ultimately presents the best of the best and the most significant science to the scientific community. Publication of ones work in journals of this caliber are important to your career, and given their wide distribution can be critical in obtaining funding.

The implications of this are far more than simple "peer mating" and "copy editing" as one other poster suggested. Granted, there is nothing that can keep an online journal from eventually becoming the place to publish, but it will take time and a commitment to excellence that will have to be maintained for scientists to become comfortable in submitting their hard earned results to. Publication of observational science will not cut it. The implication of this is that since most scientists view Science and Nature (among a select few) as the pre-eminent journals, they will be concerned about submitting the most significant scientific results to a new online journal. Typically from what I have seen, when one gets rejected from the more prestigious journals, you start moving down your ladder of preference until somebody accepts your article. Of course results targeted for specific journals with a readership that would be interested in your results always matters and this is where online journals stand the best chance of making it as opposed to large pre-eminent general interest scientific journals such as Science and Nature.

Modern science is in a sorry state, IMHO

[kcbrown]

Bear with me on this, please, and please correct any errors I might make here.

The whole point behind science, its entire reason to exist, is to provide us with a predictive explanation of the world around us. It needs the "many eyes" approach more than just about any other human endeavor, because the entire point is to model the real world and you can't do that without a lot of observation.

Of course, science has also proven to be useful, and that's been something of an anathema to it. The reason is that things which are useful are things which people (corporations in particular) want to capitalize on in an exclusive way. It seems to me that there was a time when everyone recognized the truth that public disclosure and widespread collaboration is necessary for science to advance.

That no longer seems to be the case from where I sit. Today, corporations fund a great deal of research at the university level, and there is a great deal of pressure from both corporations and from the universities themselves to keep ongoing research under wraps as much as possible, in order to maximize the chances not just of publishing but also of getting patents on the results (which are probably then transferred to the corporations that funded the research).

Those people in the scientific community that I've spoken to believe, to a man, in collaboration with their peers in order to further science. They're held up by the people that fund their research.

How does this relate to publishing on the web? Well, publishing on the web removes a lot of the exclusivity that currently exists, so there will naturally be opposition to it from those who benefit from the control they have over scientific publishing right now. And my cynical mind tells me that there's a good chance that those who fund research exert an additional level of control through the current publishers (it would make sense, right?). It's my hope that research over the web will help in reducing the amount of exclusivity that seems to exist currently in the scientific community. But then, that's probably wishful thinking.

As long as that level of exclusivity exists, our understanding of the universe won't advance as quickly as it might otherwise. Perhaps things have always been this way and I'm just pining for better days that have never existed. But if there's even a chance that publishing on the web will improve the amount of collaboration and peer review, I think it's worth doing.

But this proposal doesn't do much to help with that, because it still concentrates the power of peer review and publishing into the hands of a few. What prevents researchers from collaborating with each other, getting peer review from each other, and publishing on the web directly, instead of going through middlemen like they do now? Seems to me that they're being held up by those that fund the research. And unfortunately, this proposal wouldn't change that.

Yes, it's a step in the right direction, and the current scientific publishers need some competition. But it shouldn't be seen as the end goal.

Our experience

[apsmith]

I work for the Physical Review journals at the American Physical Society, and I've been somewhat involved in these debates from the physics publisher side of things for the last 8 or so years - for example in the American Scientist Forum discussion that's been going on since 1998...

Anyway, I wish Brown all the best success, but as others have mentioned, it's a somewhat harder problem than it first seems (at least he's asking for $20 million, which is somewhat realistic for handling real peer review for a substantial number of articles - 10's of thousands at least).

What's behind this nebulous "peer review" concept, at least for us, is a complex and historically based system of checks and balances involving communications between authors, editors, and (anonymous and non-anonymous) reviewers; we're essentially a legal/court system for scientific articles. There's a lot of information-related issues in there, and information technology helps a lot (that's the part I'm involved in). But fundamentally, at least the way we do it, there needs to be a paid, responsible human being reading most communications and monitoring the process, and as far as we've been able to work out, you can't get the cost under about $500 or so per article.

Now, just distributing the papers can be done essentially for free (to as many people as would want to read for about $1-5 per article, for hardware, software, disk, network, etc.) which is what the famous physics e-print archive does so well. Of course it doesn't cost publishers any more than that to distribute articles online either - the costs are in the review part (and whatever copyediting they do), not in distribution.

You'll hear about journals now that are essentially free - this is almost always for one of two reasons:

1. The journal is very small, and some institution is picking up all the salary and incidental costs - $500/article works out to just $50,000/year for a 100 article/year journal.
   
2. The journal is heavily skimping on the "peer review" side of things - publishing conference proceedings papers for example with no review beyond the acceptance of the paper at the conference. Nothing wrong with that, but it's not what we normally mean by peer review.
   

Given the $500/article cost, the other question is does science really need this level of peer review, or can it get by with less? Well, we've already seen a couple of instances of scientific fraud that slipped by in physics in the last few months even with the current level of review - is skimping really a good idea? And is the $500 minimal cost or even $1000-$2000 typical cost per article now all that bad, compared to the typical $50,000-$100,000 research grant that generally funded such research?

Yet another proposed solution has been to publish fewer papers in those journals that receive the full peer-review treatment. Unless authors miraculously constrain themselves somehow, the only way that would save us money would be to reject a lot of things without review (because the costs are in the review process itself) - but then you've thrown out the whole "peer" process you're using to determine what's published!

So, maybe Brown has found a way through this morass - but the scientific system has a complex, little studied dynamic in which peer review as it currently stands plays an important role... if we really can't afford it (the old way) any more, we're headed into some uncharted waters...

Re:Our experience

[apsmith]

Well :-)

Are we a "near monopoly"? There are hundreds of physics journals out there, in competition for authors and readers. It's a pretty free market, in my opinion, though you'll hear all sorts of arguments on that front. I assume we receive so many papers because we do our job well. Even though we publish a lot, I believe the total number of physics articles published worldwide (and we receive about 70% of our submissions from outside the US) is about five times our volume. And there's always Science or Nature!

Now the economics are a little odd because we sell subscriptions primarily to libraries, which are sort of a captive market. However, you'd be surprised at the number of small colleges and public libraries and such that subscribe - if they weren't happy, what reason would they have to continue?

The key point is your #4:

This simply begs the question of whether there is anything inherently good about "what we normally mean by peer review." Your colleagues are going to form opinions of the quality and reliability of your work anyway. A lot of bad and/or unimportant stuff gets published in journals, so peer reviewing simply doesn't fulfill the whole function of quality control. Since we need further quality-control beyond traditional peer-reviewing, why do we need traditional peer-reviewing?

I don't know that we do. Perhaps we don't. It's an experiment worth trying. But is it an experiment that's sufficiently important to have the government force it on us (as some have suggested)?

Here's a triumph of illogic: traditional peer-reviewing is works badly; therefore we must continue to use it.

Ummm. So you're in favor of more rigorous review? Wouldn't that be more expensive? It seems to me the current system works reasonably well, is improving in speed and efficiency, and at only a couple of percent of research dollars, is reasonably affordable.

The main complaint of the public library of science people etc. seems to have been that the research isn't available free online. Well, in our happy medium in physics we have no problem with researchers posting their research on their own web sites, or on the e-print server, etc. Go ahead and do it! That makes it available free - but don't expect the journal publishers to make everything free for you; we're doing a different job here.

publish or perish

[rnd()]

The article doesn't mention who would be the peer review board in the online journal system.

I think Brown could learn a lot from the open discussion forum used by /.

Anyone could "publish" an article. People would receive alerts when an article was published in a topic area of their interest. Readers would be able to rate the article on several points, and would be able to add commentary, notes, etc.

Commentary, ratings, etc., could be sorted according to the evaluators' verified academic credentials (maybe I only care about what Harvard academics think of article X on particle physics, but someone else may be interested in what the general public, or for that matter 8th graders think of article X).

Any new system would have to preserve the aspect of the status quo that generally dictates that unless the big shots in your field think you are onto something, you don't get recognition.