Royal Society Wants to Keep Science off Web

truckaxle writes "Britain's national academy of science, The Royal Society, which publishes one of the world's oldest journals, Philosophical Transactions, has joined the debate of the publishing of scientific publications on the internet. In a article by the Guardian a spokesman for the Royal Society was quoted as saying: 'We think it conceivable that the journals in some disciplines might suffer. Why would you pay to subscribe to a journal if the papers appear free of charge?' They believe that internet publishing would harm the exchange of knowledge between researchers."

Comments

My previous post on this subject

[Catamaran]

From my previous post

I believe that the scientific Journal has outlived its usefulness, and will be replaced by ... Slashdot!

But seriously, reviewers are biased and sloppy, as are the editors. The fact that reviews are blind means that they are also unaccountable, which fosters even more bias.

Journals take months or years to respond to a submision, and often as not they respond with a rejection so the submitter has to give up or start the whole process over with another journal. There are so many scandals that one could quote. The whole process seems more designed to support the status quo than to promote knowledge.

I have discussed this with many people in academia and they react not with logic, but with horror that I would dare to question a system that they view almost mystical reverence.

Re:My previous post on this subject

[kebes]

(Note: I'm a publishing academic, so you may consider me to be biased to support the current system or to hate it.)

Some reviewers are good, some are bad. Peer review is not perfect, but when I compare it to how things get done in alot of other areas, I'm amazed at how good it is. The journals take the anonymity very seriously, which is a good thing. Yes, anonymous reviews may enable bias, but they also enable honesty. A good editor can differentiate between insightful reviews and biases, and make the right call (yes, editors can have biases also). Peer review has many good features.

As to how long it takes for the review process... well it's getting much faster than it used to be. With online submission, emailing of PDFs, and so on, a review can take as little as a month (compared to snail mail days, where a year was more typical). Many journals will release articles online as soon as they are approved, months in advance of the paper copies. High profile journals keep amazingly tight schedules. From submission to appearing online can be only a few weeks. That's pretty fast. Not all journals are that good, mind you.

Can the system be improved? Absolutely. Will the web play a crucial role? I think so. Having the peer reviews be online, and allowing the authors of the paper to respond to comments (in an anonymous and regulated slashdot-like way, perhaps)... or even allowing the various reviewers to exchange comments with each other (again, anonymously) would make the current system just that much faster and more robust. Also, there is no reason why the reviewers comments (and author's rebuttals) could not be added to the online version of the paper (under "supplementary material" or whatever).

What we need to do is come up with better systems without ignoring the good aspects of current peer review.

Re:My previous post on this subject

[s20451]

Slashdot moderation bears very little resemblance to peer review.

Imagine that every slashdot comment got at least three moderations, and that each moderation involved not only a score but a written justification for the score, along with editing comments and questions for clarification.

Also imagine that you were expected to be a good moderator in exchange for the privilege of posting comments. And imagine that people's careers (including yours) hinged on timely and thoughtful reviews.

In reality, Slashdot moderation is much more like a popularity contest than a review. And items that have already been modded up are most likely to get further moderations, which is inherently unfair: the loudest voices are the ones that are most likely to get louder.

Re:My previous post on this subject

[flyingsquid]

A collaborative reviewing system like Slashdot's would probably work better than the status quo in the academic world. Especially 'metamoderation'.

I'm not sure I'd go so far as to hold up Slashdot as some kind of a model, but some aspects of the system are definitely worth looking at. The idea of reviewing the reviewers is a good one.

I've repeatedly had to deal with hostile reviewers who, when they didn't have any evidence or logic to back up their claims, resort to rhetoric, sarcasm, illogical arguments, inaccurate facts, and the nitpicking of typos. I've also had some good reviewers who have pointed out legitimate flaws in my work and made useful suggestions on how to improve it, and really helped me improve my papers. There ought to be some way to discourage the first and reward the second, but the system of anonymous reviewers means you're pretty much unaccountable for what you say. How, is the question.

The system can work wonders on a paper, I'll admit. But it's also given too much importance. The Origin of Species is one of the most important and influential books in human history, and it remains the single most important book in evolutionary biology. Yet it wasn't peer reviewed, and I seriously wonder how well Darwin's theory would have fared if he had been subjected to peer review.

Re:My previous post on this subject

[elakazal]

Peer review is a little like evolution: it's sloppy, it's brutal, it makes its share of mistakes, but in the end it works. There are loads of horror stories out there, but most of the time things shake out. And in most fields, even if your paper gets rejected one place, unless the whole field is against you, it can generally get published somewhere else, assuming there's some merit to it. Unfair reviews are balanced by other reviewers, and if you feel like you've been truly screwed, the final decision always rests with the editor. Some one in my lab is fighting that fight right now.

It is important to remember the reviewers are only anonymous to the authors (though, as you say, they aren't that anonymous...looking at the review of my last three papers I can guess with about 90% certain who all but one of them are). The editors know who they are, and giving a sloppy, unfair, or inappropriate review still hurts a reviewer's reputation with editor, who many times is a fairly well-respected member of the field. I'm often tempted to give marginal papers in review a pass, just because I know how much it sucks to have a paper rejected, I know the amount of work that went into it, and I'm a fairly nice guy (I think). But I also know that the editor is judging me on how I respond to the paper, and I don't want him/her to take my willingness to let subpar work slide as a reflection of my understanding of what qualifies as quality science, and so I do my best to give my honest opinion on the quality of the work.

Re:My previous post on this subject

[flyingsquid]

If a reviewer used rhetoric, sarcasm, illogical arguements, innacurate facts, or rejected your manuscript on the basis of typos, you should have complained to the editor. Any editor worth a damn would see the problem, send your paper for another reviewer for review, and stop sending any more submissions to the offending reviewer. That's how the system is ment to work.

I put up with this kind of stuff for a few rounds of review, real turn-the-cheek kind of a thing. Finally, I took a couple of weeks, sat down, revised my manuscript, and carefully dissected every single point the reviewer made, citing evidence, theory, and papers. I conceded a few things, and made a couple changes, but mostly implied that the reviewer was bullshitting and didn't know what the hell he was talking about- because that's what was going on. It was a risky move: I'm an unknown from an unknown university, he's a tenured Ivy League prof with a Harvard PhD, so all else being equal, who's the editor gonna side with? But I was tired of spending all this time battling bullshit, so I did the intellectual equivalent of dragging this guy out behind the pub and working him over with a two-by-four.

The paper was accepted for publication, without further review.

So yes, it did work... eventually. But I went through five journals and a total of seven submissions before getting accepted. The whole process gave me a new appreciation for Kuhn's _Structure of Scientific Revolutions_. And I really took heart by looking at examples of persistence rewarded, like Lynn Margulis. Her Serial Endosymbiosis Theory (the idea that chloroplasts and bacteria were once free-living organisms) was rejected a dozen times(!), before finally ending up in Journal of Theoretical Biology. Now it's in every biology textbook and nobody would even think of questioning it. So after being rejected by the fourth journal, I could tell myself, "well, I'm still only a third of the way towards Margulis' score!"

But that's also the classic refuge of the crank: point out the examples of unappreciated genius. "They reject my idea... but they also rejected continental drift! Everyone says I'm crazy and there's no evidence for the Chupucabra, but people thought the first stuffed platypus specimen was a fake and wouldn't believe the evidence!" Sure, it's possible that you're right and everyone else is wrong, like with Margulis. It's also possible you're a freakin' loon. Without too much knowledge of the specific subject of your paper, how is the editor supposed to tell the difference between science which provokes hostility because it's dead wrong/plain bad, and science which is right, but provokes hostility because people are narrow minded and dogmatic? For that matter, if you're confident in your work, and the reviewers hate it, somebody's perception of reality is tweaked: how do you make sure you're not the one with the warped perception? Back when I was still trying to get this paper accepted, I liked to joke "They laughed! They all laughed!" in a classic Evil Scientist voice... it helped me keep sane, but it also made me a bit uncomfortable because I was giving voice to the doubts: "am I really crazy for thinking this?"

Seriously though... what's the easiest way to tell when you're an undeservedly unappreciated Archimedes, and when you're a deservedly unappreciated Archimedes Plutonium?

Exchange of ideas?

[0racle]

Exchange of ideas or exchange of currency? I'm not really sure which one they don't want hurt.

Yup!

[Nom du Keyboard]

They believe that internet publishing would harm the exchange of knowledge between researchers.

Yup, knowledge is only true and valuable when you pay lots of money for it and distribute it to a limited group. Everyone knows that. After all, that's how it's always been. Can't change that now, can we? Heaven forbid the chaos that would ensue if there were a peer reviewed, moderated system like Slashdot to replace our sacred institutions.

(Oh, and yes, some publishers making a good living might lose their monopoly gravy train in the process.)

NEWSFLASH

[tomstdenis]

You're obsolete. :-)

It's really quite simple. Adapt or die [well the other alternative is to use your undue influence to make your approach last longer than it naturally would otherwise ... (glances at Microsoft)].

How any academic could think that the wide spread distribution of information could HURT academia is beyond me. Me thinks they have other issues on the mind [namely $$$ and power]. Given I've never read anything from their journal [nor consider myself an academic] I can't say I'd miss them if they disappeared. I get enough free shit [decent quality] from citeseer and eprint.iacr.org

The dude has one point though. Random acceptance [or blind] of papers can lead to some low quality material. Once in a while on eprint there are some really lack lustre crypto papers but quite a few are well written and interesting. And they are the sort of things that close minded expensive conference tours (...looking at the IACR conferences...) routinely rejected.

That said though, I've seen some REALLY POOR peer reviewed talks at conferences. Like the Indian students who presented on highly hardware optimized multivariate boolean equations at a SOFTWARE conference. Their talk was so horibly presented as to make me wish I had literally died at the time. Then there were the talks on one time pads at Crypto'03, etc, etc, etc.

Point is, quality material is subjective. The more open your publication is to peer review the more likely you will see quality material. The more close minded and aloof your publication is the less likely you will have insightful or interesting material to publish.

Tom

What's to prevent it?

[John Jorsett]

These guys sound like they think there's a way to stop it. Short of their fellow scientists organizing a formal shunning of research data that's web-published, what could actually prevent a researcher from putting his/her results on the web? Particularly if they get turned down by the journals? If I had devoted a lot of time and effort to some research and couldn't get a journal to publish it, you can bet that I'd publish it in web form rather than just let it rot.

on the other hand

[kebes]

so they ask:

Why would you pay to subscribe to a journal if the papers appear free of charge?

As a publishing scientist, I would venture another question: "Why should I publish in a journal that is not freely available? Why would I pick a journal that limits its readership?"

Science is produced (by and large) by scientists using public funds. It makes no sense that the results of this research become locked away and unavailable to the public. Scientific results should be available to the public, free of charge. The fact that this also helps foster international collaborations, makes science overall more effective, and levels the playing field between rich and poor nations is also a good thing.

Alternate funding models for the journals and publishers are being pursued. For instance, when a scientist publishes a paper, he could pay a fee to cover administrative costs. Then the article appears online, free to all. Some journals have already implemented such systems. It seems to work fine. At the end of the day, it's always the same people paying (universities and scientists pay for it, using public funds).

So to answer the question "Why would you pay to subscribe to a journal if the papers appear free of charge?" Just like now, the public will pay for the journals to operate. However, the public should be allowed access to that which they are funding.

Colateral damage (please RTFA)

[robindmorris]

Remember that the Royal Society is a non-profit organization, and does more than just publish journals. They also fund research, organize meetings, and do public outreach. What the Royal Society said is that they use the revenue from their journals to subsidise these other activities, and if the revenue from journals went away, they would most likely have to cut back on public outreach etc.

They're not saying they're against free information flow. They are saying that for them it will have a significant impact on the scope of their activities.

Well, there is some truth to what you say

[sam_handelman]

But there is a reason for reverence for peer review - as a procedure, it weeds out a lot of bullshit. There are many scandals - but far more successes (the entirety of biology, from the sometime in the early 20th century to the present.) I'm a biologist, so I cannot speak with confidence on the impact in other disciplines, or where the corresponding institutions of peer review may lie on the continuum between old boys network and tireless defenders of the scientific method, for other journals in other disciplines. In Biology, in spite of some failings, the record is overall very good.

  The comments by the royal society are nakedly self serving. The fear at the royal society is that organizations like the Public Library of Science will sideline them. This will only happen if organizations like PLoS can maintain the same quality of peer review as the Royal Society (I will assert - so far they are doing better) without charging money. The implicit claim, that free journals deliver a lower quality of review, does not stand up to even cursory examination. I will say (and this is a subjective assertion on my part) that PLoS actually provides a better grade of peer review, and that a system where professional editors preside over large budgets and a permanent base of prestige breeds the sort of cronyism and corruption that the parent post is (legitimately) concerned about.

  From a moral standpoint, of COURSE research done at public expense should be freely available to everyone, now that the technology exists to easily do so. In sum: if the royal society doesn't want to adapt to a modern era where real publication costs approach zero, let them be sidelined.

The process

[jtangen]

I'm not sure what journals you're submitting to, but the turnover rate for most journals in science are only a few months, and some just a few weeks. As an academic with a wife who works as an editorial co-ordinator for three journals, I think I have a bit of insight into the process, and you've greatly misrepresented the process.

Indeed, the process is flawed, but it's what we have at the moment. Blind reviews are lame, and blind authorship is even worse (where the reviewers have no idea who wrote the paper - but can quickly guess given their reference list). It's the editor's job, however, to ensure that the quality of the reviews are adequate. The peer review process certainly isn't without flaws, but I have yet to see a better process. If you have a better suggestion, please speak up.

On the topic of the availability of scientific publications on the web, this really isn't new. Many researchers already post their papers as pdf on the web, and Scholar Google provides instant access to them. I suspect he trouble seems to be with greedy publishers. Academics are expected to hand over their rights to the publishers to distribute their own work. Many don't look favourably on posting papers for download and are trying to stop it. This is a bit odd. They have the rights to the version of the paper *as it looks in the journal*. So if you take out a comma and repost it, you're fine. Or if you're a LaTeX user, you can create nicer looking documents than the publishers do! There's also the issue of reprints. Once upon a time, if someone requested a copy of the paper, you could send it to them. The publishers even provide a number of hard copies to do so. So many researchers have added a prompt to the user before downloading the document indicating that by clicking the download link to the article, they are requesting a reprint.

Ummm...

[Liam Slider]

Wasn't the web invented in the first place by scientists so they could more easily share information?