Improvements on the Scientific Review Process?

myc asks: "A recent story has discussed the pros and cons of the publishing end of scientific literature, but what about the review end? I work in biology and recently have run into a slew of frustrations trying to get my work published. For instance, one really frustrating thing is, because of the need to keep the reviewers' identify confidential, communication between authors and reviewers is essentially one-way via the editorial office. When the reviewers encounter something that might be confusing to them in the manuscript, they take it as a negative, when really a very simple explanation from the authors would clear it up. After all, its the scientific content, not 1337 wr171ng skillz that is being reviewed. What are some of the frustrations you have come across when trying to publish your work, and have you any ideas on how to improve on the review process?"

"Another really frustrating point is many people feel that some papers are accepted on the basis of the reputation of the senior author and not purely on scientific merit (i.e., the burden of proof is lessened for established scientists), while the opposite is true for younger investigators (i.e., the burden of proof for novel findings may be higher, sometimes unreasonably so). One's scientific pedigree also helps, to a certain extent (if you trained with a big name, you are more likely to get published).

In the examples above, 2-way communication via anonymous email between the authors and reviewers might solve this particular problem; also, I think that if the identities of the reviewers are protected, why not the authors? Perhaps a solution to my second issue would be to have the author's names hidden from the reviewers until after the review process is complete."

Comments

Communicate effectively.

[Anonymous Coward]

"When the reviewers encounter something that might be confusing to them in the manuscript, they take it as a negative, when really a very simple explanation from the authors would clear it up. After all, its the scientific content, not 1337 wr171ng skillz that is being reviewed. "

Well actually two things are important. The main material, and the other is your ability to communicate to others what's important. What good is the greatest idea ever discovered if people can't understand you?

The other is true, and a bit regretable, if not at least understandable.

Clarity of exposition

[GuyMannDude]

When the reviewers encounter something that might be confusing to them in the manuscript, they take it as a negative, when really a very simple explanation from the authors would clear it up. After all, its the scientific content, not 1337 wr171ng skillz that is being reviewed.

I'm not so sure you can separate the two so easily. First, consider the fact that if the reviewer cannot understand what you are saying by reading your paper, many other people who will read your paper once it's been published won't understand it either. I can just see it: the reviewer asks the author a question using this email system you describe. The author explains their intent and the reviewer understands and accepts the paper. The confusing wording in the paper is never cleared up. Expecting all those people who have to read the thing once it's published to make heads or tails out of something worded poorly is just not fair. And don't give me the "once the author sees the reviewer's confusion, s/he will take it upon themselves to fix up the wording" bit. That would be really nice but don't count on it.

Second, you seem to make the distinction between having great scientific thoughts and the ability to communicate them effectively. Brilliant physists like Richard Feynman were able to explain very complicated concepts to a wide audience. If you really, truly understand something you should be able to make it understandable to others. If you cannot describe something without falling back on a lot of jargon, then perhaps you don't really have that deep an understanding after all.

Third, writing really isn't that difficult. Let's face it: a lot of science-types look down on those who majored in literature and humanities stuff in college. The feeling among scientists is that all the writing stuff is easy compared to the hardcore technical stuff that they do. If that's so, then prove it. It really shouldn't be too difficult for them to produce something reasonably easy to read. No one is going to kill you if you don't get all the rules for comma usage correct. But you should really have the ability to communicate your thoughts clearly to others in your field.

GMD

Re:Clarity of exposition

[Lochin Rabbar]

Good post, I agree with everything you say apart from one nitpick.

writing really isn't that difficult. Let's face it: a lot of science-types look down on those who majored in literature and humanities stuff in college. The feeling among scientists is that all the writing stuff is easy compared to the hardcore technical stuff that they do.

Speaking as someone who can write technical papers very well, but who can't write prose, other than poetry for love nor money I have to take issue with the typical 'scientist'. The two skill sets are very different. Good prose seems to depend on an ability to lay aside one's inhibitions and let yourself be exposed to ridicule, good technical writing sets emotion aside and concentrates on the argument.

Now to add something to the debate: I have a rule for technical writing, and it is that before stating a conclusion the reader should have already reached that conclusion. If I can't achieve this then either my writing skills are lacking, or I have discovered a flaw in my argument.

Re:Clarity of exposition

[The Fun Guy]

Absolutely. IAAS, and I review manuscripts for a number of scientific journals, mostly related to microbiology and microbial ecology. Only in a very few instances have I had to give a review of "Reject" based on poor language, and that was because the writing was so bad I couldn't make heads or tails of what the authors were actually trying to say that their data meant. Papers which were obviously written by authors whose primary language facility is not English need to be proofed/polished by a native- or fluent-speaker (assuming that the lanugage of the journal is English). I've given a review of "Revise extensively" on that basis, and have made that specific recommendation. If the paper conveys no useful information, wether because it's poorly written or uses bad experimental design or has overreaching conclusions, it doesn't belong in a reputable journal until those problems are corrected. The readership of the journal relies on the reviewers to make sure the papers are clear and of a minimum level of quality.

Most papers are pretty well written, bearing in mind that we're not talking about gripping prose, here... it's technical writing of the style of "this is the objective, this is what I did, this is how I did it, this is what happened, this is what it means". Basic technical writing is an essential skill that the scientist has to have. If you just simply cannot string the words together properly, find a collaborator to clean up your text, and give acknowledgements as appropriate. Usually, I would give a "Reject" or "Revise extensively and re-submit" if there's a problem with the experimental design or statitistics used. Overreaching or unwarranted conclusions would also be a red flag.

By the way, I have no patience with lazy or incompetent authors who submit crappy work and assume that the editor will polish it all up. Dot your own "i"s, cross your own "t"s, and you will fare much better in your reviews.

3-way email is nearly 2 way

[ghostlibrary]

In astronomy, the review process is straightforward-- I've had reviewers email me directly, or have emails forwarded by editors. An email being forwarded keeps anonymity and, if the editor is professional, adds little drag to the process. Perhaps a 1-day delay on what is typically a 2-week process of review.

That said, since eBay, half.com, etc all have "send question to seller" that maintains anonymity but allows direct communication, I'd think a peer-review journal site could easily manage that.

Already, things like ApJ prefer/almost require electronic submission, and have a page for authors to check on their progress. It'd be easy to implement.

Finally, on anonymity-- the best reviewer comments I've received were from reviewers who declined to be anonymous. They were comfortable enough to communicate directly, and the work was better for it. So perhaps the very requirement of anonymity could be reconsidered.

So, in short:

1) Use electronic marketplace ideas to streamline and improve the review process,
2) All anonymity to (like on slashdot) be optional.

Ummm

[DorianGre]

How about learning to write better. You are looking for a complex solution to a simple problem.

Well...

[ByronEllis]

Have you ever considered that the article will be published in a journal and read by people without the benefit of two-way communication with the author(s)? If, for instance, reviewer confusion can be fixed with a simple explanation then why isn't that explanation in the article? Reviewer comments are just that: comments that will help you improve the article for resubmission, not some sort of scientific hazing ritual and anonymous communication may only serve to degrade the quality of publication.

If it really bothers you that much, have colleagues not involved in the project review papers before submission and do the same for them.