Slashdot Mirror
Peer Review Highly Sensitive To Poor Refereeing
$RANDOMLUSER writes "A new study described at Physicsworld.com claims that a small percentage of shoddy or self-interested referees can have a drastic effect on published article quality. The research shows that article quality can drop as much as one standard deviation when just 10% of referees do not behave 'correctly.' At high levels of self-serving or random behavior, 'the peer-review system will not perform much better than by accepting papers by throwing (an unbiased) coin.' The model also includes calculations for 'friendship networks' (nepotism) between authors and reviewers. The original paper, by a pair of complex systems researchers, is available at arXiv.org. No word on when we can expect it to be peer reviewed."
"The system provides an opportunity for referees to try to avoid embarrassment for themselves, which is not the goal at all," he says.
So, if a reviewer sees a paper that has actual data and a conclusion that goes against the consensus of the scientific community, the reviewer may reject it for fear of appearing foolish? Or rejecting someone just because of their publicized personal beliefs?
Here's a hypothetical, a climate scientist who's an openly devout Christian finds data that sheds doubt on human caused global warming will be rejected because someone's afraid of looking foolish.
That's the way I'm interpreting this study.
RIP America
July 4, 1776 - September 11, 2001
As much crap as /.'s moderation system gets, it actually tends to be one of the better systems I've seen on the net. First of all, it's highly customizable so if I wanted to I can easily set it to add or remove value from certain types of moderations. I usually bump flamebait and troll up a few pegs just so I can see the posts that do occasionally get unfairly moderated. I can also add other posters I find interesting to a list and bump up their post value so if I'm interested in what they have to say I can always make sure I'll see it.
I also think that the community goes a long way towards making the system work well. Sure there will always be people who abuse the system and moderate posts with which they disagree as flamebait, etc. but the community as a whole does a good job of promoting interesting lines of conversation and for any given topic there are probably a few people in the community who specialize in that area and can provide some excellent commentary.
It's not perfect, but it's probably one of the best systems in actual practice that's currently being used.
Except that I've heard of people deliberately adding people to acknowledgements to try to make sure they don't get those people as referees (and it hasn't worked)!
I think it comes down to two things.
First, the relative rarity of mod points encourages people to take it seriously. It also encourages the most active, most interesting posters to give up posting once in a while to moderate. Most other sites use a mod system that allows so many votes per article but still don't allow you to post and mod the same article. That means that the most frequent posters will seldom mod and that there can be people who only ever mod articles without ever commenting on them.
Second, attaching a reason to the mods encourages people to actually think about why they are modding the way that they are. As many people say, there is no "-1 I Disagree" mod, in order to mod someone down you have to be saying that they are actively trying to derail the conversation. Of course, lots of modders will ignore that and mod however they want, but I think that it does make at least some people stop and think before they accuse someone else of being flamebait or a troll.
I agree with all that you said, but for heaven's sake, use full-forms at least once
For those as confused as I, the paper the parent refers to is "Gerhard Gerlich, Ralf D. Tscheuschner. Falsification of the atmospheric CO2 greenhouse effects within the frame of physics.".
Contains such howlers as "There's no such thing as average temperature"... RC Wiki page