Slashdot Mirror


What Happens When Nobody Proofreads an Academic Paper

An anonymous reader writes: Drafts are drafts for a reason. Not only do they tend to contain unpolished writing and unfinished thoughts, they're often filled with little notes we leave ourselves to fill in later. Slate reports on a paper recently published in the journal Ethology that contained an unfortunate self-note that made it into the final, published article, despite layers upon layers of editing, peer review, and proofreading. In the middle of a sentence about shoaling preferences, the note asks, "should we cite the crappy Gabor paper here?" When notified of the mistake, the publisher quickly took it down and said they would "investigate" how the line wasn't caught. One of the authors said it wasn't intentional and apologized for the impolite error.

18 of 170 comments (clear)

  1. Crappy Gabor paper citation citation here? by Anonymous Coward · · Score: 3, Informative

    If anyone cares to read the passage with the insert here's a twitter pic of it in use.

  2. Re:MS Office Incompatibility by DaCo · · Score: 3, Insightful

    Let's face it, even open formats/OSS can't make people *less* stupid.

    --
    DELETE MY ACCOUNT
  3. Re:Big woop by ShanghaiBill · · Score: 3, Insightful

    I think the point is that standards and professionalism are slipping, even in science.

    No, the point is that standards and professionalism are low. To show that they are "slipping" would require showing that they were higher at some point in the past. Crappy, poorly edited papers are nothing new.

  4. Have seen this several times as reviwer... by gweihir · · Score: 5, Interesting

    There are two types of reviewers: The valuable ones that actually read a paper and try to understand it, and the worthless ones that look at title, abstract and who wrote it (usually easy to find out even in anonymous review). The first type catches these things, the second does not and quite often lest bad papers in and keeps good papers out. The second type is much more common.

    Or to put it short: Peer review is broken, as there is no quality control in most cases.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:Have seen this several times as reviwer... by timeOday · · Score: 3, Interesting
      "Peer review is broken" is such a broad statement, it's like claiming "clothes today aren't well-made." Peer-review is as good or bad as the individual journal.

      Granted, the average quality of "journals" has probably plummeted in recent decades as there are far more PhDs, papers, and journals than in the past. But by the same token, the quality of the top 100 journals (or any fixed number) has probably increased. I say that because the ease of communications now helps, and because of all the progress and recent focus on repeatability and avoiding statistical pitfalls. (A lot of reporting on this implies it is somehow a new problem, but there is no reason to think that).

    2. Re:Have seen this several times as reviwer... by Anonymous Coward · · Score: 3, Informative

      There are two types of reviewers: The valuable ones that actually read a paper and try to understand it,...

      I actually had a look at the paper in question.

      I've probably got some details wrong but it was mainly an experimental study where they look at two closely related populations of fish - one in a toxic sulfur hot string and the other not. They find that females in the sulfur hot springs have a preference for (male) fish with spots while the other females don't have a preference.

      But then they launch into a pages of random speculation about what this observation might, or might not, mean (i.e. they had no idea). Maybe someone in the field would find it deeply insightful but it looked to me like they were just padding out the paper. Anyway, the bit about the "crappy Gabor reference" was in this section.

      So what do you do if you're reviewing the paper? Do you try to take the random speculation seriously and spend days trying to make some sense of it and give it a meaningful review? Or do you just figure that the real value of the paper was the experimental observation and ignore the rest?

      Given that there's a finite number of hours in the day and lots of real science that needs doing, it's actually a tough call.

    3. Re:Have seen this several times as reviwer... by AthanasiusKircher · · Score: 5, Insightful

      Peer-review is as good or bad as the individual journal.

      While this is probably true, I would go further and say that this particular issue (from TFS) has relatively little to do with peer-review.

      Most peer reviewers are not paid. When I've written reviews for articles, I'm assuming that I'm volunteering my time as an expert on the subject matter. So my primary purpose is to critique the argument, look at the design, see whether the conclusions are justified, etc.

      Things like fixing commas, rewording sentences, and proofreading for some sort of stupid error where the authors forgot to delete something -- that's not my primary purpose. If I have time and I see pervasive problems of style, I might say something in the review. If those stylistic things end up confusing the argument or making the thing hard to read, I might say something.

      But if I were reading this article, and there were a half-dozen comments or questions I had about methodology or argument on this page, would I bother saying, "Oh yeah, and don't forget to fix the stupid missed citation!" Maybe. But it wouldn't be my highest priority.

      I don't know what happens at this journal, but most high-quality journals have at least some copyediting done before publication. If the author didn't catch this error during revision, it should have been caught by the copyeditor. But the peer reviewer? Are we going to ask for expert volunteers in some academic discipline to fix commas next?

      Granted, the average quality of "journals" has probably plummeted in recent decades as there are far more PhDs, papers, and journals than in the past. But by the same token, the quality of the top 100 journals (or any fixed number) has probably increased.

      It depends on what you mean by "quality." If, by "quality," you mean the level and rigor of articles and research in major journals, maybe you have a point.

      But, if by "quality" of a publication, you mean the copyediting -- that has absolutely DECREASED in recent years. I can't tell you how many sets of proofs I've seen with all sorts of idiotic formatting errors, places where an editor tried to fix prose or move something in the layout and caused an absolute disaster to happen, etc. Heck, this isn't just articles -- I've seen recent books from major university presses that seem to have the same level of copyediting a cheap romance novel would have received 40 years ago. And heaven forbid that you have some complex set of figures or images that need to be laid out in a specific way -- the designers seem to go out of the way to screw things up by resizing or moving things about, even if you send them images designed to fit the page layout precisely.

      I haven't read the article referenced in TFA. But this all sounds like a proofreading and a copyediting problem. Peer reviewers? Yeah, I suppose they should have caught it if that citation would actually make a difference in the argument. Otherwise, I'm not sure what this has to do with peer review quality AT ALL.

    4. Re:Have seen this several times as reviwer... by phantomfive · · Score: 3, Informative

      So what do you do if you're reviewing the paper? Do you try to take the random speculation seriously and spend days trying to make some sense of it and give it a meaningful review?

      If you're going to do peer review, then yes, you should try to make sense of it. How is that even a question?
      Secondly, if the paper has serious problems with most of the content, of course you should reject it and explain why. Then the author can fix it and resubmit. It's not like rejection is some kind of permanent, damaging thing.

      --
      "First they came for the slanderers and i said nothing."
  5. What Happens When /. Headlines... by NotSanguine · · Score: 3, Insightful

    and the articles they reference are wildly inaccurate. From TFA:

    Not sure how this made it through proofreading, peer review, and copyediting. Via http://t.co/sWaswaM2X4 #addedvalue pic.twitter.com/8krLlvthAr — Dave Harris (@davidjayharris) November 10, 2014

    [Emphasis Added]

    So the paper was proofread, peer-reviewed and copyedited. Sigh.

    People make mistakes. Life is like that sometimes. The authors of the paper will face consequences for this. Hopefully, they'll learn from them.

    Nothing to see here, unless you wrote the paper or are the person referenced.. The post and the linked TFA are a waste of time.

    --
    No, no, you're not thinking; you're just being logical. --Niels Bohr
    1. Re:What Happens When /. Headlines... by Anonymous Coward · · Score: 4, Informative

      A more useful article than TFA is over at retractionwatch.

  6. How could this happen? by pesho · · Score: 4, Funny

    Obviously Gabor did not review the manuscript.

  7. Re:Big woop by Gordo_1 · · Score: 3, Insightful

    On a slight tangent, I've been wondering about this "things are getting worse" meme as it relates to just about anything related to humanity that can be tracked over time. You read so much today about worldwide atrocities, NSA snooping, domestic crime, political skullduggery, and one starts to develop the impression that things truly are getting worse. I think it would be interesting to see if that's actually the case or whether it's a mirage perpetuated by the changing nature of how we're interconnected via the Internet, or perhaps because world events went through a sort of unusually calm period in the 80s and 90s, or perhaps it's as simple as the notion that we were mostly sheltered by our parents as children to some extent and didn't truly open our eyes to the reality of the world until we got older...

  8. The lesson we can all learn from this: by Mantrid42 · · Score: 4, Insightful

    Shit happens.

  9. Re:Error: They did not use LaTeX by vux984 · · Score: 4, Informative

    This could have been avoided if the authors had used LaTeX for writing their paper.

    Hardly. This would have been avoided if the authors had written:

    (************ SHOULD WE CITE THE CRAPPY XYZ PAPER HERE *************)

    And then it wouldn't have gotten missed even in Notepad. In anything more advanced than notepad I'd also format it bold, and in red too.

    Arguing for the commenting features of latex presume they would actually know about the feature, AND choose to use it. For all I know they did use latex, but didn't bother to mark it as a comment. (I mean, they probably used Word, and they didn't mark it as a comment with that either, which they could have done -- so why would switching to latex make them use the feature??)

    But using the commenting feature would also potentially be a detriment. They may well WANT their own review, and internal reviewers to see this stuff, so that they can render an opinion. Having it simply omitted from the PDF or printout they are looking at means they don't see it, and can't mark a note ... "Hey -- you should cite that paper" or "don't bother with that"... in their review notes.

  10. Re:Big woop by Anonymous Coward · · Score: 3, Insightful

    "The children now love luxury. They have bad manners, contempt for authority, they show disrespect to their elders.... They no longer rise when elders enter the room. They contradict their parents, chatter before company, gobble up dainties at the table, cross their legs, and are tyrants over their teachers."

    Widely attributed to Socrates, ~450BC.

    Middle aged and elderly doomsayers love to bitch about how their generation were upstanding citizens but today's kids are nothing but morally bankrupt punks who are letting civilization go to shit.

    Civilization was never as good as our rose-tinted glasses from the time when we were young, beautiful, horny, and ignorant of what an awful place the world really is make it look. In other words, "/b/ was never good."

  11. Re:Big woop by sjames · · Score: 3, Interesting

    There are a few objective measures that can be made. We know that professional employment was once considered to be life-long. We know that employers used to offer on the job training and actual entry level employment. We know that at one time retail employers believed 6 days a week and observance of national holidays was just fine. We know that single income families was once the norm.

    That's not to say things were perfect. The red scare and blacklists were real. We don't really know if the various spy agencies were more scrupulous at the time or if they just didn't have enough technology and manpower to behave as badly as they do today.

    I do know that for whatever reason (simple ability increasing or moral decay) every year the U.S. does more and more of those things that my 4th grade teacher said the 'Russians' (meaning the USSR) were bad for doing. It's not just childhood sheltering. I know for a fact that at one time you really could just walk through the airport with suitcase and ticket in hand and get on a plane with no form of ID whatsoever. Your suitcase would be run through an x-ray and you would pass through the worlds least sensitive metal detector. If you had a video camera that looked like an Uzi on an X-ray, you and the security guy could have a good laugh about it (once he looked in the bag, naturally).

    Mysterious objects found in public created funny urban legends (if they were even noticed), not civil panic.

  12. Re:Big woop by radl33t · · Score: 3, Insightful

    zero tolerance does that.

  13. Re:Big woop by ShanghaiBill · · Score: 3, Informative

    The arsenals are still in place, almost as powerful as they were at peak.

    No. Nuclear arsenals have dramatically declined, in both number and average warhead yield. In the 1960s, America had more than 30,000 warheads. Today we have less than 5,000. The Russian arsenal has declined even more. Here is a nice graph.