Slashdot Mirror

← Back to Stories (view on slashdot.org)

China's Scientists Set New International Record -- For Faked Peer Reviews (nytimes.com)

Posted by EditorDavid on from the regretting-retractions dept.

China now has more laboratory scientists than any other country in the world, reports Amy Qin in the New York Times, and spends more on research than the entire European Union. But in its rush to dominance, China has stood out in another, less boastful way. Since 2012, the country has retracted more scientific papers because of faked peer reviews than all other countries and territories put together, according to Retraction Watch, a blog that tracks and seeks to publicize retractions of research papers... In April, a scientific journal retracted 107 biology research papers, the vast majority of them written by Chinese authors, after evidence emerged that they had faked glowing reviews of their articles. Then, this summer, a Chinese gene scientist who had won celebrity status for breakthroughs once trumpeted as Nobel Prize-worthy was forced to retract his research when other scientists failed to replicate his results. At the same time, a government investigation highlighted the existence of a thriving online black market that sells everything from positive peer reviews to entire research articles...

In part, these numbers may simply reflect the enormous scale of the world's most populous nation. But Chinese scientists also blame what they call the skewed incentives they say are embedded within their nation's academic system.

36 of 75 comments (clear)

  1. Incentives are skewed everywhere by Anonymous Coward · · Score: 3, Insightful

    "In part, these numbers may simply reflect the enormous scale of the world's most populous nation. But Chinese scientists also blame what they call the skewed incentives they say are embedded within their nation's academic system."
    I don't see how these skewed incentives are any different in Western countries.

    1. Re:Incentives are skewed everywhere by nachtelfjeiu · · Score: 1

      That was me btw, i hadn't logged in.

    2. Re:Incentives are skewed everywhere by Anonymous Coward · · Score: 1, Insightful

      I don't give a shit who you are.

    3. Re:Incentives are skewed everywhere by RightwingNutjob · · Score: 1

      RTFA. The incentives may be the same (in kind, if not in degree), but here there are strong disincentives from faking shit that don't exist over there.

    4. Re:Incentives are skewed everywhere by Anonymous Coward · · Score: 3, Interesting

      I don't see how these skewed incentives are any different in Western countries.

      Presumably you have performed research in both China and in the West and your comment is drawing on your deep knowledge of the educational systems of both countries? No? The ability of slashdotters to hold strong opinions on subjects which they know next to nothing about never ceases to amaze. In China, Master's students must publish a certain number of papers above a certain impact factor in order to graduate. For PhD students, the bar is set higher. Principal investigators are given cash sums - which can be quite substantial - based on the impact factors of their papers. Add to this the weak separation between personal income and research funding and the incentive to cheat is huge.

    5. Re:Incentives are skewed everywhere by TapeCutter · · Score: 1

      Glad you cleared that up. I would have thought the fact that they have a large number of retractions indicates their peer-review system is working, not failing. For comparison, how long did it take before Andrew Wakefield's "research" was retracted?

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    6. Re:Incentives are skewed everywhere by SNRatio · · Score: 1

      I would have thought the fact that they have a large number of retractions indicates their peer-review system is working, not failing.

      Not exactly. Peer review happens before publication, not after. For an article to be retracted it first had to pass peer review. In the case mentioned in the story, the journal (Tumor Biology, which has a US/European editorial board), allows authors to suggest peer reviewers and trusted the contact information they received. It's pretty trivial to verify the email address of a professor in the US or Europe. The retractions happened because these authors used names of real Chinese professors - but fake email addresses.

      So they wrote their own peer reviews.

    7. Re:Incentives are skewed everywhere by gtall · · Score: 5, Informative

      Maybe it works for Chinese publications. However, Chinese papers don't just get submitted to Chinese publications, they get submitted to journals and conferences outside China. It isn't easy reading a paper for review. Unless you are doing precisely similar research, you must learn enough about the research to know if it is good or not. I read a (Chinese) paper (written in English...well written English, I might add) on rings (mathematics). I'm not a ring theorist but I do know a bit of algebra. I decided I wouldn't just read the paper but track down every result. Marvelous paper except for the first theorem upon which all the rest were based. I couldn't prove it, and I tried hard. Many times papers do not include all the proofs because it would make the paper too long for publication or they are considered trivial in the field.

      After writing and Latexing 15 pages of notes and proofs on the rest of the paper, I radioed back I wanted to see their proof of that theorem. What I got back was a reference and how it was a trivial conclusion from the reference. I found the reference and read it (yet another paper I had to read after tracking down and reading some of their previous refs). I couldn't see it. I radioed back I wanted to see an honest proof, not invocation to a Higher Authority. After 2 months, they retracted the paper. The total time from my first seeing the paper to that retraction was 8 months and several long days of my time....on one paper...

      My point is that few reviewers are going to dig in their heels and properly review a paper, few have that kind of time. After that, I'll be damned if I'm not going to read another paper the same exact way. It will cost me in time, but I'll learn new things and maybe another piece of shit won't make it into a journal.

    8. Re:Incentives are skewed everywhere by shaitand · · Score: 1

      While it is nice for the journal to effectively run on auto-pilot in that way it seems like a poor plan overall for peer review. Obviously the person submitting the article will be most familiar with those working on the most related research but that is just setting dishonesty as low hanging fruit tempting frustrated researchers to compromise their integrity.

    9. Re:Incentives are skewed everywhere by shaitand · · Score: 1

      Informative if accurate. Hopefully someone moderates that way.

      The GP was likely making a redundant snarky remark about there being a strong incentive to cheat both in the West and in China without details on the Chinese pressures. Based on your details there is even greater incentive to cheat in China although it also remains true that there is a great deal of incentive in the West as well.

      Even worse than incentive to outright fake data is incentive to be selective about researched topics and slant in the direction of research not to mention slant in the conclusions and summaries of papers. For example, a conclusion that chemical x is indeed harmful because data showed any sort of potential harm and completely disregarding the relative potential harm vs other used substances either related and commonly used or generally considered low risk. When there is a very small number of large players funding an area of research an extreme bias can be presented with any search on the safety of chemical x possibly returning nothing but dozens of negative seeming papers all technically true and seeming to provide an overwhelming mountain of negative evidence which is largely vapor.

    10. Re: Incentives are skewed everywhere by nachtelfjeiu · · Score: 1

      Thank you :-) It's also about taking some responsibility for my words.

  2. Incentives by JBMcB · · Score: 1

    Because the government is throwing billions of Yuan indiscriminately into research. There is incentive to get that money. Grants are somewhat less forthcoming in the US, so there is a higher requirement for better quality research.

    --
    My Other Computer Is A Data General Nova III.
    1. Re:Incentives by Anonymous Coward · · Score: 5, Insightful

      Sigh... people talk about research money like the researchers themselves actually profit from it. 99% of the time, researcher salary is fixed completely independently of grant money obtained.

      Instead, we could actually RTFA, the paragraph after...

      As in the West, career advancement can often seem to be based more on the quantity of research papers published rather than the quality. However, in China, scientists there say, this obsession with numerical goal posts can reach extremes. Compounding the problem, they say, is the fact that Chinese universities and research institutes suffer from a lack of oversight, and mete out weak punishments for those who are caught cheating.

    2. Re:Incentives by roc97007 · · Score: 1

      "researcher salary is fixed completely independently of grant money obtained."

      True. But they get to have a job, and they get to have their names on stuff.

      --
      Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
    3. Re:Incentives by shaitand · · Score: 1

      Do you work as a researcher? This is technically true at most places but in reality being able to afford and justify spending money on you ultimately depends on grants. Also, there are often others working under a researcher on various projects doing research, computation, administering the systems and if funding goes away those people will all be out of the job. Unless they are a sociopath a researcher does feel some level of responsibility for those people.

  3. Not just about large population by JoshuaZ · · Score: 2
    From the summary:

    In part, these numbers may simply reflect the enormous scale of the world's most populous nation.

    That is probably part of it, but it is worth emphasizing that that is definitely not all of it. The per a capita retraction rate for China is much higher than it is for other large countries.

    1. Re:Not just about large population by thegarbz · · Score: 1

      Genuinely interested, but what about per academic? I suspect it would be high too since much of the Chinese population won't ever get anywhere near a university.

    2. Re:Not just about large population by slashrio · · Score: 1

      Yes, for China, divide by 1.2 billion.

      --
      "Trump!!", the new Godwin.
  4. "a new international record" by roc97007 · · Score: 1

    Keep in mind that this only means that China is in first place, not that this is the only place this is happening.

    --
    Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
  5. Re: Chinese are programmed to cheat. by Kleanthes · · Score: 2

    Yeah, would be much better if they were the 'murican type of Christian. Then they would also cheat, but at least they would also loudly complain about cheaters.

  6. sounds familiar. by Gravis+Zero · · Score: 4, Insightful

    Chinese scientists also blame what they call the skewed incentives they say are embedded within their nation's academic system.

    It sounds like they have a similar problem to the US's collapsing "publish or perish" paradigm. People should be less focused on what the scientists are doing and focus on the cause of such behavior.

    To change the behavior of a group you must correct the feedback loops that control them.

    --
    Anons need not reply. Questions end with a question mark.
    1. Re:sounds familiar. by Peter+Desnoyers · · Score: 2, Interesting

      Chinese scientists also blame what they call the skewed incentives they say are embedded within their nation's academic system.

      It sounds like they have a similar problem to the US's collapsing "publish or perish" paradigm. People should be less focused on what the scientists are doing and focus on the cause of such behavior.

      To change the behavior of a group you must correct the feedback loops that control them.

      In the US, gross misconduct (like impersonating other scientists in order to review your own papers) is a career death sentence, in part because "publish or perish" is administered by a tenure vote of the people you work with (and compete with), instead of a bean-counting administrator somewhere. There are lots of incentives to do semi-unsavory things - e.g. splitting your work into "least-publishable units", or "P-hacking", where you try every combination of data to see if one of them supports your conclusion - however if you cross the line and start doing things your colleagues aren't willing to do, they'll be happy to come down on you like a ton of bricks.

    2. Re:sounds familiar. by Anonymous Coward · · Score: 5, Insightful

      Oh please spare me. I'm involved in nutrition research and I see the contortions others go through to support whatever conclusion is best for the corporation helping fund said study. It's no accident, statistically, when Corporation F funds Study U, that Study U will have an 80% chance to put Corporation F's dubious products in a good or at least neutral light.

    3. Re:sounds familiar. by Anonymous Coward · · Score: 2, Interesting

      In the US, gross misconduct (like impersonating other scientists in order to review your own papers) is a career death sentence, in part because "publish or perish" is administered by a tenure vote of the people you work with (and compete with), instead of a bean-counting administrator somewhere.

      https://science.slashdot.org/story/15/02/13/2113248/unearthing-fraud-in-medical-trials

      you new to the research thing?

    4. Re:sounds familiar. by Peter+Desnoyers · · Score: 1

      Sorry, I'm used to my area of research - computer science - which is a hell of a lot less shady, at least in the US. (probably because almost all the money is being made totally outside of academic and research settings)

      The article you link makes me wonder whether there's a way that publishers (e.g. NEJM and JAMA) could force clinical trial data out into the open. If it were a requirement for publication in the top venues, then the drug companies would have to either 'fess up or work with second-rate researchers. (unfortunately there's probably an option 3, which they'll find and we won't like...)

  7. Can't fake everything... by Peter+Desnoyers · · Score: 4, Funny

    Gotta love this quote from one of the linked articles:

    "When a lot of the fake peer reviews first came up, one of the reasons the editors spotted them was that the reviewers responded on time"

  8. And in a related story by mnemotronic · · Score: 1

    The Saint Petersburg-based Internet Research Agency (IRA) announced the closure of operations that for the past 4 years have been targeting subscribers and users of Twitter, Reddit and Facebook. According to Agency General Directory Vyascheslav Fontyaev, "The minds of the American public are now under our control via other channels. We no longer need to use artificial Facebook or Twitter accounts to obtain our political targets". Last Saturday the agency claims it had closed or deleted all of it algorithmically created accounts on Facebook and Twitter. This coincided with what Facebook officials say is an unexplained disappearance of 48% of it's user base. Twitter release a press briefing with similar news, claiming that 64% of it's accounts were mysteriously closed and within an 48 hour period. On Monday morning, investors in both companies dumped shares sending the stock price of both companies tumbling; down 77% for Facebook and 44% for Twitter.

    Director Fontyaev said that the employees of Internet Research Agency would be retaining most of the Reddit accounts. "The Reddit users are more fun to poke" said Fontyaev. "They accept complete falsifications far quicker and are great fun to toy with. Besides, the restricted "gonewild" sub-reddit is a favorite of the Agency's mostly male work force".

    --
    The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
    1. Re:And in a related story by mnemotronic · · Score: 1

      I hope you didn't spend too much time writing that as it didn't turn out very well.

      Yea. TMI. Gotta learn to get to the point or punch line. Or in this case, have one.

      --
      The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
  9. More fake biology/medical research by riverat1 · · Score: 4, Interesting

    One thing I see when I look at faked research and retractions of papers is that it often is in biology and medical research or things like sociology. In the hard sciences like physics, chemistry, astronomy, geology, meteorology and dare I say it climatology it doesn't seem to happen nearly as often. Maybe it's harder to fake the data in those sciences or maybe there's just more variability open to interpretation in the results from biology/medicine.

    1. Re:More fake biology/medical research by SNRatio · · Score: 3, Insightful

      One thing I see when I look at faked research and retractions of papers is that it often is in biology and medical research or things like sociology. In the hard sciences like physics, chemistry, astronomy, geology, meteorology and dare I say it climatology it doesn't seem to happen nearly as often. Maybe it's harder to fake the data in those sciences or maybe there's just more variability open to interpretation in the results from biology/medicine.

      I think a big part is that when it's medicine the media is more likely to pick up the story. Jan Hendrik Schön was one of the biggest scandals - but it didn't really make a big splash in the news because solid state physics just isn't something most people get worked up about. Ditto for Adrian Maxim. So unless you are a regular at Retraction Watch, you hear about Wakefield and Hwang Woo-Suk (cloning), but not about physics or engineering fraud.

      I also wonder if the problem is really that much worse than it was in the past. Granted, academia is more competitive than it used to be. But really, the big change is that it is so much easier to find fraud than it was in the past. You're right - it is not that hard to make some fake data for a biology paper. You photoshop the picture of your gel (a technique for showing which proteins are present in a sample and whether they are interacting with each other) to show the results you want. 30 or 40 years ago: same thing, though you would have had to do it manually. But now image analysis can catch that easily. Ditto for plagiarism. Ditto for analyzing sets of numbers to see if they were observed or invented. And because it is easier to examine for fraud, and fraud is actually routinely talked about, more people are looking for it than in the past. Now, fraud stands a good chance of being caught. Back then: not so much.

    2. Re:More fake biology/medical research by riverat1 · · Score: 1

      It always surprises me that smart guys like most scientists are think they can get away with stuff like that. After all sooner or later some one else is likely going to try and use their results and discover their fraud especially it it appears to be ground breaking research.

      Also it's true that sometimes retractions are a result of honest mistakes that weren't caught by peer review rather than by attempted fraud.

    3. Re:More fake biology/medical research by Peter+Desnoyers · · Score: 1

      In the hard sciences there are few companies that make money off of the success or failure of an academic research project.

      That doesn't explain sociology; however searching through retractionwatch.com I'm not sure how much misconduct there is in that field (at least within the US and western Europe) as opposed to just plain errors. Which shouldn't be surprising in a field with small sample sizes, poor funding, and high noise.

  10. Re:Chinese are programmed to cheat. by slashrio · · Score: 1

    The good news is: They actually retracted them.

    --
    "Trump!!", the new Godwin.
  11. Re: Chinese are programmed to cheat. by Anonymous Coward · · Score: 1

    Our present economic boom is due to the revolution in electronics and computer technology.

    Our present economic "boom" is a recovery from the country being driven to the edge of bankruptcy by the real estate bubble collapse and most of those gains went to the 1% in the form of returns on cheaply purchased shares in bailed out financial institutions. The middle and lower class have less purchasing power than ever due to stagnant wages that show no sign of improving without government intervention.

    In other words, congratulations on drinking the Kool-Aid. You've done an excellent job defending rich old white men who are raping the economy, the environment and the people of the country for their own gain. I'm sure you'll enjoy your reward of having your health care cut, less funding for FEMA to help others and and a big increase in your taxes while the 1% suffer through their taxes and obligations to society being reduced.

  12. Re: Chinese are programmed to cheat. by Pinky's+Brain · · Score: 1

    A lot of civilizations just collapsed with no lasting legacy but ruins, their philosophical and technological innovations forgotten until our Civilization's archaeologists started to uncover them. We got extremely lucky by inheriting the progression of preceding civilizations instead of just letting it all be reduced to rubble. Modern civilization is a complete fluke.

    Seculars have now taken over the baton for the most part, lets see if they can keep it going. I kinda doubt it, they are falling into the same trap as Christianity in its dying days, lack of conviction in their superiority, hollowed out by human rights worship. Sure liberals think themselves superior to white Christians and are willing to twist every which way to combat them, but at some point Christians and whites are going to be irrelevant and they'll be surrounded by the faster breeding "allies" they brought in to defeat them. Liberals will have to go full commie and give up on human rights worship for parts of their culture to survive, I'm not sure they will be able to.

  13. Re:Doesn't the editor control who reviews by SunTzuWarmaster · · Score: 1

    Editor: "Please suggest 8 people to review your work"
    Researcher: "Okay, I suggest WellKnownResearcher, their mailing address [POBox that I own], and their E-mail address [WellKnownResearcherTopic@yahoo.com, an E-mail I just registered]"
    Editor: "To:WellKnownResearcherTopic@yahoo.com Body: Dear WellKnownResearcher, what do you think of this paper?"
    Researcher: "Wow. What great results. You should totally 100% publish this in your journal because it is, like, totally legit. Here are some ways in which it can be better, but mostly just act as a disguise."
    Editor: "Dear Researcher, your paper was well reviewed, we are happy to publish it."
    Researcher: "Dear University, I now meet the XXX minimum papers required to graduate requirement." ...
    Time passes and the paper is rescinded for all of the right reasons (cannot replicate, fraudulent review discovered, etc.). Chinese "academic" gets away more-or-less scott free, depending, because academic dishonesty in China is treated very different than, say, the US or EU, where it is punishable by "career death."