Slashdot Mirror


1 In 4 Statisticians Say They Were Asked To Commit Scientific Fraud (acsh.org)

As the saying goes, "There are three kinds of lies: lies, damned lies, and statistics." We know that's true because statisticians themselves just said so. From a report: A stunning report published in the Annals of Internal Medicine concludes that researchers often ask statisticians to make "inappropriate requests." And by "inappropriate," the authors aren't referring to accidental requests for incorrect statistical analyses; instead, they're referring to requests for unscrupulous data manipulation or even fraud. The authors surveyed 522 consulting biostatisticians and received sufficient responses from 390. Then, they constructed a table that ranks requests by level of inappropriateness. For instance, at the very top is "falsify the statistical significance to support a desired result," which is outright fraud. At the bottom is "do not show plot because it did not show as strong an effect as you had hoped," which is only slightly naughty.

19 of 95 comments (clear)

  1. Only 1 in 4? by HornWumpus · · Score: 4, Interesting

    1 in 4 biostatisticians...

    Dollars to donuts it's much worse in the soft 'sciences'. Slightly remediated by the fact they're too stupid to realize what they were asking was wrong.

    --
    John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    1. Re:Only 1 in 4? by hey! · · Score: 4, Insightful

      This study is talking about biostatisticians. Most of those guys are bound to be working for pharmaceutical companies.

      As for the social science Brian Wansink was recently stripped of his Cornell professorship when he and is lab were caught doing extensive "p-hacking". Interestingly, the research they were doing was essentially psychological in nature, but Wansink has no academic training in psychology; he has a BA in business administration, an MA in journalism and a PhD in marketing, and his lab was operated out of Cornell's business school.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    2. Re:Only 1 in 4? by Z80a · · Score: 2

      Well the true question is, can we have the opinion of another 3 statisticians? If one in four are wrong, this one could be as well.

    3. Re:Only 1 in 4? by dcw3 · · Score: 2

      Brian Wansink was recently stripped of his Cornell professorship when he and is lab were caught doing extensive "p-hacking"

      But p-hacking is a thing in pretty much every field, and it needs to be stopped.
      https://fivethirtyeight.com/fe...

      --
      Just another day in Paradise
  2. Not a shocker by SirAstral · · Score: 4, Interesting

    1 out of 4 are asked to commit fraud.
    2 our of 4 are "expected" to commit fraud without being asked.
    1 out of 4 are actually trying to get at some form of truth.

    Statistics are always biased by their sample sizes, and criteria.

    There are lies, damn lies, and then there are statistics.

  3. The scientific method by petes_PoV · · Score: 2, Insightful
    And when the experiment is repeated - many times, by different teams in different labs using different statistical techniques to analyse the results, the truth will come out.

    But if an experiment is only performed once, never scrutinised, never checked, never tested then there can be little or no confidence in its conclusions.

    --
    politicians are like babies' nappies: they should both be changed regularly and for the same reasons
    1. Re:The scientific method by ClickOnThis · · Score: 3, Informative

      And when the experiment is repeated - many times, by different teams in different labs using different statistical techniques to analyse the results, the truth will come out.

      But if an experiment is only performed once, never scrutinised, never checked, never tested then there can be little or no confidence in its conclusions.

      Even if an experiment is not repeated exactly, its results still provide a way-point that can be scrutinized in future studies. Other scientists will try to build on previous results, and if something subsequently does not make sense, they will back-trace to find the problem. This is often how science evolves.

      Experiments are often repeated, at least implicitly, if some process that previous experimenters followed must be followed again to pick up where they left off. And often it is worthwhile to repeat an experiment with improved equipment, to see whether additional insights can be found.

      In short, don't dwell on whether there is a cadre of scientists who make it their mission to repeat other scientists' experiments. That's impractical, and frankly silly. Scientific studies do get scrutinized and repeated (at least implicitly) -- just not in the narrow way you suggest.

      --
      If it weren't for deadlines, nothing would be late.
    2. Re:The scientific method by LynnwoodRooster · · Score: 2

      Well, they can also hide their data and not publish it, just the "processed" results...

      --
      Browsing at +1 - no ACs, I ignore their posts. So refreshing!
    3. Re:The scientific method by Megol · · Score: 4, Informative

      https://retractionwatch.com/

      Notice that papers that have been used to direct research and being used as supporting data often have been detected as frauds long after the publication. That means that the falsified data have already escaped most scrutiny and have already wasted time, money and effort.

    4. Re:The scientific method by terrycarlino · · Score: 2

      Fair enough. But I would say that these are examples of the scientific method working as it should. Mistakes or fraud may take time to detect, but sooner or later they are corrected.

      And keep in mind that science is not the only human endeavour that has occasionally wasted time, money, and effort. Science progresses most efficiently when honest actors work together in good faith, scrutinizing each other's work but also building mutual trust. The waste from occasional bad actors is eclipsed by the benefit from the good ones.

      That's all very well when science is relegated to its own little corner of the universe interested only in unimportant issues like when did the universe begin or is there life on some distance planet which will be unreachable for the next big bite of eternity.

      It's quite another thing when fraudulent science is used to direct public policy which has an immediate and negative effect on people's actual lives. It is even more a problem when fraudulent science is used to precipitate cultural change that effects whole populations.

    5. Re:The scientific method by phantomfive · · Score: 2

      Experiments are often repeated, at least implicitly, if some process that previous experimenters followed must be followed again to pick up where they left off. And often it is worthwhile to repeat an experiment with improved equipment, to see whether additional insights can be found.

      This is an extremely inefficient way of doing it, and it can take decades for the error to be corrected, even in a hard field like physics (Feynman gives the example of the oil drop experiment. It was also an example of incorrect previous studies leading newer studies astray).

      In short, not double-checking studies can lead to wrong results for decades, or longer. Think of the confusion in nutritional science.

      --
      "First they came for the slanderers and i said nothing."
    6. Re:The scientific method by dcw3 · · Score: 2

      Inefficient? Yes, but somehow the world hasn't come up with a better way yet. We can't even get decent peer review in many cases because there's simply no money/glory in reviewing other people's work. The peer review system has mostly been broken for a long time because of that simple fact.

      https://www.ncbi.nlm.nih.gov/p...
      https://www.wired.com/2014/12/...
      https://wattsupwiththat.com/20...

      --
      Just another day in Paradise
  4. Can We Trust This Statistic? by careysub · · Score: 2

    Statistics about statistical fraud. Down the rabbit hole we go.

    --
    Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
  5. Re: want your next grant? by c6gunner · · Score: 3, Funny

    Was it the guy who wrote the fucking summary?

  6. Re:*ALL* stats are lies. They only show what... by ClickOnThis · · Score: 3, Informative

    They only show what the author wants you to see, period. Whether biostats guy wants to lie to us or not... if his data bogus then stats are bogus.

    No stats are even likely valid without *FULL* data being presented with nothing hidden or omitted. It is filtering process or dropping outliers, makes the stats falsehoods at best.

    Remember with stats... any data can *prove* anything by at least misdirection.

    A liar (scientist or not) can make a lie sound like the truth. Science does occasionally have bad actors who lie. Their lies are discovered and corrected sooner or later.

    What was your point again?

    This is same as the proof that shows 1=2.
    A=B
    A*A=B*A
    A*A-B*B=B*A-B*B
    (A+B)(A-B)=B(A-B)
    A+B=B
    B+B=B
    2=1

    "The same?" Well, no. Anyone who has take high-school math (and that includes scientists) can spot the flaw in your "proof." When you divided out the (A-B) factor, you divided by zero.

    Or the hotel $1
    3 guys check into a room
    Room cost $30 (long ago)
    Each paid $10.
    Night Audit determined the over charged, should be $25 (honest place)
    Bellman sent up with $5 to return to them (yes still have them too)
    Guys did not have change to split... so each took $1
    They gave the bellman $2
    So, Each paid $9 for the room for $27
    and paid $2 to bellman, for a total of $29
    Where is the missing dollar?

    There is no missing dollar. The hotel ended up charging the 3 guys $25. They paid $27. The bellman collected a $2 arbitrage ($27 - $25 = $2.)

    --
    If it weren't for deadlines, nothing would be late.
  7. Re:want your next grant? by rtb61 · · Score: 3, Interesting

    Like the sugar rush animals did. Only when you look at the detail did you discover the children were under calorie controlled diets the whole time. Here is this one lolly and a pound of celery, see no sugar rush, a complete fraudulent lie and splashed all over corporate main stream media as proof eating a pound of candy does not cause a sugar rush. Not for the sick cunts who did that test because they were paid to produce advertising statistics that would kill children because greed.

    Not to forget the calorie counters, ignoring human digestible calories versus calories a human can simply not digest but will burn in a https://en.wikipedia.org/wiki/.... Yeah the fuckers compared Almonds to a can of coke but carefully left out the block of wood because, yeah, you would realise the scam. This fuckers need to be charged with man slaughter.

    --
    Chaos - everything, everywhere, everywhen
  8. Re:want your next grant? by ClickOnThis · · Score: 3, Insightful

    Uh, what???

    [...wait, let's read this again...]

    UH, WHAT???

    --
    If it weren't for deadlines, nothing would be late.
  9. Re:want your next grant? by Layzej · · Score: 3

    You do know that scientists are expected to acknowledge their sources of funding in their publications, right?

    And you do know that often they do not, right?

    Here's an example of the same. Willie Soon is heavily funded by the fossil fuel industry.

    The documents show that Dr. Soon, in correspondence with his corporate funders, described many of his scientific papers as “deliverables” that he completed in exchange for their money. He used the same term to describe testimony he prepared for Congress.

    But in those same papers he failed to disclose and often explicitly denied any conflict of interest or outside funding.

  10. Re:want your next grant? by GuB-42 · · Score: 2

    As long as there is an incentive to get any result, there will be fraud. In fact you don't need any incentive at all.

    If I love the color red and the blue stuff turns out better, I will be tempted to tweak the results.