How a Scientist Fooled Millions With Bizarre Chocolate Diet Claims

__roo writes: Did you know chocolate helps you lose weight? You can read all about this great news for chocoholics in the Daily Star, Daily Express, Irish Examiner, and TV shows in Texas and Australia, and even the front page of Bild, Europe's largest daily newspaper. The problem is that it's not true. A researcher who previously worked with Science to do a sting operation on fee-charging open access journals ran a real—but obviously flawed—study rigged to generate false positives, paid €600 to get it published in a fee-charging open access journal, set up a website for a fake institute, and issued press releases to feed the ever-hungry pool of nutrition journalists. The doctor who ran the trial had the idea to use chocolate, because it's a favorite of the "whole food" fanatics. "Bitter chocolate tastes bad, therefore it must be good for you. It's like a religion."

Comments

Heh.

[Aaron_Pike]

Schadenfreude is the best freude.

Re:Heh.

[Penguinisto]

True, though it sadly proves P.T. Barnum's maxim, and says more about a gullible public, the lack of peer review in the field of nutrition (and worse, the sheer incompetence of so-called 'nutrition journalists' and 'specialists'), than it does about a science journal's shady/sloppy practices.

Long story short, it exposes a hell of a lot more than just what the scientist initially wanted exposed.

Maybe someone could do and publish a sociology study from it?
(/me ducks and runs like hell...)

Re:Heh.

[Opportunist]

Mostly it exposes that people love to believe stories they like. And of course journalists love to publish stories that their readers like.

I just don't think that qualifies as news, though.

Re:Heh.

[michelcolman]

The big problem is that everyone will now remember the fake study and continue to believe it, because the rectification doesn't get nearly as much coverage. People are still refusing to vaccinate children because they're afraid of autism even though the author of that study actually confessed having made the whole thing up.

Re:Heh.

[RespekMyAthorati]

Andrew Wakefield was deliberately fraudulent, and that is why the paper was retracted and his medical license revoked.

chocolate helped me lose weight

[turkeydance]

i just don't eat it.

This was done by a journalist, not a scientist!

[metageek]

The OP is wrong on this, the guy is a JOURNALIST not a scientist. (He may have a science degree but he acts as a journalist).

If a scientist had done this they would be losing their job any minute. Any of the following would be enough to disgrace a practicing scientist (I am one):

1- carrying out research on human subjects without approval of the study by an independent review board
2- asking people to undergo a study that he knew before hand that was not beneficial to the subjects, in fact could likely be the opposite (this would mean he'd never get approval of the study)
3- lying to people about his affiliation and credentials in the paper

There are a lot of other problems with this "study", but it surely was not done by a scientist.

What it does reveal is that people cannot rely on popular press stories about science as journalist pay no attention to the important details of publications and fall for any hype. Unlike what the guy says, journalist can never be "peer" reviewers of any science... their role is different and yet they are not doing it properly.

Re:This was done by a journalist, not a scientist!

[Anonymous Coward]

0) Oh, he's not a scientist? He's doing fucking SCIENCE here. He had a hypothesis, conducted an experiment, published the fucking result. He created new information about the state of scientific journalism. But maybe you're confusing academics with science.

1) Get real. There was no potential for harm. Your "review board" requirement raises the bar high enough to render unethical every science fair project ever done.

2) Exactly like, for example, every control group, ever?

3) That was the basis. Of. The. Study.

Re:This was done by a journalist, not a scientist!

[captnjohnny1618]

I think you misread the post (and/or the article).

Yes, the author of the story is a science journalist. The "study" was intentionally bad to demonstrate the current sad state of "science journalism" (with an easy target like nutrition) where few of these journalists actually take the time to actually read the article, much less actually fact check on a critical, challenging or even marginally scientific level.

So your final conclusion is exactly what he wanted to show: you can't blindly trust popsci journalism... but your methods for arriving there are a little roundabout.

Re:This was done by a journalist, not a scientist!

[AthanasiusKircher]

You're absolutely correct that this story is mostly about how we should distrust science journalism, and the author of the "study" makes that clear.

However...

If a scientist had done this they would be losing their job any minute. Any of the following would be enough to disgrace a practicing scientist (I am one):

Yes, all of that is true. But I think you're exaggerating quite a bit about the impact any of this would have had on whether a study like this could have been done by a reputable scientist (it certainly could have been), or whether it would have been published if that scientist had appropriate credentials (as noted in the article, there are plenty of places to publish bad research with little vetting).

2- asking people to undergo a study that he knew before hand that was not beneficial to the subjects, in fact could likely be the opposite (this would mean he'd never get approval of the study)

The details of methodology in this study might have been changed a bit to get it approved by a review board, and there's certainly nothing about the basic idea of this study that suggests it would NECESSARILY (or even likely) have significant detrimental effects for the subjects. And there is certainly at least a possibility of beneficial effects.

I suggest you have a look at some of the research articles linked even in an article of Wikipedia on the subject. There have been dozens, and probably hundreds of studies that have tried to measure health effects of chocolate -- many of them have involved people eating small amounts of chocolate and observing effects, just as the study here did.

So, the idea that an actual scientist, with the appropriate amount of time, could "never get approval of the study" is just ridiculous. As I said, there would probably be some more detailed methodological justification and tweaks, but lots of nutrition studies like this happen all the time.

Unlike what the guy says, journalist can never be "peer" reviewers of any science... their role is different and yet they are not doing it properly.

Yes, journalists have their own jobs to do, and there are certainly flaws in the system.

But there are flaws in the science system too, which makes the job of journalists (and scientists trying to look at research out there and evaluate it) much harder. "P hacking" is not just something made up by this journalist -- it's a real thing, and it's a real problem. Yes, many reputable journals have tried to make review procedures better to avoid various statistical problems, but they often don't really fight them head-on (with a few notable exceptions of brave editors or boards). There are a LOT of problems with common statistical procedures followed by researchers -- even those who have proper credentials and have gotten independent reviews.

Anyhow, most of your criticism boils down to "this guy wasn't credentialed to do what he did." That's great, but it doesn't address the larger flaws here -- not only in journalistic reporting, but in some aspects of scientific methodology. This study was clearly not rigorous, but I've seen worse studies published in reputable journals.

Re:This was done by a journalist, not a scientist!

[Anonymous Coward]

Scientist: Lawful Neutral
HD d4
  BaB Fort Ref Will Special 0 1st
1st +0 +0 +0 +2 "Summon Assistant, Scribe Scroll" 3 1
Incurs casting penaly when not wearing white robe
If a Scientist fails its alignment check, it loses all special abilities.

Re:This was done by a journalist, not a scientist!

[taustin]

If a scientist had done this they would be losing their job any minute.

Given that we're talking about the "nutrition" industry, if a scientist had done this, it would likely be his job. Specifically.

That's the point.

Re:This was done by a journalist, not a scientist!

[narcc]

It wasn't a study, it was a publicity stunt.

Re:Scientists are generally trusted

[CraigGlaser]

Anyone who trusts scientists has no idea how science works. You don't trust the person or title, you trust the chain of independent verification of the data. This is a critical thinking issue.

Re:Scientists are generally trusted

[Anonymous Coward]

I disagree.

Scientists are widely distrusted.

Except when they tell us what we want to hear.

Re:Because I did not read the original article...

[Ignacio]

It was never science in the first place. He used known flaws in experimentation to prove that you can make the mainstream publish anything. Which was the point he was trying to make.

This works 100%

[bogaboga]

Forget all those so called diet plans.. The only thing I know that works is actually eating less. Many of my friends will not agree, but I wonder how one can gain weight by putting less calories in their system.

Required reading: xkcd's "Significance"

[dwheeler]

xkcd's "Significance" pithily explains p-hacking.

Re:Scientists are generally trusted

[Penguinisto]

Here's the trick: You and I know this, but the average schlub out there does not.

That distinction is kinda vital, and it's what I think GP was driving at.

Re:Scientists are generally trusted

[Anonymous Coward]

More to the point, it's impossible to independently (& personally) verify the data and claims of everything that you would like verified. There's not enough time in the world.

Re:Scientists are generally trusted

[ShanghaiBill]

Here's the trick: You and I know this, but the average schlub out there does not.

Science journalists should be better than "average schlubs". They owe it to their readers to do at least a few minutes of fact checking before publishing.

Re:Logic

[tomhath]

For most people, a diet of bitter chocolate covered broccoli would probably result in a weight loss. You read it here first.

Seriously, BILD?

[nospam007]

"and even the front page of Bild, Europe's largest daily newspaper. "

That's the yellowest yellow rag existing. If it's in there, it must be wrong.

Re:Bitter chocolate tastes bad?

[ArchieBunker]

You guys are already ruining craft beer by flooding it with cheap IPA and ever increasing levels or bitter. Its not a pissing contest to see who can stand the most bitter chocolate or beer.

best submission contest

[steak]

_roo wins best pun category with "and issued press releases to feed the ever-hungry pool of nutrition journalists."

Scalzi's Schadenfreude Pie

[billstewart]

There's really only one thing to eat while reading a story like this (other than Moar Chocolate, for Research Purposes.) It's Scalzi's Schadenfreude Pie. Dark, bitter, sticky, chocolately.