How a Scientist Fooled Millions With Bizarre Chocolate Diet Claims

__roo writes: Did you know chocolate helps you lose weight? You can read all about this great news for chocoholics in the Daily Star, Daily Express, Irish Examiner, and TV shows in Texas and Australia, and even the front page of Bild, Europe's largest daily newspaper. The problem is that it's not true. A researcher who previously worked with Science to do a sting operation on fee-charging open access journals ran a real—but obviously flawed—study rigged to generate false positives, paid €600 to get it published in a fee-charging open access journal, set up a website for a fake institute, and issued press releases to feed the ever-hungry pool of nutrition journalists. The doctor who ran the trial had the idea to use chocolate, because it's a favorite of the "whole food" fanatics. "Bitter chocolate tastes bad, therefore it must be good for you. It's like a religion."

Comments

Re:Heh.

[Penguinisto]

True, though it sadly proves P.T. Barnum's maxim, and says more about a gullible public, the lack of peer review in the field of nutrition (and worse, the sheer incompetence of so-called 'nutrition journalists' and 'specialists'), than it does about a science journal's shady/sloppy practices.

Long story short, it exposes a hell of a lot more than just what the scientist initially wanted exposed.

Maybe someone could do and publish a sociology study from it?
(/me ducks and runs like hell...)

This was done by a journalist, not a scientist!

[metageek]

The OP is wrong on this, the guy is a JOURNALIST not a scientist. (He may have a science degree but he acts as a journalist).

If a scientist had done this they would be losing their job any minute. Any of the following would be enough to disgrace a practicing scientist (I am one):

1- carrying out research on human subjects without approval of the study by an independent review board
2- asking people to undergo a study that he knew before hand that was not beneficial to the subjects, in fact could likely be the opposite (this would mean he'd never get approval of the study)
3- lying to people about his affiliation and credentials in the paper

There are a lot of other problems with this "study", but it surely was not done by a scientist.

What it does reveal is that people cannot rely on popular press stories about science as journalist pay no attention to the important details of publications and fall for any hype. Unlike what the guy says, journalist can never be "peer" reviewers of any science... their role is different and yet they are not doing it properly.

Re:This was done by a journalist, not a scientist!

[Anonymous Coward]

0) Oh, he's not a scientist? He's doing fucking SCIENCE here. He had a hypothesis, conducted an experiment, published the fucking result. He created new information about the state of scientific journalism. But maybe you're confusing academics with science.

1) Get real. There was no potential for harm. Your "review board" requirement raises the bar high enough to render unethical every science fair project ever done.

2) Exactly like, for example, every control group, ever?

3) That was the basis. Of. The. Study.

Re:This was done by a journalist, not a scientist!

[captnjohnny1618]

I think you misread the post (and/or the article).

Yes, the author of the story is a science journalist. The "study" was intentionally bad to demonstrate the current sad state of "science journalism" (with an easy target like nutrition) where few of these journalists actually take the time to actually read the article, much less actually fact check on a critical, challenging or even marginally scientific level.

So your final conclusion is exactly what he wanted to show: you can't blindly trust popsci journalism... but your methods for arriving there are a little roundabout.

Re:This was done by a journalist, not a scientist!

[AthanasiusKircher]

You're absolutely correct that this story is mostly about how we should distrust science journalism, and the author of the "study" makes that clear.

However...

If a scientist had done this they would be losing their job any minute. Any of the following would be enough to disgrace a practicing scientist (I am one):

Yes, all of that is true. But I think you're exaggerating quite a bit about the impact any of this would have had on whether a study like this could have been done by a reputable scientist (it certainly could have been), or whether it would have been published if that scientist had appropriate credentials (as noted in the article, there are plenty of places to publish bad research with little vetting).

2- asking people to undergo a study that he knew before hand that was not beneficial to the subjects, in fact could likely be the opposite (this would mean he'd never get approval of the study)

The details of methodology in this study might have been changed a bit to get it approved by a review board, and there's certainly nothing about the basic idea of this study that suggests it would NECESSARILY (or even likely) have significant detrimental effects for the subjects. And there is certainly at least a possibility of beneficial effects.

I suggest you have a look at some of the research articles linked even in an article of Wikipedia on the subject. There have been dozens, and probably hundreds of studies that have tried to measure health effects of chocolate -- many of them have involved people eating small amounts of chocolate and observing effects, just as the study here did.

So, the idea that an actual scientist, with the appropriate amount of time, could "never get approval of the study" is just ridiculous. As I said, there would probably be some more detailed methodological justification and tweaks, but lots of nutrition studies like this happen all the time.

Unlike what the guy says, journalist can never be "peer" reviewers of any science... their role is different and yet they are not doing it properly.

Yes, journalists have their own jobs to do, and there are certainly flaws in the system.

But there are flaws in the science system too, which makes the job of journalists (and scientists trying to look at research out there and evaluate it) much harder. "P hacking" is not just something made up by this journalist -- it's a real thing, and it's a real problem. Yes, many reputable journals have tried to make review procedures better to avoid various statistical problems, but they often don't really fight them head-on (with a few notable exceptions of brave editors or boards). There are a LOT of problems with common statistical procedures followed by researchers -- even those who have proper credentials and have gotten independent reviews.

Anyhow, most of your criticism boils down to "this guy wasn't credentialed to do what he did." That's great, but it doesn't address the larger flaws here -- not only in journalistic reporting, but in some aspects of scientific methodology. This study was clearly not rigorous, but I've seen worse studies published in reputable journals.

Re:This was done by a journalist, not a scientist!

[taustin]

If a scientist had done this they would be losing their job any minute.

Given that we're talking about the "nutrition" industry, if a scientist had done this, it would likely be his job. Specifically.

That's the point.

Re:This was done by a journalist, not a scientist!

[narcc]

It wasn't a study, it was a publicity stunt.

Bitter chocolate tastes bad?

[HornWumpus]

WTF?

Go eat your milk chocolate, fools. I'll take the 80% Madagascar coco bad bitter stuff, you can have the Cadbury/Hershies.

Re:Bitter chocolate tastes bad?

[ArchieBunker]

You guys are already ruining craft beer by flooding it with cheap IPA and ever increasing levels or bitter. Its not a pissing contest to see who can stand the most bitter chocolate or beer.

Re:Scientists are generally trusted

[CraigGlaser]

Anyone who trusts scientists has no idea how science works. You don't trust the person or title, you trust the chain of independent verification of the data. This is a critical thinking issue.

Re:Scientists are generally trusted

[Anonymous Coward]

I disagree.

Scientists are widely distrusted.

Except when they tell us what we want to hear.

This works 100%

[bogaboga]

Forget all those so called diet plans.. The only thing I know that works is actually eating less. Many of my friends will not agree, but I wonder how one can gain weight by putting less calories in their system.

Re:Scientists are generally trusted

[Penguinisto]

Here's the trick: You and I know this, but the average schlub out there does not.

That distinction is kinda vital, and it's what I think GP was driving at.

Re:Heh.

[Opportunist]

Mostly it exposes that people love to believe stories they like. And of course journalists love to publish stories that their readers like.

I just don't think that qualifies as news, though.

Re:Scientists are generally trusted

[Anonymous Coward]

More to the point, it's impossible to independently (& personally) verify the data and claims of everything that you would like verified. There's not enough time in the world.

Re:Scientists are generally trusted

[ShanghaiBill]

Here's the trick: You and I know this, but the average schlub out there does not.

Science journalists should be better than "average schlubs". They owe it to their readers to do at least a few minutes of fact checking before publishing.

Re:Logic

[tomhath]

For most people, a diet of bitter chocolate covered broccoli would probably result in a weight loss. You read it here first.

Seriously, BILD?

[nospam007]

"and even the front page of Bild, Europe's largest daily newspaper. "

That's the yellowest yellow rag existing. If it's in there, it must be wrong.

Re:This could never happen with global warming...

[Holi]

Peer review hasn't killed AGW so whats your problem with it? Of the scientists who have expressed an opinion on AGW 97.2% endorse the consensus. only .7% reject it.

Now I understand science is not a popularity contest, but what are your credentials compared to theirs? If you are not willing to accept peer reviewed science then why the hell are you bothering to post here.

Re:Heh.

[michelcolman]

The big problem is that everyone will now remember the fake study and continue to believe it, because the rectification doesn't get nearly as much coverage. People are still refusing to vaccinate children because they're afraid of autism even though the author of that study actually confessed having made the whole thing up.

Re: Heh.

[Anonymous Coward]

It's not that they aren't aware that the author confessed, it's that they love to hear about "secrets that 'they' don't want you to know about" because there's a giant global conspiracy by this 'they' to keep poor people poor and make people sick for profit.

I don't personally know an anti vaxer, but I'd wager that most of them who heard about the author confessing to a fraud believe that he was coerced by 'they' to admit it, and it really does cause autism after all.

It's really the same reason why Greenpeace thinks Patrick Moore was bribed by big oil just because he fell out of line with their radical views. Or why anybody who says that GMO food isn't actually bad and is indeed a good thing gets accused of shilling for Monsanto, even though they have nothing to do with Monsanto or the food industry.

Re: Scientists are generally trusted

[houghi]

They are journalists not scientists. the reader is the product. Except for the subject there is no difference between a scientific or e.g. a sports or gossip journalist.

Re:Scientists are generally trusted

[lgw]

More to the point, it's impossible to independently (& personally) verify the data and claims of everything that you would like verified. There's not enough time in the world.

Very true. The rational man realizes this, and doesn't hold strong political opinions on the rest of it. We're all going to be ignorant of most science in the modern world - the time has long passed when the educated man could know all of the scientific knowledge there was. It's important to therefore set arrogance aside, and not try to tell others they're idiots, or force your uneducated opinion on others by law, unless you actually care enough to do the diligence first.

Far too many people mistake fashion for education. If you're going to call others fools for trying to stop the teaching of "evolution" in schools, call them fools because you took the time to understand the science, the counter-arguments, and why a smart, ration person could somehow not believe in evolution. Until you understand the other side, and why it's wrong, stay out of the argument. For the evolution case: if you had a solid biology class, this takes just a few days of reading the talk.origins site. It's not an undue burden, and otherwise arrogance about your uninformed opinion is just idiocy.

For newer fields like the climate change debate, it will take longer to dig up the details, as there isn't a handy website that collects all the pro and con arguments. For climate change, can read through the pro and con sites and understand where they're coming from, understand the Vostok ice core data for perspective, spend time pondering the satellite temperature data, and so on.

For any such issue, treat both sides as intelligent people who are in earnest in their beliefs and not trolling, and read enough to understand how this can be true. When you understand how intelligent people can disagree on the issue, and see where both sides are coming from, then you can act out of knowledge instead of arrogance, and stop polluting the debate with idiocy. If your only basis for argument is "everyone knows the smart people believe X, and the losers believe not-X", well, that's fashion, not knowledge. This pretty much applies to anything being debated politically, BTW, not just the science stuff.

Re:Heh.

[RespekMyAthorati]

Andrew Wakefield was deliberately fraudulent, and that is why the paper was retracted and his medical license revoked.