Slashdot Mirror
How a Scientist Fooled Millions With Bizarre Chocolate Diet Claims
__roo writes: Did you know chocolate helps you lose weight? You can read all about this great news for chocoholics in the Daily Star, Daily Express, Irish Examiner, and TV shows in Texas and Australia, and even the front page of Bild, Europe's largest daily newspaper. The problem is that it's not true. A researcher who previously worked with Science to do a sting operation on fee-charging open access journals ran a real—but obviously flawed—study rigged to generate false positives, paid €600 to get it published in a fee-charging open access journal, set up a website for a fake institute, and issued press releases to feed the ever-hungry pool of nutrition journalists. The doctor who ran the trial had the idea to use chocolate, because it's a favorite of the "whole food" fanatics. "Bitter chocolate tastes bad, therefore it must be good for you. It's like a religion."
Schadenfreude is the best freude.
i just don't eat it.
Anyone who trusts scientists has no idea how science works. You don't trust the person or title, you trust the chain of independent verification of the data. This is a critical thinking issue.
It was never science in the first place. He used known flaws in experimentation to prove that you can make the mainstream publish anything. Which was the point he was trying to make.
Forget all those so called diet plans.. The only thing I know that works is actually eating less. Many of my friends will not agree, but I wonder how one can gain weight by putting less calories in their system.
xkcd's "Significance" pithily explains p-hacking.
- David A. Wheeler (see my Secure Programming HOWTO)
0) Oh, he's not a scientist? He's doing fucking SCIENCE here. He had a hypothesis, conducted an experiment, published the fucking result. He created new information about the state of scientific journalism. But maybe you're confusing academics with science.
1) Get real. There was no potential for harm. Your "review board" requirement raises the bar high enough to render unethical every science fair project ever done.
2) Exactly like, for example, every control group, ever?
3) That was the basis. Of. The. Study.
I think you misread the post (and/or the article).
Yes, the author of the story is a science journalist. The "study" was intentionally bad to demonstrate the current sad state of "science journalism" (with an easy target like nutrition) where few of these journalists actually take the time to actually read the article, much less actually fact check on a critical, challenging or even marginally scientific level.
So your final conclusion is exactly what he wanted to show: you can't blindly trust popsci journalism... but your methods for arriving there are a little roundabout.
Here's the trick: You and I know this, but the average schlub out there does not.
That distinction is kinda vital, and it's what I think GP was driving at.
Quo usque tandem abutere, Nimbus, patientia nostra?
More to the point, it's impossible to independently (& personally) verify the data and claims of everything that you would like verified. There's not enough time in the world.
You're absolutely correct that this story is mostly about how we should distrust science journalism, and the author of the "study" makes that clear.
However...
If a scientist had done this they would be losing their job any minute. Any of the following would be enough to disgrace a practicing scientist (I am one):
Yes, all of that is true. But I think you're exaggerating quite a bit about the impact any of this would have had on whether a study like this could have been done by a reputable scientist (it certainly could have been), or whether it would have been published if that scientist had appropriate credentials (as noted in the article, there are plenty of places to publish bad research with little vetting).
2- asking people to undergo a study that he knew before hand that was not beneficial to the subjects, in fact could likely be the opposite (this would mean he'd never get approval of the study)
The details of methodology in this study might have been changed a bit to get it approved by a review board, and there's certainly nothing about the basic idea of this study that suggests it would NECESSARILY (or even likely) have significant detrimental effects for the subjects. And there is certainly at least a possibility of beneficial effects.
I suggest you have a look at some of the research articles linked even in an article of Wikipedia on the subject. There have been dozens, and probably hundreds of studies that have tried to measure health effects of chocolate -- many of them have involved people eating small amounts of chocolate and observing effects, just as the study here did.
So, the idea that an actual scientist, with the appropriate amount of time, could "never get approval of the study" is just ridiculous. As I said, there would probably be some more detailed methodological justification and tweaks, but lots of nutrition studies like this happen all the time.
Unlike what the guy says, journalist can never be "peer" reviewers of any science... their role is different and yet they are not doing it properly.
Yes, journalists have their own jobs to do, and there are certainly flaws in the system.
But there are flaws in the science system too, which makes the job of journalists (and scientists trying to look at research out there and evaluate it) much harder. "P hacking" is not just something made up by this journalist -- it's a real thing, and it's a real problem. Yes, many reputable journals have tried to make review procedures better to avoid various statistical problems, but they often don't really fight them head-on (with a few notable exceptions of brave editors or boards). There are a LOT of problems with common statistical procedures followed by researchers -- even those who have proper credentials and have gotten independent reviews.
Anyhow, most of your criticism boils down to "this guy wasn't credentialed to do what he did." That's great, but it doesn't address the larger flaws here -- not only in journalistic reporting, but in some aspects of scientific methodology. This study was clearly not rigorous, but I've seen worse studies published in reputable journals.
It wasn't a study, it was a publicity stunt.
Required reading for internet skeptics