Slashdot Mirror
Busting the MythBusters' Yawn Experiment
markmcb writes "Most everyone knows and loves the MythBusters, two guys who attempt to set the story straight on things people just take for granted. Well, maybe everyone except Brandon Hansen, who has offered them a taste of their own medicine as he busts the MythBusters' improper use of statistics in their experiment to determine whether yawning is contagious. While the article maintains that the contagion of yawns is still a possibility, Hansen is clearly giving the MythBusters no credit for proving such a claim, 'not with a correlation coefficient of .045835.'"
If they find out, they may very well make an announcement on the show. Wouldn't be the first time.
it always seems to me that their conclusions are specious. I can't think of any specific episodes right now but they over simplify the data, build elaborate setups that are prone to error, and use inadequate controls.
not to mention that they always try to prove stupid crap like "a rolling stone gathers no moss". I'm waiting for them to try "the grass is always greener on the other side", or "it takes one to know one".
I dunno, the fact that he's willing to state the correlation coefficient so precisely makes me leary of his own statistical expertise.
Please, for the love of God, no more car analogies.
In almost every episode they do something that invalidates their own findings.
Sometimes they don't things more than once (even when required), other times they don't adequately recreate the conditions of the "myth."
The show is entertaining as hell, and sometimes they do conclusively prove things.
Latewire
Science and entertainment do not play well together, it is mostly true because science requires real thought and watching TV basicly does not. If you attempt to put real science on TV today you will watch the other 6.9 million TV stations each gain 1 more viewer while you get a dust bowl rolling through. Maybe it's time we started to realize what the mass public want are crappy reality shows, cooking and some bullshit made to look information but that is infact 75% CGI or "docudrama".
The above is why I wouldn't trust Mythbusters as far as I could throw them. The entire show screams entertainment rather than Science. Unfortunaely I can't find the name of a program that aired in the UK about 6 months ago. It took a team of 4 people to a deserted island and each week they had a task to complete each, they were only allowed to use what was on the island and what was given to them each week (as well as a tool set because, well no tools = screwed). They had to do things like make fireworks, record a song and various other "minor" things which required them to render down various things to achieve the chemicals they needed to complete each task. What they did and what it resulted in was very clearly labeled, having real science explained behind it.
Saddly as I recall it basicly got replaced with some crappy school based soap opera where the kids say "innit" and the teachers fuck anything with two legs (including the kids as the current trailer at least implies). So after this long rant, I guess we just give up on science and go back to the discoery channel, maybe we can catch the 3 minutes of it that isn't Nazis or some form of sport!
I like muppets.
TFA's conclusion is correct but their methods are wrong. For these kind of data, correlations aren't the appropriate test; they should have used a chi-square distribution test. Using TFA's assumptions -- total sample size of 50, 4 yawners out of 16 not seeded, 10 yawners out of 34 seeded -- the chi-square value is .10, which pretty strongly misses the critical value of 3.84 for significance.
Not that it matters anyway, but it's pretty funny to read an article debunking statistics that employs inappropriate statistics itself...
You do not report five significant figures derived from data with only two.
If Slashdot were chemistry it would look like this:Cadaverine
Not only was MythBusters embarassingly statistics-free, but the "busting" was done using a wholly inappropriate statistical technique. Hansen used a correlation-based test, which assumes that the data follows a Normal distribution (which a bunch of 1s and 0s do not).
There is a very well-known test, the chi-square test, that deals with exactly this case. (Given the small sample sizes, the Fisher exact test may give better results.) Someone should point Hansen to the Wikipedia page on the topic.
For example, if there are 16 non-primed people, with 4 yawning and 12 not (for 25%), and there are 34 primed people, with 10 yawning and 24 not (for 29%), the chi square test gives a p value of 0.74.
The values Hansen supposes are significant 4,12 and 12,24 are not: p = 0.29.
You have to go all the way to 4,12 and 17,19 (i.e. 47% on a sample of 36) to get significance.
MythBusters was wrong to conclude that their results were significant, but Hansen was equally wrong to conclude that he had shown that Mythbusters was wrong.
Engineering is the art of compromise.
Look, to spare everyone the continued arguing over which statistical test to use at what probability level and the lack of proper control groups, let me say that MythBusters has never claimed to be a science show like Mr. Wizard. The guys are special effects designers for crying out loud! They are good at what they do, and while their scientific methodology and statistics may be a bit wonky at times, there are some experiments I've seen in peer-reviewed journals that aren't much better. Science education in the United States gets worse all the time, and if these guys can inject some life and curiosity into the current generation to get them interested in science, I applaud the effort.
GetOuttaMySpace - The Anti-Social Network
I've been reading through the comments and I'm fairly alarmed by how many people think Mythbusters isn't worthwhile based solely on scientific merit.
Look, the show never said it was teaching people about science. Adam and Jaime themselves have said many times they're more entertainment than science. They're special effects people by trade, not scientists. They build things and blow shit up. It's what they enjoy doing. You can even see it on Jaime's face when they're doing myths that don't involve blowing things up (e.g. Adam building a wind tunnel for the penny drop myth).
When the show first started, there wasn't even mention of science. They looked at urban legends such as rocket car and getting airborne in a lawnchair. The show was about the stories themselves, not the methods. Only in about season 2 or so did they start including things like "controls" and "variables" (probably by Discovery's request), but they never lost sight of the fact that they're a TV show, and television (by and large) is meant to entertain.
But that leads to an interesting question: even if they DID follow proper scientific method, how do you even apply that to some of the myths they examine? For example, they did a myth where a hillbilly chased a raccoon into a sewer pipe, decided to throw gas down it, attempted to fill the thing with fire to kill the raccoon and was purportedly "shot out". How on earth do you test that scientifically? Nowhere in the myth does it says how big the pipe was, how much gasoline was used, etc. Nowhere does it mention if he was stuck (which is important, as they found the man could only be shot out of he was wrapped in a sabot). All they have is a fun story to go off of.
If nothing else, Mythbusters gets people interested in the process of examining life, not teaching how to use proper scientific method. If their only accomplishment is making people critically question things that are usually taken at face value, they'll have succeeded in my mind.