Struggle With Statistics? Your 'Fixed Mindset' Might Be To Blame (arstechnica.com)

A new study in Frontiers in Psychology examined why people struggle so much to solve statistical problems, particularly why we show a marked preference for complicated solutions over simpler, more intuitive ones. Chalk it up to our resistance to change. From a report: The study concluded that fixed mindsets are to blame: we tend to stick with the familiar methods we learned in school, blinding us to the existence of a simpler solution. Roughly 96 percent of the general population struggles with solving problems relating to statistics and probability. Yet being a well-informed citizen in the 21st century requires us to be able to engage competently with these kinds of tasks, even if we don't encounter them in a professional setting. "As soon as you pick up a newspaper, you're confronted with so many numbers and statistics that you need to interpret correctly," says co-author Patrick Weber, a graduate student in math education at the University of Regensburg in Germany. Most of us fall far short of the mark.

Part of the problem is the counterintuitive way in which such problems are typically presented. Meadows presented his evidence in the so-called "natural frequency format" (for example, 1 in 10 people), rather than in terms of a percentage (10 percent of the population). That was a smart decision, since 1-in-10 a more intuitive, jury-friendly approach. Recent studies have shown that performance rates on many statistical tasks increased from four percent to 24 percent when the problems were presented using the natural frequency format.

Comments

Statistics

[ledow]

I'm a mathematician.

The second someone digs out statistics, I can always pick 20 holes with their methodology used, presentation of, choice of, or analysis of their numbers. Usually, I could make things come out "oppositely" with only minor tweaking and use of the other statistics from the same dataset that they discarded out of hand, and usually I could provide much better justification for the numbers I used than the ones they did.

People use statistics to back up their claims. That's it. And if you go looking hard enough and present statistics to do that, you can ignore all the stuff that doesn't match your claims. It's really easy to do.

And because nobody understand statistics (I would posit this category even includes statisticians!), you can get away with it.

I like to shout at shampoo adverts when they say "Women agree*" where the * leads you to a footnote saying they tested 19 women and 67% of those agreed... so you're telling me that, actually, worldwide, 12.7 women agreed... What the hell kind of selection criteria did you use to get that, and what use is that if you don't specify that they were random women from the street in a controlled trial rather than, say, the people who work in the office?

The old saying is right - there are lies, damn lies and statistics. If someone quotes statistics are you, assume it's a lie. It almost always is. Even when it's not, it's merely the portion of the truth that can be spun positively if you don't mind looking through an n-dimensional kaleidoscope at the data.

And what are they trying to do by telling you this? They're trying to tell you "Hey, look, you're stupid and have no idea what's going on and actually everyone else is really onboard". Statistics are used as the worst kind of "peer pressure" - if they are trying to convince you of something, rather than inform you.

Statistics can be incredibly useful, very revealing, and can lead to a better understanding. If you get them from a professional. Who'll then tell you what the statistics *mean* and whether or not they have caveats.

If you get them from shampoo ads and random junk on the Internet, they are no better than any other "fact" spewed at you in such a manner... wrong.

(Interesting fact: There's a TV program called QI, in which all kinds of "you'll never believe" stuff is presented in what's supposed to be a highly intellectual quiz show. QI facts are heavily researched, almost always counter-intuitive or contrary to what everyone has been told, and they spend years with some of the cleverest people from the top universities doing research for each series. And in one episode they reveal that the portion of facts that they themselves got wrong, or which have changed since, was over 50%).

If someone quotes statistics at you, don't just nod and go "Oh really?" because you're then likely to repeat that statistic without every checking it. The correct response is to think "What's he trying to convince me of?". Because you don't use statistics for anything else.

And unless you understand, truly understand, statistics, you know that any kind of amateur data-gathering or analysis by even the most well-intentioned people is a bottomless pit of potential failure.

Hell, to be honest, 99% of the time, I can't even work out the right answer for some statistics so I make it up and say 99%.