Slashdot Mirror

← Back to Stories (view on slashdot.org)

Struggle With Statistics? Your 'Fixed Mindset' Might Be To Blame (arstechnica.com)

Posted by msmash on from the closer-look dept.

A new study in Frontiers in Psychology examined why people struggle so much to solve statistical problems, particularly why we show a marked preference for complicated solutions over simpler, more intuitive ones. Chalk it up to our resistance to change. From a report: The study concluded that fixed mindsets are to blame: we tend to stick with the familiar methods we learned in school, blinding us to the existence of a simpler solution. Roughly 96 percent of the general population struggles with solving problems relating to statistics and probability. Yet being a well-informed citizen in the 21st century requires us to be able to engage competently with these kinds of tasks, even if we don't encounter them in a professional setting. "As soon as you pick up a newspaper, you're confronted with so many numbers and statistics that you need to interpret correctly," says co-author Patrick Weber, a graduate student in math education at the University of Regensburg in Germany. Most of us fall far short of the mark.

Part of the problem is the counterintuitive way in which such problems are typically presented. Meadows presented his evidence in the so-called "natural frequency format" (for example, 1 in 10 people), rather than in terms of a percentage (10 percent of the population). That was a smart decision, since 1-in-10 a more intuitive, jury-friendly approach. Recent studies have shown that performance rates on many statistical tasks increased from four percent to 24 percent when the problems were presented using the natural frequency format.

11 of 151 comments (clear)

  1. Huh? by dtmos · · Score: 4, Insightful

    Part of the problem is the counterintuitive way in which such problems are typically presented. Meadows presented his evidence in the so-called "natural frequency format" (for example, 1 in 10 people), rather than in terms of a percentage (10 percent of the population). That was a smart decision, since 1-in-10 a more intuitive, jury-friendly approach. Recent studies have shown that performance rates on many statistical tasks increased from four percent to 24 percent when the problems were presented using the natural frequency format.

    I've heard this argument before, and I just don't get it. "Percent" means per hundred, as the word is derived from the Latin "per centum," literally, "per hundred." It's a natural frequency format, just as much as saying "1 in 10 people." It's saying "10 per 100" people. What's so confusing?!?

    1. Re:Huh? by umafuckit · · Score: 4, Insightful

      I've heard this argument before, and I just don't get it. "Percent" means per hundred, as the word is derived from the Latin "per centum," literally, "per hundred." It's a natural frequency format, just as much as saying "1 in 10 people." It's saying "10 per 100" people. What's so confusing?!?

      It's not confusing, it's just that many people don't do the conversion in their heads. Further, presenting the natural frequency is more useful for small percentages: e.g. 1 in 4,000 is definitely easier to digest than 0.025%

      --
      soylentnews.org
    2. Re:Huh? by hey! · · Score: 5, Insightful

      Nobody is saying that 1-in-10 is mathematically different than 10%. It is heuristically more helpful to people with less mathematical competence.

      When you're good at math, you naturally line up all the "givens" in a problem. You go over each one an interpret what it means, "So that means if I had 100 people, ten of them would prefer vanilla to chocolate..." It's like a wood carver examining a block of wood to find a good place to start cutting. You do this so automatically it seems intuitive to you, but it's actually the result of long training and practice.

      To people who aren't as well trained in math, the "givens" look like an impenetrable wall of text, because the individual bricks in the wall don't instantly convey useful information to them. Well, of course they don't; you have to *think* about them, and the less accustomed you are to numbers, the more work it is for you for less certainty of reward.

      But if you put a picture into peoples' heads, you give them an immediate handhold on the problem. It's not difficult for a mathematically fluent person to make his own handhold, but it is a stumbling block for a lot of people.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    3. Re:Huh? by Anonymous Coward · · Score: 3, Insightful

      "Percent" means per hundred

      A lot of people don't even get that. The problem is innumeracy compounded by poor vocabulary.

    4. Re:Huh? by Anonymous Coward · · Score: 3, Insightful

      Because most people, especially now, have little physical grasp of numbers, beyond what they can count on their hands. Expressing numbers in ratios, rather than percentages, becomes much more meaningful when the percentage gets smaller and the numbers become indivisible by 2. 1 in 700 is much easier to grasp than 0.142%. To paraphrase George Carlin," think of how dumb the average person is, and remember that half of them are dumber than that."

    5. Re:Huh? by clovis · · Score: 3, Insightful

      It's not confusing, it's just that many people don't do the conversion in their heads. Further, presenting the natural frequency is more useful for small percentages: e.g. 1 in 4,000 is definitely easier to digest than 0.025%

      1. What "conversion"?

      2. What makes "1 in 4,000" easier to digest than "0.025%"?

      I suppose it's because .025% is a poor choice of a way to express a value. Percent means parts of a hundred, and they make more sense when the values is between 1 and 100. When you're using percents that are far less than 1%, it is hard to get an intuitive feel for the relative size of whatever is being measured. Sure it's easy enough to do the conversion, but why not express it as a number that is scaled to the measurement in the first place.

      It's sort of like when someone asked for the height of my son. I could say he is 0.0011 mile tall, and although you may have a good feeling for how long a mile is, you have no idea whether he is average, short, or tall until you've done the conversion.

    6. Re:Huh? by gotan · · Score: 4, Informative

      That's not it.

      It's easier to grasp what to do to those numbers. Presented with percentages it's often hard to see what mathematical operations are necessary to arrive at the desired answer in bayesian statistics problems.

      E.g.
      A medicinal test for disease X gives false positives in 0.1% of cases. It gives a false negative in 1% of the cases (i.e. correct positive in 99% of the cases). The disease afflicts 0.01%.
      Of those tested positive, how many have disease X.

      Of course one now could employ the statistics toolbox to solve that problem. OTOH one could compare the 10 in 10,000 false positives (with a slight error since only 9,999 are without disease), to the 1 in 10,000 diseased (noticing that the false negatives have negligible impact for the question at hand and we can work with 100% correct positives as well as 99% if we want an estimate).

      So now we need to compare only small numbers, 10 false positives to 1 diseased positive or 1 in 11 which is about 9%.
      (the correct result without the approximations is 10 in 111 or 9,009...%).

      Also note the easy expansion of 1 in 1,000 to 10 in 10,000 to get to comparable numbers. It's not important to have an accurate image of those 10,000, what's of interest is to compare the 10 false positives to the 1 diseased.

      Such medicinal tests help a lot to find candidates that should undergo more sophisticated (and much more expensive) tests, to see if they really have X (it'll reduce the expensive tests by a factor of 1,000), but patients need to be informed even with a "positive" result it's still unlikely that they have X, but advisable to do the more sophisticated test. One might think that the test is pretty useless if it delivers 91% false positives when in fact it is pretty accurate, only the occurence of disease X is so rare.

      So such "frequencies" do not only help to get a (pretty) correct result without knowing any bayesian statistics tools, but also to understand how the information affects the result, and how the unintuitive (to someone not used to such statistics) result comes about.

      --
      "By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
  2. Intuitive And Jury Friendly by tsqr · · Score: 5, Funny

    Recent studies have shown that performance rates on many statistical tasks increased from four percent to 24 percent when the problems were presented using the natural frequency format.

    Hmm, let me make that more intuitive and jury friendly for everyone: recent studies have shown that performance rates on many statistical tasks increased from 1 in 25 to about 1 in 4 when the problems were presented using the natural frequency format.

  3. Even more jury-friendly ... by fahrbot-bot · · Score: 3, Funny

    ... since 1-in-10 a more intuitive, jury-friendly approach.

    Which would be 1-in-12.

    --
    It must have been something you assimilated. . . .
  4. Bound for failure by reanjr · · Score: 4, Insightful

    If you can't figure out 10% is 1-in-10, you have no hope of wading through the standard level of obfuscation added to any publication when discussing statistics.

  5. Statistics by ledow · · Score: 5, Interesting

    I'm a mathematician.

    The second someone digs out statistics, I can always pick 20 holes with their methodology used, presentation of, choice of, or analysis of their numbers. Usually, I could make things come out "oppositely" with only minor tweaking and use of the other statistics from the same dataset that they discarded out of hand, and usually I could provide much better justification for the numbers I used than the ones they did.

    People use statistics to back up their claims. That's it. And if you go looking hard enough and present statistics to do that, you can ignore all the stuff that doesn't match your claims. It's really easy to do.

    And because nobody understand statistics (I would posit this category even includes statisticians!), you can get away with it.

    I like to shout at shampoo adverts when they say "Women agree*" where the * leads you to a footnote saying they tested 19 women and 67% of those agreed... so you're telling me that, actually, worldwide, 12.7 women agreed... What the hell kind of selection criteria did you use to get that, and what use is that if you don't specify that they were random women from the street in a controlled trial rather than, say, the people who work in the office?

    The old saying is right - there are lies, damn lies and statistics. If someone quotes statistics are you, assume it's a lie. It almost always is. Even when it's not, it's merely the portion of the truth that can be spun positively if you don't mind looking through an n-dimensional kaleidoscope at the data.

    And what are they trying to do by telling you this? They're trying to tell you "Hey, look, you're stupid and have no idea what's going on and actually everyone else is really onboard". Statistics are used as the worst kind of "peer pressure" - if they are trying to convince you of something, rather than inform you.

    Statistics can be incredibly useful, very revealing, and can lead to a better understanding. If you get them from a professional. Who'll then tell you what the statistics *mean* and whether or not they have caveats.

    If you get them from shampoo ads and random junk on the Internet, they are no better than any other "fact" spewed at you in such a manner... wrong.

    (Interesting fact: There's a TV program called QI, in which all kinds of "you'll never believe" stuff is presented in what's supposed to be a highly intellectual quiz show. QI facts are heavily researched, almost always counter-intuitive or contrary to what everyone has been told, and they spend years with some of the cleverest people from the top universities doing research for each series. And in one episode they reveal that the portion of facts that they themselves got wrong, or which have changed since, was over 50%).

    If someone quotes statistics at you, don't just nod and go "Oh really?" because you're then likely to repeat that statistic without every checking it. The correct response is to think "What's he trying to convince me of?". Because you don't use statistics for anything else.

    And unless you understand, truly understand, statistics, you know that any kind of amateur data-gathering or analysis by even the most well-intentioned people is a bottomless pit of potential failure.

    Hell, to be honest, 99% of the time, I can't even work out the right answer for some statistics so I make it up and say 99%.