Why Standard Deviation Should Be Retired From Scientific Use

An anonymous reader writes "Statistician and author Nassim Taleb has a suggestion for scientific researchers: stop trying to use standard deviations in your work. He says it's misunderstood more often than not, and also not the best tool for its purpose. Taleb thinks researchers should use mean deviation instead. 'It is all due to a historical accident: in 1893, the great Karl Pearson introduced the term "standard deviation" for what had been known as "root mean square error." The confusion started then: people thought it meant mean deviation. The idea stuck: every time a newspaper has attempted to clarify the concept of market "volatility", it defined it verbally as mean deviation yet produced the numerical measure of the (higher) standard deviation. But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of commerce and the Federal Reserve partaking of the conflation, even regulators in statements on market volatility. What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life.'"

Comments

So you want to retire a statistical term...

[Anonymous Coward]

...because people use it incorrectly in economics? Get bent. The standard deviation is a useful tool for statistical analysis of large populations.

Re:So you want to retire a statistical term...

[Fouquet]

+1 this. The problem here is the author's impression that "social scientists" and economists are scientists. The groups that he excludes in the first paragraph (physicists) are scientists. Anyone attempting to implement a statistical model designed for a large (and Gaussian) data set on a small number of data points (as the article's example does) should expect to get an answer that is at best marginal. Any scientists who ever received even the most basic of statistics and/or data analysis training knows this. Understand the problem first, then take enough data points, then carry out your statistical analysis & formulate conclusions.

Re:So you want to retire a statistical term...

[mysidia]

the mean *absolute* deviation, rather than the square root of the mean *squared* deviation (the standard deviation).

The mean absolute deviation is a simpler measure of variability. However....

The algebraic manipulation of the standard deviation is simpler; the absolute deviation is more difficult to deal with.

Further, when drawing a number of samples from a large population --- the standard deviation of their mean deviations is substantially higher than the standard deviations of their individual standard deviations; that is to say, the standard deviation of a sample provides an estimate that is more in-line with the whole.

That is to say.... there are cases where the Standard Deviation may be better, AND, much of statistics is using standard deviation as its basis.

Fisher, R. 1920 Monthly Notes of the Royal Astronomical Society, 80, 758-770:

the quality of any statistic could be judged in terms of three characteristics. The statistic, and the population parameter that it represents, should be consistent , The statistic should be sufficient, and the statistic should be efficient -- e.g. the smallest probable error as an estimate of the population. Both the standard deviation and mean deviation met the first two criteria (to the same extent); however, in meeting the third criterion -- the standard deviation proves superior.

Re:So you want to retire a statistical term...

[Fouquet]

That actually was part of my point. In my day job (and night job and weekend job, and, oh god I need a vacation) I'm an astrophysicist. I have more data sets that I can recall, and the number of problems for which I'm confident that the errors are Gaussian is at most 2 or 3. We're finally in an era where computational power facilitates forward modeling & Bayesian techniques that can provide good estimates of true uncertainties. But I (and many of my colleagues) barely understand how they work. Any expectation that most researchers are willing to invest the time to understand anything beyond Gaussian statistics is unrealistic.

Basic Statistics

[TechyImmigrant]

The meaning of standard deviation is something you learn on a basic statistics course.

We don't ask biochemists to change their terms because the electron transport chain is complicated.
We don't ask cryptographers to change their terms because the difference between extra entropy and multiplicative prediction resistance is not obvious.

We should not ask statisticians to change their terms because people are too stupid to understand them.

Re:Basic Statistics

[Mr D from 63]

We should not ask statisticians to change their terms because people are too stupid to understand them.

But doesn't that give an unfair advantage to statisticians? You have to give everyone a chance!

Re:Basic Statistics

[gninnor]

Then it would be the same as pi, and that would just be silly.

Re:Basic Statistics

[stoborrobots]

... think of the standard deviation as the "average distance from the average" ... That is simple, neat, and easy to remember... It is also wrong.

In fact, it is wrong in exactly the way that TFA suggests: you're describing the mean deviation...

Re:Basic Statistics

[camperdave]

The phrase "orbital process" means entirely different things to brain surgeons and rocket scientists.

Re:Basic Statistics

[Ford Prefect]

Nuclear Resonance Imaging (NMR) was changed because people were afraid of word Nuclear despite it describing the process, unlike its replacement term.

Also, if you arrived at a hospital saying you were there for an NMR, you might have received something other than what you were expecting.

Issues

[Edward Kmett]

On the other hand, you also need to use 2-pass algorithms to compute Mean Absolute Deviation, whereas STD can be easily calculated in one pass. And you still need standard deviation as it relates directly to the second moment about the mean.

Also, annoyingly, Median Absolute Deviation competes for the MAD name and is more robust against outliers.

That's not the problem.

[khasim]

The problem is that people think they understand statistics when all they know is how to enter numbers into a program to generate "statistics".

They mistake the tools-used-to-make-the-model for reality. Whether intentionally or not.

Re:That's not the problem.

[Deadstick]

Three characterizations of statistics, in ascending order of accuracy:

1. There are lies, damned lies, and statistics.

2. Figures don't lie, but liars figure.

3. Statistics is like dynamite. Use it properly, and you can move mountains. Use it improperly, and the mountain comes down on you.

Re:That's not the problem.

[JoeMerchant]

The problem is that peoples' attention spans are rapidly approaching that of a water-flea.

Up until the past 50 or so years, people who learned about Standard Deviation would do so in environments with far less stimulation and distraction. Their lives weren't so filled with extra-curricular activities and entertainments that they never sat for a moment from waking until sleep without some form of stimulus based pastime. When they "understood" the concept, there was time for it to ruminate and gel into a meaningful set of connections with how it is calculated and commonly applied. Today, if you can guess the right answer from a set of 4 choices often enough, you are certified expert and given a high level degree in the subject.

Not bashing modern life, it's great, but it isn't making many "great thinkers" in the mold of the 19th century mathematicians. We do more, with less understanding of how, or why.

Re:That's not the problem.

[TsuruchiBrian]

I think it's also true that a larger percentage of people are going to university, so the average "intelligence" of people in university in terms of natural ability is probably lower now than when it was just the very best students attending.

Most of the mediocre students today would have simply not gone to university in the past. I think the same principle holds when it comes to things like blogs. The fact that public discourse can sometimes make it seem as if people are getting dumber, when it is really just that more and more people know how to read and write and can now even be published, whereas in the past, there was a higher cost to publishing, and you were more likely to have something important to say before being willing to incur that cost.

Standard Deviation is Important

[njnnja]

Standard Deviation is the square root of the second moment about the mean, an important fundamental concept to probability distributions. Looking at moments of probability distributions gives us lots of tools that have been developed over the years and in many cases we can apply closed form solutions with reasonably lenient assumptions. Then we apply the square root in order to put it in the same units as the original list of observations and get some of the heuristic advantages that he attributes to the mean absolute deviation.

But it is a balance, and any data set should be looked at from multiple angles, with multiple summary statistics. To say MAD is better that standard deviation is a reasonable point (with which I would disagree), but to say we should stop using standard deviation (the point made in TFA) is totally incorrect.

Re:response

[flibbajobber]

First!

... to within 0.5 standard deviations.

Actually, the more posts this story attracts, the more accurate your statement is, and the fewer standard deviations you are away from true first. Response times not being distributed in a Gaussian curve perhaps complicates things.

Standard deviation BAD, but mean GOOD?

[PacoSuarez]

Perhaps non-mathematicians don't have a problem with this, but it rubs me the wrong way.

What makes the mean an interesting quantity is that it is the constant that best approximates the data, where the measure of goodness of the approximation is precisely the way I like it: As the sum of the squares of the differences.

I understand that not everybody is an "L2" kind of guy, like I am. "L1" people prefer to measure the distance between things as the sum of the absolute values of the differences. But in that case, what makes the mean important? The constant that minimizes the sum of absolute values of the differences is the median, not the mean.

So you either use mean and standard deviation, or you use median and mean absolute deviation. But this notion of measuring mean absolute deviation from the mean is strange.

Anyway, his proposal is preposterous: I use the standard deviation daily and I don't care if others lack the sophistication to understand what it means.

I hate averages

[tthomas48]

I also think averages should go away. Most people think they are being reported the median (the number in the middle) when people tell them the average. It's great for real estate agents, and people trying to advocate for tax reform, but the numbers are not what people think they are.

Re:The big picture

[boristhespider]

I often change CSensiblyNamedClassThatDescribesItsFunctionWell to bTrue throughout the code for precisely this reason and no-one ever appreciates it :(

Re:The big picture

[FriendlyStatistician]

Hi, I'm a statistician.

It's not so simple to just say "ok, we're going to use the Mean Absolute Deviation from now on." The use of standard deviation is not quite the historical accident that Taleb makes it out to be--there are good reasons for using it. Because it is a one-to-one function of the second central moment (variance), it inherits a bunch of nice properties that the mean absolute deviation does not. There is not a one-to-one correspondence between variance and mean absolute deviation.

Taleb is correct that the mean absolute deviation is easier to explain to people, but this is not just a matter of changing units of measure (where there is a one-to-one correspondence) or changing function and variable names in code (where there is again a one-to-one correspondence). Standard deviation and mean absolute deviation have different theoretical properties. These differences have led most statisticians over the last hundred years to conclude that the standard deviation is a better measure of variability, even though it is harder to explain.

Re:The big picture

[mythosaz]

I would have said "18 half gallon pottles to the quarter-barrel firkin."
Wolfram Alpha says 15.75 pottles to the firkin, but that's because of US/UK gallon conversions, I reckon.

352 nails in a chain - which was interesting to me, in that Google includes those units in its calculator.

I now know more about pottles, firkins, nails and chains that I did when I woke up. I shudder to think about what got pushed out of my old head to make way for this new minutia.

Re:So you want to decertify a management degree...

[rwa2]

...and besides... JUST THINK of all the rigorous Lean Management courses that will have to re-certify all of their "Six-Sigma Black Belts" to some kind of "Half-Dozen of the Other" degrees!

PANDEMONIUM!!!

Data Science

[Lamps]

Data science is a field that combines machine learning and statistics to derive meaning from data. Data scientists should be reasonably well-versed in classical stats, but the data sets they deal with are often huge, ill-defined, and not amenable to analysis using classical methods. To deal with such challenges, data science recruits a healthy combination of certain areas of comp-sci (databases, machine learning, NLP, AI), statistical methods, and, quite often, improvisation.

Strange that there are so many people on here that are unfamiliar with data science.