Open Source Pioneer Michael Tiemann On the Myth of the Average

StewBeans writes: In a recent article, Michael Tiemann, one of the world's first open source entrepreneurs and VP of Open Source Affairs at Red Hat, highlights an example from the 1950s US Air Force where the "myth of the average resulted in a generation of planes that almost no pilots could reliably fly, and which killed as many as 17 pilots in a single day." He uses this example to argue that IT leaders who think that playing it safe means being as average as possible in order to avoid risks (i.e. "Buy what others are buying. Deploy what others are deploying. Manage what others are managing.") may be making IT procurement and strategy decisions based on flawed data. Instead, Tiemann says that IT leaders should understand elements of differentiation that are most valuable, and then adopt the standards that exploit them. "Don't aim for average: it may not exist. Aim for optimal, and use the power of open source to achieve what uniquely benefits your organization."

Comments

Same thing happens in dating

[gurps_npc]

People do things like say "I only want someone with > average height, > average salary,

They get 3 people - one of which is married, one gay, and the other refuses to date someone as tall/fat/stupid/poor as them.

People just don't understand the selectivity of multiple and requirements.

Please Explain

[BradleyUffner]

" an example from the 1950s US Air Force where the "myth of the average resulted in a generation of planes that almost no pilots could reliably fly, and which killed as many as 17 pilots in a single day"

Did I miss the part of the story that explains HOW it managed to kill 17 pilots in one day?

Re:Please Explain

[Anonymous Coward]

FTA: "Using the size data he had gathered from 4,063 pilots, Daniels [Lt. Gilbert S. Daniels, who majored in physical anthropology at Harvard before joining the Air Force] calculated the average of the 10 physical dimensions believed to be most relevant for [optimal cockpit] design, including height, chest circumference and sleeve length. These formed the dimensions of the “average pilot,” which Daniels generously defined as someone whose measurements were within the middle 30 per cent of the range of values for each dimension. So, for example, even though the precise average height from the data was five foot nine, he defined the height of the “average pilot” as ranging from five-seven to five-11. Next, Daniels compared each individual pilot, one by one, to the average pilot.

Before he crunched his numbers, the consensus among his fellow air force researchers was that the vast majority of pilots would be within the average range on most dimensions. After all, these pilots had already been pre-selected because they appeared to be average sized. (If you were, say, six foot seven, you would never have been recruited in the first place.) The scientists also expected that a sizable number of pilots would be within the average range on all 10 dimensions. But even Daniels was stunned when he tabulated the actual number.

Zero."

Re:Please Explain

[Immerman]

One more reason at least basic probability should be taught as a high-priority subject, starting in high school at the latest. Probability is after all something that we all deal with almost constantly in informing our decisions, and by nature we're actually pretty lousy at assessing it.

If there's a 30% chance that someone will fall within the designated "average" range on any given dimension, and assuming the dimensions are roughly independent, there's only a roughly 0.30^10 = 0.0006% chance that someone will fall into that range for all dimensions. That's only one individual in 169,350. Nobody with the most basic grasp of probability would expect many people to qualify.

And these are scientists that expected otherwise? They should be ashamed of themselves.

Re:Please Explain

[OzPeter]

But even Daniels was stunned when he tabulated the actual number.

Zero."

Which has actually jack squat explanation as to why the pilots died.

For all the details we are given the dead pilots could have been flying bespoke aircraft with hand crafted cockpits that matched there personal ergonomics, but were flying against enemies with aircraft that were 10 times superior in flight ability, but were mass produced.

Without any further details, the idea that ergonomics killed those pilots is pure assumption.

Re:Please Explain

[Anonymous Coward]

Did I miss the part of the story that explains HOW it managed to kill 17 pilots in one day?

... But even Daniels was stunned when he tabulated the actual number.

Zero."

"... at which point he flew into a homicidal rage, killing seventeen of the pilots before he could be subdued and sent to a mental institution."

Re:Please Explain

[Immerman]

Very true, thinking on my post I was actually hoping someone would bring that up. As you say it's obviously false that there's no correlation; however, unless you've actually collected the data to know what the actual degree of correlation is, assuming independence gives you a useful lower bound on probability to sanity-check your assumptions. You could also run the numbers for some "reasonable" guesses at correlation:

For example - lets say you only select individuals who have the first dimension within "acceptable" range - 100% of them, so we can ignore it. Furthermore we'll assume that all other dimensions have a freakishly high 80% correlation rate with the first of falling within their respective "average" range as well. You still end up with only 0.8^9 = 13% of candidates having all 10 dimensions within the average range.

So even with freakishly optimistic assumptions about correlation, you *still* end up with fewer than 1 in 8 candidates having all ten dimensions within your desired range.

Re:Please Explain

[spinozaq]

The quoted and linked article with the original article explains it a bit.... Here is a summary... When we started making jet fighters we had lots and lots of crashes, but they weren't from mechanical issues, so that seemed like pilot error, but the pilots didn't really think they were doing anything wrong. The cockpit design they were using was from 1926 and based on un-adjustable controls with positioning calculated from the "average pilot". While the upper management at the military was arguing about the costs of redesigning the planes engineerings invented adjustable seats and controls and pilots stopped crashing so much.

Averages are misleading

[mark-t]

For example, the average person has approximately 1 testicle.

Re: Averages do exist

[iluvcapra]

When managers deploy "average" security solutions, they're not trying to protect against threats, they're trying to avoid getting fired.

If they deploy something unusual and it doesn't work, they'll be fired, regardless of how it failed or the merits. If they deploy something everyone else has deployed and it doesn't work, they will be commended for following "industry best practices."

Not all organizations work this way, but many do. When something breaks, there's a big temptation to avoid an investigation into exactly what happened- who knows what that could turn up! Much easier just to fire middle managers for prima facie reasons.

Book misquoted; pilot crashes were in 1940's

[clovis]

The book "The End of Averages" by Todd Rose was misquoted
First of all the exact quote from the first paragraph of the book was this:

At its worst point, seventeen pilots crashed in a single day

There is a huge difference between crashing and dying.

Anyway, he (Teimann) got the sequence of events wrong, but the general gist of what he said follows the intent of the book.
The crashing planes in the study were the in the 1940's. We're talking about planes like the P-80 and possibly the F-86.
That was the first generation of jets and they had many many problems in design.

Here's where the average pilot comes in. Those planes (the 1940's) had been designed for the average pilot's size as measured in 1926. The cockpit was non-adjustable, so The Army/Air Force sought pilots whose size fit the planes, but only that person who matched the average 1926 pilot would fit properly. In the highly demanding jets of the late 1940's, a pilot that didn't fit could have problems when split second control reactions were needed, and those planes needed it.

The study conducted by Lieutenant Gilbert Daniels in 1950 which examined modern average pilot sizes, was completed in 1952.
The upshot of that study was that the Air Force immediately decided to take the study's recommendation:
Everyone is different, and to get the maximum performance from people you adjust the environment to the soldier, not the soldier.

The Air Force immediately mandated that the manufacturers make many elements of the cockpit be adjustable for the range of sizes from 5% to 95% of men from the seats, to pedal positions, to belts, and helmet straps, and so on. The result was that pilot performance soared and the US Air Force became the most dominant air force on the planet.

The book gives other example studies and goes on to say

Any system designed around the average person is doomed to fail

This is the gist of the book and what Michael Tiemann was getting at.

Anyway, the summary implied that the generation of planes designed in the 1950's were a generation of pilot killers.
The 1950's planes had the cockpit fit problems solved.
The crashing planes were in the late 1940's. The study was begun in 1950. Obviously, those crashes were not combat-related. Those planes were demanding and possibly evil, and a bad-fitting cockpit made it worse.