Slashdot Mirror
Why Programmers Need To Learn Statistics
David Gerard writes "Zed Shaw writes an impassioned plea to programmers: Programmers Need To Learn Statistics Or I Will Kill Them All. Quoting: 'I go insane when I hear programmers talking about statistics like they know s*** when it's clearly obvious they do not. I've been studying it for years and years and still don't think I know anything. ... I have taken a bunch of math classes, studied statistics in grad school, learned the R language, and read tons of books on the subject. Despite all of this I'm not at all confident in my understanding of such a vast topic. What I can do is apply the techniques to common problems I encounter at work. My favorite problem to attack with the statistics wolverine is performance measurement and tuning. All of this leads to a curse since none of my colleagues have any clue about what they don't understand. I'll propose a measurement technique and they'll scoff at it. I try to show them how to properly graph a run chart and they're indignant. I question their metrics and they try to back it up with lame attempts at statistical reasoning. I really can't blame them since they were probably told in college that logic and reason are superior to evidence and observation.'"
Maybe the problem is in your presentation. Even here, you tell programmers that you want to kill them for not understanding a topic that even you are unwilling to acknowledge mastery of. Then you tell us how hard the topic is to understand, even though you've spent so much time trying to learn it.
Is it any wonder that no one takes your suggestions seriously? You are practically sabotaging yourself with self-effacement.
These aren't homework problems you're tackling here. They are business problems and you need to sell yourself and your ideas if you want to get any traction. Do you have any evidence that your methods are better than the SOP thus far? Do you have any case studies that show how effective statistic analysis is in *any* of your projects?
Or are you simply taking something that seems like a data point and extrapolating it to cover a vast swath of applications?
Statisticians need to learn programming or I will kill them all.
We know as much statistics as we need to know.
Some know more, some less. Each has traded off hours vs. knowledge in many fields.
For example: Why would a programmer who's job is to automate bean counting need to know more then basic statistics? (s)he rightfully focuses his efforts on accounting.
One post calculus statistics course gives me enough grounding to know what I don't know and punt to experts when I need to.
Fucking specialists forget all the things they don't know and only look at the world through one lens.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
He's just as arrogantly claiming that he's right and they're wrong. Now, he may very well in fact be right, but he's taking the same obstinate position the people he criticizes do.
It's important to know when your input is not desired. Even if you think it should be.
is not because they don't understand statistics. It is because you are a dick.
I know enough about statistics to know statistically I know I'm safe from his threats. I suspect if I were a bag of Cheetos the odds were be against me but that's not the case.
A manga statistics book, eh?
I just realized I was a nerd. I looked at the table of contents and closed it down, then realized I hadn't even looked at the short skirt-wearing protagonist.
Sigh...
But to answer the article's point, elementary statistics are very easy. Advanced statistics are very hard. It's kind of like how people think "knowing the difference between circles and squares" is geometry and so analytical geometry must be just more of the same, right? It's quite possible the programmers think they know statistics because they know they're vaguely supposed to do a run multiple times, and maybe average the results or something.
It's also possible the author of the article is a know-it-all douchebag who tries to solve problems with overwrought solutions.
From TFA: "Zed: Fuck! Fuck! I have eyes! You do not! See!? No?! Exactly! Because you can't fucking see because you have no fucking eyes! Arrggh!"
Just throwing that theory out there.
I've been studying it for years and years and still don't think I know anything.
And yet you're expecting someone whose expertise is in a different field to know more about it than you?
We can't all be experts in everything. If you're the expert in the field of discussion, get used to educating your coworkers on the topic, or find another job where you're surrounded by people with the same education and expertise as you.
The average person is an expert in no more than two or three related areas. That's why people work in teams, to cover each other's blind spots.
I work for the Department of Redundancy Department.
I disagree that CS is just "programming and troubleshooting", but I do agree that Computer Science is a complete misnomer. It's extremely misleading, and difficult to explain to people, "I'm a computer scientist, but no I'm not actually a scientist, instead I understand how to describe formal languages in terms of strict grammar rules and transform abstract syntax trees from one representation to another."
It shouldn't be called Computer Science, it should be called Computational Mathematics, because that's what it is.
(On the other hand, there is whole branch of CS that extends very deeply into statistics called Machine Learning, but at the core I'd say it is still more mathematics than science. There is also human-machine interaction which often goes under CS, but is actually more like psychology.. so it's not so cut and dry.)
...unfortunately, they are mostly lost in the irony of statements like this:
I think women are better programmers because they have less ego and are typically more interested in the gear rather than the pissing contest.
I doubt I've seen anyone more thoroughly entrenched in a pissing contest than Zed Shaw, of the website formerly known as "Zed's So Fucking Awesome".
Don't thank God, thank a doctor!
Statistics is HARD, for two reasons:
I'd argue that probability theory isn't as hard as people make it seem, but statisticians are wankers. Most of what we think of statistics was developed by people who were intimately engaged with empirical research, but modern statisticians are mathematicians, many of whom have never actually performed an experiment. They think the statistics are real, whereas experimental scientists know the truth: God made the Probability Distribution Functions. All else is the work of man.
Furthermore, modern computing has made a lot of the conceptual apparatus of conventional statistics irrelevant, as it is designed to deal with the problem of reducing problems to something that can be computed by hand and finished off with a single table lookup. Today its a rare case that we can't get at the PDFs directly, bypassing much of conventional statistics. But due to how badly the stats are taught, and how poorly probability theory is understood, we are still living in a world where p-values are the exception, not the norm, and when they are quoted they are frequently unrealistic because they are based on statistical assumptions that are not warranted given the non-idealities of the data.
So I'd argue that statistics is basically a dead field populated by zombies who are dedicated to infecting as many students as possible. If we taught thermodynamics or mechanics with equally outmoded concepts they would be really hard too.
Blasphemy is a human right. Blasphemophobia kills.
So, since so many people don't seem to want to actually read Zed's stuff -- and I honestly don't blame you -- I'll try to summarize:
Eventually, every major science adopted an empiricist view of the world. Except Computer Science of course.
He tends to bitch a lot about computer scientists. I'm just starting a CS degree, and there is a Statistics class in the curriculum. Is he working with people with good degrees, people from a technical college with a "programming" degree, people from a diploma mill, or high school students with no degree at all?
Of course, he seems to be implying it's everyone, and doing so in a typically Zed-like way.
"All you need to do is run that test [insert power-of-ten] times and then do an average." Usually the power-of-ten is 1000...
I don't know that I've ever heard that particular statement. But it's a good point:
How do you know that 1000 is the correct number of iterations to improve the power of the experiment?
Generally because it was probably closer to a million, so I'm erring on the side of taking more, rather than fewer, measurements. But without careful consideration, I could be way off.
How are you performing the samplings?
I think this is vastly less important than how you are dealing with the data, but it is also a good point. For example, his complaint is that an average isn't enough; with detailed enough logging, he could easily go back into my data and figure out min, max, standard deviation, histograms...
How do you know that 1000 is enough to get the process into a steady state after the ramp-up period?
Not a huge deal -- the "steady state" will almost certainly be faster than the "ramp-up" period. Worst case, I'm over-optimizing.
What will you do if the 1000 tests takes 10 hours?
Either ctrl+c, or try it 10 times.
How does 1000 sequential requests help you determine the performance under load?
Very good point here. It's still a useful statistic, but you still need to measure things like 1000 simultaneous requests, not just 1000 all in sequence.
On the other hand, if your performance is acceptable with them all in sequence, you could just run it through something like Event Machine, so it's all sequential on production, too.
The most troubling problem with these single number “averages” is that there’s two common averages and that without some form of range or variance error they are useless. If you take a look at the previous graphs you can see visually why this is a problem. Two averages can be the same, but hide massive differences in behavior...
So yes, always make sure you can record enough statistics so that someone else can come along and use your data to give you something meaningful.
The moral of the story is that if you give an average without standard deviations then you’re totally missing the entire point of even trying to measure something. A major goal of measurement is to develop a succinct and accurate picture of what’s going on...
It doesn't have to be statistically accurate. It just has to be close enough.
Ah, confounding. The most difficult thing to explain to a programmer, yet the most elementary part of all scientific experimentation. It’s pretty simple: If you want to measure something, then don’t measure other shit.
This is both a very good and a very bad idea. It ties into the peeve he had before -- ramp-up time. For example:
If we want to take one single line of code and test it then we can. If we want to only verify one single query on a database then what’s stopping us?
What's stopping us is that our applications don't actually work like that.
Don't thank God, thank a doctor!
"Lies, damn lies and statistics" is all you need to know about statistics.
This is right up there with "'click on the big blue e' is all you need to know about the internet."
Speaking as both a statistician and a computer scientist, I've seen the statistics-vs.-CS argument play out many times before, and the lack of knowledge on both sides is really striking, but not all that surprising -- both are hard subjects which take a lot of work to master. The lack of mutual respect is both infuriating and pathetic, and there's no excuse for it.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
Yup. Also, for a guy who claims to know so much about statistics and measurement, it's weird how he judges programmers so sweepingly on the sole basis of his anecdotal experiences.
Today's weirdness is tomorrow's reason why. -- Hunter S. Thompson
"It's also possible the author of the article is a know-it-all douchebag who tries to solve problems with overwrought solutions."
That was kinda what I got from this. Sure, my powers of ten runs to determine performance isn't statistically sound. Did I say it was? No. Why don't I care? Because my samples are cheap. Spiking vs non-spiking is something pretty easy to see when you glance at the data.
I mean, he said we're going to die if we don't learn statistics, but he never gave a compelling argument for it.
The best example was users, but even that was lacking. If you design a script that's as aggressive on a system as a high use user and your system supports as many 'users' as students, you're safe, if it supports less you work on qualifying the problem better then.
Thing is: You can only be expert in ONE of them. Period.
Hundreds of cryptologists prove you wrong.
You can only be expert in ONE of them. Period.
[shrug] Depends on how you define "expert," I suppose. I have one MS in CS and another in biostatistics, and am currently working on a PhD in bioinformatics, where I use the knowledge I've gained in both fields pretty much every day. If you think CS is "waaayy more interesting," that's fine for you; personally I find them equally interesting and valuable.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.