Slashdot Mirror
What Medical Tests Should Teach Us About the NSA Surveillance Program
First time accepted submitter Davak writes "In many ways finding the small amount of terrorists within the United States is like screening a population of people for a rare disease. A physician explains why collecting excessive data is actually dangerous. Each time a test is run, the number of people incorrectly identified quickly dwarfs the correct matches. Just like in medicine, being incorrectly labelled has serious consequences."
No, because the point is that the false positive results lead to more invasive tests (which in themselves may do harm), over-interpretation of other physical signs, worry etc.. The parallel with terrorism is that people end up on no-fly lists, get invasively searched and questioned, might get turned down for jobs or credit etc.. The uselessness of screening tests for low prevalence diseases is well known in the medical world, which is why tests need to be targeted to a high-risk population to have any value.
When you screen huge masses of people needlessly, almost all to all of your hits are going to be incorrect.
Yes, this is something that apparently even most doctors don't understand. Suppose who had a simple problem like this:
1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?
The correct answer (calculated from Bayes' Theorem, or simple logic) is 7.8%. Most doctors cannot do this problem, and that not only get the answer wrong, but they often get it wildly off -- estimating the answer to be much greater than 50% (often 70% or so, probably from simply subtracting the two numbers).
If you don't believe me, have a look at this link. As the author says there:
usually, only around 15% of doctors get it right. ("Really? 15%? Is that a real number, or an urban legend based on an Internet poll?" It's a real number. See Casscells, Schoenberger, and Grayboys 1978; Eddy 1982; Gigerenzer and Hoffrage 1995; and many other studies. It's a surprising result which is easy to replicate, so it's been extensively replicated.)
The author here is being generous. I looked at these studies years ago, and many of them show only 5-10% getting the answer to such problems correct.
And if this is true of physicians, it's probably true of just about anyone else who encounters a lot of false positives and isn't used to thinking statistically. That means most people are very likely to draw incorrect conclusions about the prevalence of something when the false-positive rate is high... making those using the methodology assume that (1) their methodology is better than it is, and (2) that with more "assumed positives" from incorrect logic, the incidence of whatever they're looking for in the population is higher than it is.