What Medical Tests Should Teach Us About the NSA Surveillance Program

First time accepted submitter Davak writes "In many ways finding the small amount of terrorists within the United States is like screening a population of people for a rare disease. A physician explains why collecting excessive data is actually dangerous. Each time a test is run, the number of people incorrectly identified quickly dwarfs the correct matches. Just like in medicine, being incorrectly labelled has serious consequences."

Comments

The crazy is strong in this one

[davebarnes]

You, sir, are a nutjob.

Re:hmm...doctors just don't worfk as hard

[uglyduckling]

No, because the point is that the false positive results lead to more invasive tests (which in themselves may do harm), over-interpretation of other physical signs, worry etc.. The parallel with terrorism is that people end up on no-fly lists, get invasively searched and questioned, might get turned down for jobs or credit etc.. The uselessness of screening tests for low prevalence diseases is well known in the medical world, which is why tests need to be targeted to a high-risk population to have any value.

Here's the same argument about drug testing

[nbauman]

Here's another doctor who made the same argument about testing for illegal drugs. Be sure to catch the distinction between screening tests and diagnostic tests.

http://www.kevinmd.com/blog/2013/07/drug-testing-considered-screening-tests.html
Should drug testing be considered screening tests?
Chris Rangel, MD | Conditions | July 12, 2013

... The problem of a false positive test is frequently encountered in the practice of medicine. Depending on the clinical circumstances and the nature of the initial test, follow up evaluation with more expensive and possibly more invasive testing is often required in order to verify the results. For example, an abnormality found on the chest x-ray of a smoker with a bad cough requires further evaluation. A CT scan of the chest, bronchoscopy, and even a needle biopsy to obtain a tissue sample for analysis are required before making a diagnosis of lung cancer and starting treatment.

However, the possibility of a false positive drug screen and the need for further testing and evaluation is rarely considered outside the context of clinical practice. Employers, school administrators, government agencies, and law enforcement can and do consider a positive drug test to be perfectly equivalent to an admission of illicit drug use. This frequently results in the administration of some form of punishment or corrective action being delivered without giving the accused the right to defend themselves in any way. Essentially, drug testing is an effective way to violate a person’s right to due process since most drug screening is managed by lay people in non-clinical roles who believe that drug testing is 100% reliable. But this would be the same absurdity as giving chemotherapy to the smoker with the abnormal chest x-ray without first trying to verify the diagnosis with further evaluation (due process).

The other problem comes from the mass drug testing of large numbers of people (either random or at the initial point of contact). The interpretation of the results of a medical test are never as simple as positive or negative. The statistical probability of a false positive or a false negative result must be considered in concert with the pretest probability....

Re:Flawed Analogy

[AthanasiusKircher]

When you screen huge masses of people needlessly, almost all to all of your hits are going to be incorrect.

Yes, this is something that apparently even most doctors don't understand. Suppose who had a simple problem like this:

1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

The correct answer (calculated from Bayes' Theorem, or simple logic) is 7.8%. Most doctors cannot do this problem, and that not only get the answer wrong, but they often get it wildly off -- estimating the answer to be much greater than 50% (often 70% or so, probably from simply subtracting the two numbers).

If you don't believe me, have a look at this link. As the author says there:

usually, only around 15% of doctors get it right. ("Really? 15%? Is that a real number, or an urban legend based on an Internet poll?" It's a real number. See Casscells, Schoenberger, and Grayboys 1978; Eddy 1982; Gigerenzer and Hoffrage 1995; and many other studies. It's a surprising result which is easy to replicate, so it's been extensively replicated.)

The author here is being generous. I looked at these studies years ago, and many of them show only 5-10% getting the answer to such problems correct.

And if this is true of physicians, it's probably true of just about anyone else who encounters a lot of false positives and isn't used to thinking statistically. That means most people are very likely to draw incorrect conclusions about the prevalence of something when the false-positive rate is high... making those using the methodology assume that (1) their methodology is better than it is, and (2) that with more "assumed positives" from incorrect logic, the incidence of whatever they're looking for in the population is higher than it is.