Slashdot Mirror
Another Dementia Test Oversold
An anonymous reader writes: Many prominent news organizations, including the BBC, are reporting on a study (PDF) that claims a new blood test is 87% accurate in predicting which patients will develop cognitive impairment. It's hailed as a major step forward in efforts to fight dementia and Alzheimer's disease. Unfortunately, reality isn't quite so impressive. An article at MedPage Today explains all of the statistical facts that the mainstream press glosses over: "Only about 10% of patients of patients with MCI convert to clinical dementia per year. With nearly 30% of positive results false (remember, the specificity was 71%) as well as 15% of negative results false, most of the positive results in such a group will be false. Yes, it's time once again for a tutorial in positive predictive values. If we have 100 MCI patients and a 10% conversion rate, then 10 of them will develop dementia. These are the true positives. There will be 90 true negatives — the ones who don't convert. But with a specificity of 71%, the test will falsely identify 29% of the 90 true negatives, or 26, as positive. Meanwhile, with a false negative rate of 15%, only nine (rounding up from 8.5) of the 10 true positives will be correctly identified. ... It's easy to get a high negative predictive value when the annual event rate is 10%. If I simply predict that no one will convert, I'll be right 90% of the time."
All the mainstream articles I've read, whilst as usual glossing over the stats, are presenting this as a vast improvement, not as something that should be rolled out clinically. Didn't read it as hype.
Is it a major step towards a viable test, or is that just not true?
Or that they have learned, passed the test, and forgotten. There was quite an interesting read on exactly this subject on the BBC website this week, Do doctors understand test results?:
In one session, almost half the group of 160 gynaecologists responded that the woman's chance of having cancer was nine in 10. Only 21% said that the figure was one in 10 - which is the correct answer. That's a worse result than if the doctors had been answering at random.
And yet, I just read a study that most doctors and clinicians involved with breast cancer can not correctly identify the liklihood of having breast cancer if someone tests positive for it. Answers ranged from 10-90%. Correct answer is 10%. Being taught it in medical school is one thing, but applying it on the job is another apparently.
Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
This is a generic problem with media reports of medical results. No understanding of the statistics.
Add in the lack of repeatability of many studies (for example just about anything claiming bad effects from low doses of bisphenol-A has proven to not be repeatable) and outright fraud (Wakefield) or attempts to push the data using questionable statistics (Séralini) and you have a giant problem with media and medical research results.
Said Thad Cochran.
Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
Learned is such a strong word. I teach evidence based medicine, including test evaluation to med students and residents. I also have to try and explain it to practicing physicians.
Hey, the condescending article manages to not know what positive predictive values are!
These sales people are demented, and I have the tests to prove it.
Table-ized A.I.
Sounds almost like they read the question wrong.
A while back, a friend of mine went to the doctor complaining of fatigue. He was just constantly tired and unsure why. They checked him into the hospital for tests, and shortly declared he had Leukemia and needed to start chemotherapy right away. After several weeks, including a long stay in the ICU, the doctors declared the treatment a "success". He remained in the ICU and died yesterday.
Fucking quacks "practicing" medicine and performing "treatments" that have a proven low survival rate - but, hey, everybody including the pharmaceutical companies are raking in the money, so it's all good. Except for my friend, who would have lived longer and had less pain than he did under their "treatment".
Fuck them. Stop it with the myopic single diagnosis / single treatment crap. People are NOT petri dishes.
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
There are three kind of lies: lies, damn lies and statistics.
THis is basic statistics learned by every doctor in medical school. specificity and sensitivity, prevalence, pretest and post test probability and false positive/true negative, false negative/true positive. They all factor in to deciding to use a medical test. Every person who comes in and demands a test ussually gets a lecture on this (at least from me) (at work have to post as AC)
Statistics aside there are two more pressing questions: 1, are patients with MCI more likely to convert (10% or greater) in subsequent years? 2, is there a preventative process that can be used by those that test positive, that is safe for the general population? In other words, do we get to narrow our treatment focus with these results, even a little bit? We can piss and moan about statistics but at the end of the day, who really gives a fuck if you were right about the onset of someone's dementia, unless you can do something about it?