Slashdot Mirror
Giving Doctors Grades Has Backfired
HughPickens.com writes: Beginning in the early 1990s a quality-improvement program began in New York State and has since spread to many other states where report cards were issued to improve cardiac surgery by tracking surgical outcomes, sharing the results with hospitals and the public, and when necessary, placing surgeons or surgical programs on probation. But Sandeep Jauhar writes in the NYT that the report cards have backfired. "They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates," says Jauhar. "When the statistics were publicized, some talented surgeons with higher-than-expected mortality statistics lost their operating privileges, while others, whose risk aversion had earned them lower-than-predicted rates, used the report cards to promote their services in advertisements."
Surveys of cardiac surgeons in The New England Journal of Medicine have confirmed that reports like the Consumer Guide to Coronary Artery Bypass Graft Surgery have limited credibility among cardiovascular specialists, little influence on referral recommendations and may introduce a barrier to care for severely ill patients. According to Jauhar, there is little evidence that the public — as opposed to state agencies and hospitals — pays much attention to surgical report cards anyway. A recent survey found that only 6 percent of patients used such information in making medical decisions. "Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences," concludes Jauhar. "It would appear that doctors, not patients, are the ones focused on doctors' grades — and their focus is distorted and blurry at best."
Surveys of cardiac surgeons in The New England Journal of Medicine have confirmed that reports like the Consumer Guide to Coronary Artery Bypass Graft Surgery have limited credibility among cardiovascular specialists, little influence on referral recommendations and may introduce a barrier to care for severely ill patients. According to Jauhar, there is little evidence that the public — as opposed to state agencies and hospitals — pays much attention to surgical report cards anyway. A recent survey found that only 6 percent of patients used such information in making medical decisions. "Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences," concludes Jauhar. "It would appear that doctors, not patients, are the ones focused on doctors' grades — and their focus is distorted and blurry at best."
How could no one have foreseen the potential abuse and pitfalls of a system like this? Without even reading any further than "Giving Doctors Grades..." I immediately conjured images of a bunch of doctors huddled around each other saying, "I don't want that one." "Well I don't want that one either. My feedback is back at 85% and I can't risk another death screwing me over."
bad metrics lead to bad results. Who would've guessed?
Gotta go, must write a million lines of code so I am "productive".
In NY, where I live, we're now "grading" teachers based on how well their students do on standardized tests. Any teacher who strays from the "prep for the test" subject matter and uses inventive ways of helping their students learn is going to have students who might know more, but who will perform worse on the tests. Teachers who stick to the script and drill test preparation into their students will wind up with better scores even though their students will know less (except how to fill in bubbles).
Just like the Doctors example in the article, the "teacher grading" system is going to backfire. Talented teachers will be kicked out (test scores are tied to their jobs now, your students get low scores and you're out) and mediocre teachers will remain. It's almost like trying to take the jobs that teachers and doctors do and standardize their job functions across every student/patient they see doesn't work. Maybe because their jobs require using their brains and trying different techniques as opposed to an assembly line worker who just needs to perform the same task every time with no variation.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
It's still a good idea, but the metrics need to be better thought-out to account for the patients that are being seen. A proper system will also "grade" each patient based on how bad their condition is, and then combine the mortality rates to come up with a metric that reflects how well the doctor is doing at improving outcomes where it is possible to do so.
I think there's a ton of money being dumped into the walking dead.
When my mom was at stage 4 of metastasized breast cancer, we had a family meeting with the oncologist to discuss my mom's situation. When asked what -- if any -- chances she had for life extension (not a cure, but more than 12 months) he was totally equivocal about it and was basically looking to start another round of chemotherapy. I felt like he was just looking for another round of payments before she died. They give you the thinnest hope to try to get you to keep using their services.
I've heard similar stories before from other people with older relatives, very sick and unlikely to every recover in any meaningful sense of the word yet the doctors insist on expensive and invasive treatments. The only explanation I can think of is that it's good business for them.
"They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates," says Jauhar.
It is true, some surgeons who are willing to treat very difficult cases would be adversely graded. But shouldn't there be some mechanism to apply brakes to the aggressive treatment? Some patients, and some of the relatives will be seeking treatment even when the situation is utterly hopeless. There are incentives for the doctors and the hospitals to pursue aggressive treatment. So, under these circs, is it really bad these grades are making them reevaluate the cases and be more realistic about the prognosis?
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact