Slashdot Mirror
Giving Doctors Grades Has Backfired
HughPickens.com writes: Beginning in the early 1990s a quality-improvement program began in New York State and has since spread to many other states where report cards were issued to improve cardiac surgery by tracking surgical outcomes, sharing the results with hospitals and the public, and when necessary, placing surgeons or surgical programs on probation. But Sandeep Jauhar writes in the NYT that the report cards have backfired. "They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates," says Jauhar. "When the statistics were publicized, some talented surgeons with higher-than-expected mortality statistics lost their operating privileges, while others, whose risk aversion had earned them lower-than-predicted rates, used the report cards to promote their services in advertisements."
Surveys of cardiac surgeons in The New England Journal of Medicine have confirmed that reports like the Consumer Guide to Coronary Artery Bypass Graft Surgery have limited credibility among cardiovascular specialists, little influence on referral recommendations and may introduce a barrier to care for severely ill patients. According to Jauhar, there is little evidence that the public — as opposed to state agencies and hospitals — pays much attention to surgical report cards anyway. A recent survey found that only 6 percent of patients used such information in making medical decisions. "Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences," concludes Jauhar. "It would appear that doctors, not patients, are the ones focused on doctors' grades — and their focus is distorted and blurry at best."
Surveys of cardiac surgeons in The New England Journal of Medicine have confirmed that reports like the Consumer Guide to Coronary Artery Bypass Graft Surgery have limited credibility among cardiovascular specialists, little influence on referral recommendations and may introduce a barrier to care for severely ill patients. According to Jauhar, there is little evidence that the public — as opposed to state agencies and hospitals — pays much attention to surgical report cards anyway. A recent survey found that only 6 percent of patients used such information in making medical decisions. "Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences," concludes Jauhar. "It would appear that doctors, not patients, are the ones focused on doctors' grades — and their focus is distorted and blurry at best."
In NY, where I live, we're now "grading" teachers based on how well their students do on standardized tests. Any teacher who strays from the "prep for the test" subject matter and uses inventive ways of helping their students learn is going to have students who might know more, but who will perform worse on the tests. Teachers who stick to the script and drill test preparation into their students will wind up with better scores even though their students will know less (except how to fill in bubbles).
Just like the Doctors example in the article, the "teacher grading" system is going to backfire. Talented teachers will be kicked out (test scores are tied to their jobs now, your students get low scores and you're out) and mediocre teachers will remain. It's almost like trying to take the jobs that teachers and doctors do and standardize their job functions across every student/patient they see doesn't work. Maybe because their jobs require using their brains and trying different techniques as opposed to an assembly line worker who just needs to perform the same task every time with no variation.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
Wasn't this addressed by the Scrubs TV show years ago?
Yes. I've made a similar argument for the education system, there are not enough good metrics that measure both student and teacher performance, and having the teacher provide most 'graded' assessment with little more than a pass/fail for occasional standardized testing for graduation doesn't work. For education, use an independent means to assess for all tests, so that the instructor does not even proctor thes tests let alone grade them, and account for student ratings coming-in to the teacher's class as a means of quantifying how the teacher does based on change in performance by the end of the year.
Do a similar thing for surgeons and other doctors. Grade the case, then grade the doctor's performance on the case. If anything it might reveal that some doctors are taking a disproprtionate number of untreatable cases too far in treatment when hospice really should be the course of action, in addition to revealing when doctors, faced with difficult cases that might be treatable, are doing well considering their cache of patients.
Do not look into laser with remaining eye.
This sounds tough, but how much of the high risk- low success operations being done contribute to the high cost of health care in the US? maybe in some of the high risk situations somebody needs to say no. sorry, but costs are out of control and we need some realistic assessments. On a similar note, its been some years since I've heard people say ' I don't care how much it costs, if it just saved one life it was worth it.'
Govt is exceptionally good a proving the 'theory' of unintended consequences. n-level thinking requires more cynicism than most politicians can muster.
"They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates," says Jauhar.
It is true, some surgeons who are willing to treat very difficult cases would be adversely graded. But shouldn't there be some mechanism to apply brakes to the aggressive treatment? Some patients, and some of the relatives will be seeking treatment even when the situation is utterly hopeless. There are incentives for the doctors and the hospitals to pursue aggressive treatment. So, under these circs, is it really bad these grades are making them reevaluate the cases and be more realistic about the prognosis?
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Think standardized testing, where suddenly teachers are only teaching what you need to pass the standardized test.
Poor analogy. Doctors can choose their patients. Teachers can rarely choose their students.
If what is on the test is not what you want the students to be learning, then the problem is with the design of that test, not with standardized testing in principle. Most people that object to our current system of testing, have no interest in improving it, but rather prefer no accountability at all.