Slashdot Mirror

← Back to Stories (view on slashdot.org)

88% Of Medical 'Second Opinions' Give A Different Diagnosis - And So Do Some AI (mayoclinic.org)

Posted by EditorDavid on from the doctor-doctor dept.

First, "A new study finds that nearly 9 in 10 people who go for a second opinion after seeing a doctor are likely to leave with a refined or new diagnosis from what they were first told," according to an article shared by Slashdot reader schwit1: Researchers at the Mayo Clinic examined 286 patient records of individuals who had decided to consult a second opinion, hoping to determine whether being referred to a second specialist impacted one's likelihood of receiving an accurate diagnosis. The study, conducted using records of patients referred to the Mayo Clinic's General Internal Medicine Division over a two-year period, ultimately found that when consulting a second opinion, the physician only confirmed the original diagnosis 12 percent of the time. Among those with updated diagnoses, 66% received a refined or redefined diagnosis, while 21% were diagnosed with something completely different than what their first physician concluded.
But in a related story, Slashdot reader sciencehabit writes that four machine-learning algorithms all performed better than currently-used algorithm of the American College of Cardiology, according to newly-published research, which concludes that "machine-learning significantly improves accuracy of cardiovascular risk prediction, increasing the number of patients identified who could benefit from preventive treatment, while avoiding unnecessary treatment of others."

"I can't stress enough how important it is," one Stanford vascular surgeon told Science magazine, "and how much I really hope that doctors start to embrace the use of artificial intelligence to assist us in care of patients."

17 of 74 comments (clear)

  1. Different != more accurate by scsirob · · Score: 4, Insightful

    So what the study finds is that doctors as well as technology disagree. There's nothing indicating that the second or third opinion is correct.

    --
    To Terminate, or not to Terminate, that's the question - SCSIROB
    1. Re:Different != more accurate by ColdWetDog · · Score: 4, Interesting

      Pretty much this. Cardiovascular risk is one of the best studied disease states known. Which is probably why they studied it. Even then, the 'AI' algorithms only improved risk stratification around 5% - nothing to sneeze at but hardly earth shattering.

      OK, now, for extra credit lets risk stratify middle age hypertensive diabetics who are depressed.

      Like the typical 'real world' patient. I'd just love some help here but the underlying data just doesn't support it. Which is kinda surprising since we've been studying these folks for a while. Simple medical problems are simple. Typical medical problems are not.

      --
      Faster! Faster! Faster would be better!
    2. Re:Different != more accurate by Mr+D+from+63 · · Score: 2

      It is an important consideration that those that seek a second opinion often have reason to. It can be that the first doctor is uncertain, or the patient senses it, or the diagnosis doesn't seem right. It is not uncommon for doctors to recommend a second opinion when they are not sure or to make certain before treatment is started.

      The next questions is, how often is the first doctor's diagnosis correct?

    3. Re:Different != more accurate by quantumghost · · Score: 2

      I think you're on the right track....I have not RTFA, but the synopsis raises some concern about interpreting the conclusion. The data was taken from "General Internal Medicine Division", which are, typically, first line or Primary Care Physicians (PCPs). Often they will make a diagnosis based on their generalized knowledge of a medical problem and often refer them to a sub-specialist (i.e. you have trouble breathing with exertion, you find you need to sleep on multiple pillows, your ankles are swollen and you physical exam and office EKG lead me to refer you to a cardiologist (sub-specialist)) - so yes, the opinion will be different a lot of the time....the generalist can't know everything that a specialist can, but the generalist often triages the problem to someone who does know more.

      This could explain the "Among those with updated diagnoses, 66% received a refined or redefined diagnosis, while 21% were diagnosed with something completely different". It'd be interesting to see how they correlated this.

    4. Re:Different != more accurate by ShanghaiBill · · Score: 5, Insightful

      So matters are even worse than the articles suggest.

      Not necessarily. The study is an example of "selection bias". They only looked at cases where the patient requested a 2nd opinion. So these were cases where the patient already had a reason to question the 1st opinion. It is unlikely that the error (or "different diagnosis") rate would be as high for other cases.

      Before I go to a doctor, or take a family member, I do some online research to inform myself. If the doctor's diagnosis is significantly different from what I concluded, I ask for an explanation. If the doctor's "explanation" is that he is the expert and I should just trust him, then that usually means he is full of crap and doesn't know what he is doing.

    5. Re:Different != more accurate by mattwarden · · Score: 2

      As usual, journalists and /. submitters knowingly or unknowingly use statistics and data to imply things they don't actually imply. Here, we are trying to suggest there is a high error rate in medical diagnoses. But the rate of different diagnosis given you seek a second opinion is not at all the same as the base rate of a different diagnosis if randomly selected individuals (or everyone) sought a second opinion. You seek a second opinion when you don't like the first opinion, or when it's so serious and you had some reason to doubt or you hope you have a reason to doubt. The second doctor likely knows this is a second opinion, either because you told him/her or because he/she can see it in your medical records. And you may have sought out a doctor you know will think differently. For a similar example, many chiropractors tell you to talk to them before back surgery. Not quite the same thing, but if you go there, you know what they are going to tell you in every case except when the X-ray shows it would be dangerous to perform chiropractic adjustments.

      So what does this rate tell you? Not much. It certainly doesn't suggest the second opinion is correct. And it doesn't tell you much if anything about the base diagnosis error rate.

  2. WebMD? by houstonbofh · · Score: 2, Insightful

    So, WebMD really is the better option... ;)

  3. Of course by Gibgezr · · Score: 5, Insightful

    I would expect that many people who ask for a second opinion have a reason to ask for a second opinion: in fact, the article even mentions the situation where the first doctor recommended they do so. What would be more telling would be if they did a large study and gave EVERYONE second opinions, and then told us how many differed.
    This smells like another case of "lies, damned lies, and statistics", which is designed to make the Mayo Clinic look good.

    1. Re:Of course by Registered+Coward+v2 · · Score: 3, Interesting

      I would expect that many people who ask for a second opinion have a reason to ask for a second opinion: in fact, the article even mentions the situation where the first doctor recommended they do so. What would be more telling would be if they did a large study and gave EVERYONE second opinions, and then told us how many differed. This smells like another case of "lies, damned lies, and statistics", which is designed to make the Mayo Clinic look good.

      Correct. A sample size of 286 is pretty small and without knowing details on the data it's hard to draw from conclusions, unless you want to put out a press release. As for the /. header, 88% didn't give a different diagnosis 21% did, with some 67% getting a refined or redefined diagnosis. I would guess that the second opinion MD, knowing what the first said, would focus in on those aspects of the diagnosis to confirm or refine it; medicine is an inexact science, I'd be more worried about 90% agreement than the numbers from this study. More to the point, what was the actual conditions of the patients and what was the accuracy of diagnosis once the condition was positively identified.

      Mayo later points out that this study supports getting second opinions even though they cost money. Duh, an outfit that makes money off of MD visits supports more visits? I'm shocked, shocked. Next thing you know there'll be gambling at Rick's...

      What was interesting is the conclusion "There were no significant differences between provider types;" i.e. PA's and NPs did just as well as MDs; so using them as the primary entry point in healthcare may be one good way to lower costs while maintaining the quality of care.

      As for AI, it certainly is good to use it to aid in diagnosis, as it can learn about specific conditions and continue to build a database to draw from, far beyond an MD's ability to see a broad range of patients to help refine his or her diagnostic capability. AI is good at drawing conclusions from large datasets but not good at recognizing other symptoms that only manifest themselves in person, such as odors, odd way of walking or speaking, that can clue an MD into looking further.

      --
      I'm a consultant - I convert gibberish into cash-flow.
    2. Re:Of course by wisnoskij · · Score: 4, Interesting

      I was thinking more of the opposite. A doctor who is asked for a second opinion knows the patient does not want the same diagnosis. He knows that the patient is shopping around for the "best" diagnosis. He knows that the only way he is likely to be able to start treatment is to give a different diagnosis. I would say that medical diagnosis are complicated things that patients are not likely to be able to gauge correctly. If a patient is better at diagnosis than a doctor, and better able to tell if the correct one has been given, why even use doctors?

      --
      Troll is not a replacement for I disagree.
  4. Re:So Do Some Al by K.+S.+Kyosuke · · Score: 4, Funny

    Point 1: Who is Al?

    I've heard some call Paul Simon Al.

    --
    Ezekiel 23:20
  5. remAIn calm by Impy+the+Impiuos+Imp · · Score: 2

    concludes that "machine-learning significantly improves accuracy of cardiovascular risk prediction

    Computer: Subject has 728% increase in cardiovascular heart disease risk due to obesity, sedentary lifestyle, nuggets, and left-handed masturbation to furry Dragon Ball drawings.

    Impressed Doctor: God damn.

    --
    (-1: Post disagrees with my already-settled worldview) is not a valid mod option.
  6. Just maybe.. by AndyKron · · Score: 2

    Maybe the people who went for a 2nd diagnosis had reasons to do so.

  7. Key point it logic by ckatko · · Score: 2

    It should be read as: of people who were UNSATISFIED with their first doctor to the point they got a second opinion, were correct in their belief.

    Not that 88% of doctors visits are wrongly identified. 99.999% of people who get a cold, and get some antibiotics, and feel better in a couple of days, aren't seeking a second doctor.

  8. Re:What the AI is actually diagnosing by 0123456 · · Score: 2

    Patients should be the ones who have the medical records, and provide access to doctors when required.

  9. I needed FIVE opinions to get to the right Dx by Anonymous Coward · · Score: 3, Insightful

    I needed FIVE medical opinions to get to the correct diagnosis of my condition, and I had to go to Mexico to get it.

    Thank God that the doctor in Mexico correctly diagnosed my lung cancer, because FOUR American doctors kept on putting me off, two of them accusing me of being psychosomatic and telling me "nothing is wrong with me," and the other two incorrectly diagnosing me of, get this, allergies and asthma, respectively... because coughing up blood and having a constant metallic taste in the mouth are definitely characterizing symptoms of allergies and asthma.

    The good news is that I have civil suits against four well-insured American doctors and an insurance company now that will guarantee my family's financial well-being once I am gone, which will be in 3-9 months from now, especially given that, had my cancer been correctly diagnosed 3 years ago, it would have been operable and I likely would have had a good chance of survival.

  10. Re:I can explain by Anonymous Coward · · Score: 2, Insightful

    "I used to work at a hospital and I think I can explain the basis for this. Doctors are clueless, refuse to use technology, they rush everything, and they're incredibly lazy. I don't mean some of them either. 100% of them."

    Right....'Used to work at a hospital,' now more clueful than 100% of doctors, by his own estimate. With rippling abs I'll bet, too. Thank god we can all bask in this magnificent being's radiance! "My name is slashmydots, King of Kings; Look on my Works, ye Mighty, and despair!"

    What an ass.