IBM Watson Reportedly Recommended Cancer Treatments That Were 'Unsafe and Incorrect'

An anonymous reader quotes a report from Gizmodo: Internal company documents from IBM show that medical experts working with the company's Watson supercomputer found "multiple examples of unsafe and incorrect treatment recommendations" when using the software, according to a report from Stat News. According to Stat, those documents provided strong criticism of the Watson for Oncology system, and stated that the "often inaccurate" suggestions made by the product bring up "serious questions about the process for building content and the underlying technology." One example in the documents is the case of a 65-year-old man diagnosed with lung cancer, who also seemed to have severe bleeding. Watson reportedly suggested the man be administered both chemotherapy and the drug "Bevacizumab." But the drug can lead to "severe or fatal hemorrhage," according to a warning on the medication, and therefore shouldn't be given to people with severe bleeding, as Stat points out. A Memorial Sloan Kettering (MSK) Cancer Center spokesperson told Stat that they believed this recommendation was not given to a real patient, and was just a part of system testing.

According to the report, the documents blame the training provided by IBM engineers and on doctors at MSK, which partnered with IBM in 2012 to train Watson to "think" more like a doctor. The documents state that -- instead of feeding real patient data into the software -- the doctors were reportedly feeding Watson hypothetical patients data, or "synthetic" case data. This would mean it's possible that when other hospitals used the MSK-trained Watson for Oncology, doctors were receiving treatment recommendations guided by MSK doctors' treatment preferences, instead of an AI interpretation of actual patient data. And the results seem to be less than desirable for some doctors.

Comments

Really no surprise

[gweihir]

This is a statistics-driven automaton that has zero insight or understanding. Calling it "AI" is a marketing lie, even if the AI field has given in and calls things like this "weak AI", which is the AI without "I". As such, this machine can find statistical correlations, but it cannot do plausibility checks, because that requires insight. It cannot do predictions either, because that also requires insight. The real strength of Watson (and it is quite an accomplishment) is that unlike older comparable systems, you can feed the training data and the queries into it in natural language. This means you can train a lot cheaper, but at the cost of accuracy, as the effect described in the story nicely shows.

It is time for this "AI" hype to die down. All it shows is that many people do not chose to use what they have in general intelligence and rather mindlessly follow a crows of cheer-leaders.

Garbage in, garbage out

[blindseer]

An AI can only be as good as the data used to train it. The article pointed out that Watson was trained using what was possibly based as much on objective data as much as it was on subjective preferences of the physicians that fed it data.

I recall reading an article about someone doing a study on medical procedures done throughout the USA and they noticed "hot spots" of procedures being done in certain areas. What they found was that in these places they'd see physicians that would recommend procedures out of personal preference. One example was a an area with a lot of tonsillectomies, because a physician felt that any throat infection meant the tonsils had to come out. Another area had an elevated number of hysterectomies, because a physician felt that post-menopause women had an elevated risk of developing cysts and cancers on the uterus. The article went on to say that while such treatments may be unusual no one was willing to consider this malpractice.

So, Watson recommended a treatment for someone that might aggravate an existing problem of severe bleeding. Is this bad coding for not taking this into account? Or, is there a physician that entered such a prescription for their patient with similar symptoms? It's real difficult to second guess a physician. It's real easy to second guess the computer. Even if both the computer and the human came to the same recommendation for treatment.