Slashdot Mirror
IBM Watson Reportedly Recommended Cancer Treatments That Were 'Unsafe and Incorrect'
An anonymous reader quotes a report from Gizmodo: Internal company documents from IBM show that medical experts working with the company's Watson supercomputer found "multiple examples of unsafe and incorrect treatment recommendations" when using the software, according to a report from Stat News. According to Stat, those documents provided strong criticism of the Watson for Oncology system, and stated that the "often inaccurate" suggestions made by the product bring up "serious questions about the process for building content and the underlying technology." One example in the documents is the case of a 65-year-old man diagnosed with lung cancer, who also seemed to have severe bleeding. Watson reportedly suggested the man be administered both chemotherapy and the drug "Bevacizumab." But the drug can lead to "severe or fatal hemorrhage," according to a warning on the medication, and therefore shouldn't be given to people with severe bleeding, as Stat points out. A Memorial Sloan Kettering (MSK) Cancer Center spokesperson told Stat that they believed this recommendation was not given to a real patient, and was just a part of system testing.
According to the report, the documents blame the training provided by IBM engineers and on doctors at MSK, which partnered with IBM in 2012 to train Watson to "think" more like a doctor. The documents state that -- instead of feeding real patient data into the software -- the doctors were reportedly feeding Watson hypothetical patients data, or "synthetic" case data. This would mean it's possible that when other hospitals used the MSK-trained Watson for Oncology, doctors were receiving treatment recommendations guided by MSK doctors' treatment preferences, instead of an AI interpretation of actual patient data. And the results seem to be less than desirable for some doctors.
According to the report, the documents blame the training provided by IBM engineers and on doctors at MSK, which partnered with IBM in 2012 to train Watson to "think" more like a doctor. The documents state that -- instead of feeding real patient data into the software -- the doctors were reportedly feeding Watson hypothetical patients data, or "synthetic" case data. This would mean it's possible that when other hospitals used the MSK-trained Watson for Oncology, doctors were receiving treatment recommendations guided by MSK doctors' treatment preferences, instead of an AI interpretation of actual patient data. And the results seem to be less than desirable for some doctors.
Really where is the there, here ? You'll have doctors frequently dispute what the correct treatment is and with diseases like cancer it doesn't help that the best you can often do is offer a statistical improvement of someone's chances.
Far better that more people can afford treatment faster than this remain the province of the priesthood.
... but it will the patient. Is that a problem?"
Doctor (shaking his head): Yes, Watson... that is a problem.
(Who trained Watson for this job anyway?)
CUR ALLOC 20195.....5804M
The purpose of such a tool should be to make suggestions that a doctor may not consider themselves. It should be up to the doctor(s) to vet the suggestions or leads before any treatment is actually rendered. A doctor would have to be born in Stupidville to accept bot suggestions as-is.
Table-ized A.I.
This is a statistics-driven automaton that has zero insight or understanding. Calling it "AI" is a marketing lie, even if the AI field has given in and calls things like this "weak AI", which is the AI without "I". As such, this machine can find statistical correlations, but it cannot do plausibility checks, because that requires insight. It cannot do predictions either, because that also requires insight. The real strength of Watson (and it is quite an accomplishment) is that unlike older comparable systems, you can feed the training data and the queries into it in natural language. This means you can train a lot cheaper, but at the cost of accuracy, as the effect described in the story nicely shows.
It is time for this "AI" hype to die down. All it shows is that many people do not chose to use what they have in general intelligence and rather mindlessly follow a crows of cheer-leaders.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I'll take Incorrect Diagnosis for $200, Alex.
But is Watson cheaper than a doctor?
How many human doctors did the same or worse?
Mostly random stuff.
In practice the term "AI" is vague and continuous rather than a Boolean designation ("is" versus "is-not"). The term is not worth sweating over. The exception may be if you are making a big purchase and/or investment based on something being "AI". In that case, inspect it carefully rather than assume something with "AI" is smart and/or useful. But that's good advice for any significant purchase: test drive it & ask detailed questions rather than rely on the brochure.
Table-ized A.I.
...is this: "A Memorial Sloan Kettering (MSK) Cancer Center spokesperson told Stat that they believed this recommendation was not given to a real patient, and was just a part of system testing."
Isn't this the kind of thing that testing is designed to uncover? It sounds to me like at least this part of the process is working, unlike the asshole who fed the model "fake data".
It just wanted to help impose pro-Darwinian responses to malformed genetic abnormalities.
Next up: self-driving cars that crash on purpose because their passengers sing songs the AI hates.
-- Tigger warning: This post may contain tiggers! --
So the data fed to train Watson wasn't from actual cases? Why does it matter what the computer prescribed, then? The system that is Watson is only as good as the data you feed it. Feed it fake information, get not even wrong results. Sounds more like a smear campaign,
intentionally designed to fail, and certainly not an experiment designed to measure Watson's recommendations against actual doctor recommendations.
Here's a better idea...
Feed the damn thing actual patient records with everything included from first immunization to the patient's ultimate death. If you are looking to see if there are any correlations that humans haven't already made you need to feed that sucker as much data as is inhumanly possible and then let it do the work.
What we have now is a pseudepigrapha of Watson's capabilities. Sure the results are from Watson, but they are not what Watson would do if given accurate, real life data to work with. They made a forgery of the system and put Watson's name on it.
Shady, bro. Shady...
When the only tool you have is a claw hammer every problem starts to look like the back of someone's skull.
An AI can only be as good as the data used to train it. The article pointed out that Watson was trained using what was possibly based as much on objective data as much as it was on subjective preferences of the physicians that fed it data.
I recall reading an article about someone doing a study on medical procedures done throughout the USA and they noticed "hot spots" of procedures being done in certain areas. What they found was that in these places they'd see physicians that would recommend procedures out of personal preference. One example was a an area with a lot of tonsillectomies, because a physician felt that any throat infection meant the tonsils had to come out. Another area had an elevated number of hysterectomies, because a physician felt that post-menopause women had an elevated risk of developing cysts and cancers on the uterus. The article went on to say that while such treatments may be unusual no one was willing to consider this malpractice.
So, Watson recommended a treatment for someone that might aggravate an existing problem of severe bleeding. Is this bad coding for not taking this into account? Or, is there a physician that entered such a prescription for their patient with similar symptoms? It's real difficult to second guess a physician. It's real easy to second guess the computer. Even if both the computer and the human came to the same recommendation for treatment.
I am armed because I am free. I am free because I am armed.
You mean imposing sanctions, killing hundreds of Russian soldiers, giving $200 million in weapons to the Ukraine, expressly rejecting Russia's takeover of the Crimea, pushing to put US troops and their missile shield into Poland, increasing fracking to drive down the price of oil, trying to force Europe to stop buying Russian gas and increase their militaries...?
Trump has already done more to oppose Russia than Obama ever did - Obama didn't have the guts to enforce his own "red-line" in Syria.
But when Obama makes nice with a Russian 'reset' and asks Putin to help him win his election, it's all good. When Trump says he doesn't think Putin's hackers changed any votes, it's treason. Right.
What would happen if we started calling Ai 'Fake Intelligence' ... Fee Fi Foes?
As I understand the current fashions, AI has a fatal flaw: it's result is non-deterministic ... noone can be sure how it arrives at an answer. That might be okay for face recognition, or 'computer art' ... but for locating potential automobile collision victims, or deterministically arriving at a sound treatment for a patient? Wrong model.
I'd guess that the 'expert systems' of 20 years back outperform neural nets. Their logic trees were scrutable.
"You must try to forget all you have learned. You must begin to dream." -- Sherwood Anderson