Slashdot Mirror
IBM Watson Reportedly Recommended Cancer Treatments That Were 'Unsafe and Incorrect'
An anonymous reader quotes a report from Gizmodo: Internal company documents from IBM show that medical experts working with the company's Watson supercomputer found "multiple examples of unsafe and incorrect treatment recommendations" when using the software, according to a report from Stat News. According to Stat, those documents provided strong criticism of the Watson for Oncology system, and stated that the "often inaccurate" suggestions made by the product bring up "serious questions about the process for building content and the underlying technology." One example in the documents is the case of a 65-year-old man diagnosed with lung cancer, who also seemed to have severe bleeding. Watson reportedly suggested the man be administered both chemotherapy and the drug "Bevacizumab." But the drug can lead to "severe or fatal hemorrhage," according to a warning on the medication, and therefore shouldn't be given to people with severe bleeding, as Stat points out. A Memorial Sloan Kettering (MSK) Cancer Center spokesperson told Stat that they believed this recommendation was not given to a real patient, and was just a part of system testing.
According to the report, the documents blame the training provided by IBM engineers and on doctors at MSK, which partnered with IBM in 2012 to train Watson to "think" more like a doctor. The documents state that -- instead of feeding real patient data into the software -- the doctors were reportedly feeding Watson hypothetical patients data, or "synthetic" case data. This would mean it's possible that when other hospitals used the MSK-trained Watson for Oncology, doctors were receiving treatment recommendations guided by MSK doctors' treatment preferences, instead of an AI interpretation of actual patient data. And the results seem to be less than desirable for some doctors.
According to the report, the documents blame the training provided by IBM engineers and on doctors at MSK, which partnered with IBM in 2012 to train Watson to "think" more like a doctor. The documents state that -- instead of feeding real patient data into the software -- the doctors were reportedly feeding Watson hypothetical patients data, or "synthetic" case data. This would mean it's possible that when other hospitals used the MSK-trained Watson for Oncology, doctors were receiving treatment recommendations guided by MSK doctors' treatment preferences, instead of an AI interpretation of actual patient data. And the results seem to be less than desirable for some doctors.
Really where is the there, here ? You'll have doctors frequently dispute what the correct treatment is and with diseases like cancer it doesn't help that the best you can often do is offer a statistical improvement of someone's chances.
Far better that more people can afford treatment faster than this remain the province of the priesthood.
Of course it did, I mean no racial implications here, but Indian medicine is different than traditional, well done science, Western medicine. So of course there were cross cultural boundary recommendations made, and perhaps those can serve to broaden the perspective of the white male dominated medical field.
... but it will the patient. Is that a problem?"
Doctor (shaking his head): Yes, Watson... that is a problem.
(Who trained Watson for this job anyway?)
CUR ALLOC 20195.....5804M
The people responsible for mistraining a computer which can replace doctors: doctors
The purpose of such a tool should be to make suggestions that a doctor may not consider themselves. It should be up to the doctor(s) to vet the suggestions or leads before any treatment is actually rendered. A doctor would have to be born in Stupidville to accept bot suggestions as-is.
Table-ized A.I.
As the old engineering saying goes. Garbage In, Garbage Out.
This is a statistics-driven automaton that has zero insight or understanding. Calling it "AI" is a marketing lie, even if the AI field has given in and calls things like this "weak AI", which is the AI without "I". As such, this machine can find statistical correlations, but it cannot do plausibility checks, because that requires insight. It cannot do predictions either, because that also requires insight. The real strength of Watson (and it is quite an accomplishment) is that unlike older comparable systems, you can feed the training data and the queries into it in natural language. This means you can train a lot cheaper, but at the cost of accuracy, as the effect described in the story nicely shows.
It is time for this "AI" hype to die down. All it shows is that many people do not chose to use what they have in general intelligence and rather mindlessly follow a crows of cheer-leaders.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I'll take Incorrect Diagnosis for $200, Alex.
But is Watson cheaper than a doctor?
The survival rate for lung cancer can sometimes be as low as 4% over five years. Even if the drug combination had a 90% chance to outright kill the patient it might raise their overall chances of survival enough to actually be worth the risk. Based on what I know about lung cancer dying from severe hemorrhaging could be preferable to the relatively slow agonizing death some experience otherwise, especially if your overall chances of survival are higher.
Many women have had them over the year from a GP or Gyno that enjoyed their job a little too much.
I know several large breasted women that seemingly need a Totally Unnecessary Breast Exam every time they're in for even something like a sore hand.
How many human doctors did the same or worse?
Mostly random stuff.
In practice the term "AI" is vague and continuous rather than a Boolean designation ("is" versus "is-not"). The term is not worth sweating over. The exception may be if you are making a big purchase and/or investment based on something being "AI". In that case, inspect it carefully rather than assume something with "AI" is smart and/or useful. But that's good advice for any significant purchase: test drive it & ask detailed questions rather than rely on the brochure.
Table-ized A.I.
...is this: "A Memorial Sloan Kettering (MSK) Cancer Center spokesperson told Stat that they believed this recommendation was not given to a real patient, and was just a part of system testing."
Isn't this the kind of thing that testing is designed to uncover? It sounds to me like at least this part of the process is working, unlike the asshole who fed the model "fake data".
It just wanted to help impose pro-Darwinian responses to malformed genetic abnormalities.
Next up: self-driving cars that crash on purpose because their passengers sing songs the AI hates.
-- Tigger warning: This post may contain tiggers! --
test data provide test results.
So the data fed to train Watson wasn't from actual cases? Why does it matter what the computer prescribed, then? The system that is Watson is only as good as the data you feed it. Feed it fake information, get not even wrong results. Sounds more like a smear campaign,
intentionally designed to fail, and certainly not an experiment designed to measure Watson's recommendations against actual doctor recommendations.
Here's a better idea...
Feed the damn thing actual patient records with everything included from first immunization to the patient's ultimate death. If you are looking to see if there are any correlations that humans haven't already made you need to feed that sucker as much data as is inhumanly possible and then let it do the work.
What we have now is a pseudepigrapha of Watson's capabilities. Sure the results are from Watson, but they are not what Watson would do if given accurate, real life data to work with. They made a forgery of the system and put Watson's name on it.
Shady, bro. Shady...
When the only tool you have is a claw hammer every problem starts to look like the back of someone's skull.
An AI can only be as good as the data used to train it. The article pointed out that Watson was trained using what was possibly based as much on objective data as much as it was on subjective preferences of the physicians that fed it data.
I recall reading an article about someone doing a study on medical procedures done throughout the USA and they noticed "hot spots" of procedures being done in certain areas. What they found was that in these places they'd see physicians that would recommend procedures out of personal preference. One example was a an area with a lot of tonsillectomies, because a physician felt that any throat infection meant the tonsils had to come out. Another area had an elevated number of hysterectomies, because a physician felt that post-menopause women had an elevated risk of developing cysts and cancers on the uterus. The article went on to say that while such treatments may be unusual no one was willing to consider this malpractice.
So, Watson recommended a treatment for someone that might aggravate an existing problem of severe bleeding. Is this bad coding for not taking this into account? Or, is there a physician that entered such a prescription for their patient with similar symptoms? It's real difficult to second guess a physician. It's real easy to second guess the computer. Even if both the computer and the human came to the same recommendation for treatment.
I am armed because I am free. I am free because I am armed.
I know the probability is extremely low, but what if Watson detected a pattern nobody else was capable of and actually that combination works?
It baffles me that Watson is being considered for anything beyond a consultation at this juncture.
Recommending treatments? Right...
The soonest case scenario, barring the lawyers and insurance companies approve, is that Watson is ONLY integrated as parallel consultation with the standard human interpretation. As much as IBM, and whoever, like to 'tout' Watson as a breakthrough in medicine, the possible soon to be replacement for Radiologists, diagnosticians and the like... We aren't even 5 years off with this being consistently integrated.
Hospitals can barely keep up, with improving, replacing their existing tech. And they want to introduce 'AI' in diagnostic medicine in present day? Maybe if you're a research hospital funded by Bill Gates, even Johns Hopkins. Maybe! For the rest, the state, college hospital systems? No chance anytime soon.
You mean imposing sanctions, killing hundreds of Russian soldiers, giving $200 million in weapons to the Ukraine, expressly rejecting Russia's takeover of the Crimea, pushing to put US troops and their missile shield into Poland, increasing fracking to drive down the price of oil, trying to force Europe to stop buying Russian gas and increase their militaries...?
Trump has already done more to oppose Russia than Obama ever did - Obama didn't have the guts to enforce his own "red-line" in Syria.
But when Obama makes nice with a Russian 'reset' and asks Putin to help him win his election, it's all good. When Trump says he doesn't think Putin's hackers changed any votes, it's treason. Right.
The engineers will work on making sure it gives the correct diagnosis in a later update.
Thankfully, physicians are double checking. Doctors resistance to tech has kinda been a PIA, but here, they are saving lives.
Google around and you will see pages of hype about Watson published by public relations and regurgitated by filler web sites like this dump from 'Business Insider': "IBM's Watson Supercomputer Is Becoming The Best Cancer Doctor In The World" http://www.businessinsider.com/ibm-watson-cancer-2013-10
Real journalists who dug deeper found the truth about Watson to be very different:
>@blindseer: An AI can only be as good as the data used to train it.
You miss something: As good as the data used to train it AND THE PROGRAM ITSELF
What would happen if we started calling Ai 'Fake Intelligence' ... Fee Fi Foes?
As I understand the current fashions, AI has a fatal flaw: it's result is non-deterministic ... noone can be sure how it arrives at an answer. That might be okay for face recognition, or 'computer art' ... but for locating potential automobile collision victims, or deterministically arriving at a sound treatment for a patient? Wrong model.
I'd guess that the 'expert systems' of 20 years back outperform neural nets. Their logic trees were scrutable.
"You must try to forget all you have learned. You must begin to dream." -- Sherwood Anderson
It must have been fed stupid people twitter posts or conspiracy crap. Now its just as useless as random uncles opinion on how magnet therapy or bee stings can cure [insert random disease here].
Did you consider the company selling WATSON might be out to make a buck too?
"IBM employs 7,000 people in its Watson health division and sees the industry as a $200 billion market over the next several years."
"IBM is investing $1 billion in its Watson supercomputer in hopes of building a $10 billion business within a decade."
Evil doctors versus Altruistic IBM. Jellomizer has it all figured out.
They found "multiple examples of unsafe and incorrect treatment recommendations". How many exactly, what's the %? What's the relevancy of the "incorrectness" (totally, or mildly?). Doctors have to protect their interests and, probably, discredit AI, thus any mild error is to be publicized. Similar to complaints against Airbnb. Airbnb does close to a million rentals a day, and when an infinitesimal part of that (twice a year) makes trouble, it's largely publicized.
Slashdot, fix the reply notifications... You won't get away with it...
Right now it's still in a early learning proces, and it's a tool to help doctors. So what if it, at this point in development, makes the unsafe/incorrect treatment? It's not like doctors are right all the time, and doctors also have been well know to prescribe wrong treatments. Or, maybe the system did know about it, but calculated the risc factor of the patient dying anyway if he didn't get treatment.
But we're still at the beginning of having AI determine stuff like this, and yet Watson is already very well on the way to becoming better than most doctors, so let's see how Watson handles it in ten years, maybe by then I'd rather have Watson diagnose me than a real doctor..
Omg you're hilarious.
Watson just needs a big data cache of real-life human deaths to learn how to cure cancer.
You have a good point there. It seems Trump's words regarding Russia are conciliatory and diplomatic, yet his actions to stand up to them are far stronger than the previous administration. It appears that people who care about words more than actions are quite concerned.
Your post shows that you have no counterpoint to Trump's strong actions against Russia.
You mean strong actions like opposing sanctions and only enforcing them when the legislative branch had enough votes to overturn a veto?
Unless you combine it with dilaftin.
Which any first-year should know is
the standard prep medication your patient
was taking before surgery. Your patient
should be dead.
-Dave
For the second time today we see evidence that the poor excuse for AI they keep trotting out, in this case probably the most advanced version of it, even, is crap. I maintain that without understanding how a biological brain actually is able to think, there's no way these throw-it-at-the-wall-and-see-if-it-sticks guesses at an approach are going to ever be real AI -- and since we don't have the instrumentality to really truly see how a biological brain works, and map it's connections, in a living subject, we'll have to wait for that technology to be invented before we'll have any chance at real artificial intelligence. All your so-called 'deep learning algorithms' just don't cut it and never will; at best they're a small part of the real approach, one component in a vast system of interconnected systems that we haven't even scratched the surface of how they all work yet. And some of you people want to entrust your lives and the lives of your families to these. Madness.
Skynet has begun
"You'd have to show that he was being directly influenced by a foreign official or head of state"
All the moves he's made in Russia's favor and the disgusting sycophancy he's shown around Putin is raising and should raise a lot of questions.
It's called diplomacy.
I'm sorry if you think wanting to maintain good relations with our allies (yes, Russia is our ally) is bad. I'm sorry if you think peace between the Koreas is bad.
Most of those things (protests over Crimes, sanctions, aid to Ukraine, a proposal to put missiles in Poland) also happened under Obama.