Slashdot Mirror
IBM Pitched Its Watson Supercomputer as a Revolution in Cancer Care. It's Nowhere Close (statnews.com)
IBM began selling Watson to recommend the best cancer treatments to doctors around the world three years ago. But is it really doing its job? Not so much. An investigation by Stat found that the supercomputer isn't living up to the lofty expectations IBM created for it. It is still struggling with the basic step of learning about different forms of cancer. Only a few dozen hospitals have adopted the system, which is a long way from IBM's goal of establishing dominance in a multibillion-dollar market. And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care. From the report: The interviews suggest that IBM, in its rush to bolster flagging revenue, unleashed a product without fully assessing the challenges of deploying it in hospitals globally. While it has emphatically marketed Watson for cancer care, IBM hasn't published any scientific papers demonstrating how the technology affects physicians and patients. As a result, its flaws are getting exposed on the front lines of care by doctors and researchers who say that the system, while promising in some respects, remains undeveloped. [...] Perhaps the most stunning overreach is in the company's claim that Watson for Oncology, through artificial intelligence, can sift through reams of data to generate new insights and identify, as an IBM sales rep put it, "even new approaches" to cancer care. STAT found that the system doesn't create new knowledge and is artificially intelligent only in the most rudimentary sense of the term.
Google "IBM and the Holocaust"
And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care.
Are you seriously telling me that they sold a multi-million dollar machine and didn't even include a goddamn machine learning step to adapt to local variations? Aren't the IBM guys supposed to be experts? Or at least guys that know how to pick up a fucking phone and dial an expert?
This is the kind of rookie mistake I see in my undergrads...
I sure hope I'm reading this wrong, because it sounds like people might die from maltreatment over this.
At its heart, Watson for Oncology uses the cloud-based supercomputer to digest massive amounts of data — from doctor’s notes to medical studies to clinical guidelines. But its treatment recommendations are not based on its own insights from these data. Instead, they are based exclusively on training by human overseers, who laboriously feed Watson information about how patients with specific characteristics should be treated.
Ahh I guess I was wrong. There is no machine learning at all yet.
In the case of Watson for Oncology, those human operators are a couple dozen physicians at a single, though highly respected, U.S. hospital: Memorial Sloan Kettering Cancer Center in New York. Doctors there are empowered to input their own recommendations into Watson, even when the evidence supporting those recommendations is thin.
But hey, looks like the dying part could be correct. I only hope those doctors know what the variations across the world requires, because they will be giving recommendations both for japanese highschool girls and African village elders without even knowing it, and I don't think those groups have the same contextual issues.
Why do you think it would do that? Being better at getting the right diagnosis quickly isn't going to make things any cheaper. If you think otherwise then you're probably just a deranged pot head or vegan kidding themselves.
A lot of cancers are rare and difficult to deal with. Your random guy at "General Hospital" is going to have no clue. He won't even know well enough to throw the $10K per month med at the patient.
PubMed on steroids could actually be quite useful for the average doctor who's not at a world leading treatment center like Mayo.
A Pirate and a Puritan look the same on a balance sheet.
Except Mayo is not a 'world leading cancer treatment center'. It's one of literally thousands of places with excellent oncology teams.
Who look at the exact same data as Watson.
And come up with pretty much exactly the same result. Sans Watson.
The clueless doc at General Hospital doesn't even figure into this. If a patient has a complex / rare / difficult cancer they get referred to a regional cancer center. A lot of cancer treatments are pretty straightforward due to the large number of trials that have been done over the years. The databases have existed for decades and obviously are getting better and more complete over time. The real killer, so to speak, for Watson is that it could never really beat the industry standard 'tumor board' composed of various meat space biologic computers. Perhaps one day. When AI is actually a bit more than a marketing term.
Faster! Faster! Faster would be better!
Well, come on, there's a big difference between a program where the human sifts through the data and codes in rules of some form, and a program that sifts through the data and comes up with the rules (explicitly or implicitly).
I'm a huge skeptic about the _effectiveness_ of much-vaunted AI revolution in general based on everything I've done in machine learning and everything I've read and seen; and the Watson revolution specifically because of and what my friend who helped build Watson has told me, but you've got to admit that theoretically, the potential is there.
Can I mod something +1 Scary if it's true but I wish it weren't?
When captains of industry are talking about cancer treatment in terms of "establishing dominance in a multibillion-dollar market", does any rational person believe we're going to have an actual cure for cancer any time soon?
I've calculated my velocity with such exquisite precision that I have no idea where I am.
For something this critical, I would want the AI to explain itself. If an "expert" tells me that I need to that I need to take a pill for my cholesterol (intentionally choosing something rather minor), I will first ask her why this medicine and how does it work. I will expect to get a cogent explanation before paying for the drug. If I'm being asked to pay thousands of dollars for a cancer treatment, I'd expect someone to explain how the medicine works, and show me how it has been successful in other cases.
There are AI solutions that will show how they arrived at a recommendation. Intel has some AI that uses the feature as a selling point. Why would anyone just say, "Hmm? The computer says to give 'em hypocholoroacetiminophin. Wonder why? Where's my needle.", without getting an explanation?
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
No. There is no such thing as "AI" (yet). It's possible but probably not within our lifetimes. It's taken us well over 30 years since functional MRI came into the market and we're just beginning to understand what general areas of the brain are involved in doing "something", let alone individual neurons and synapses.
To claim the ability to "create intelligence" when we don't even understand the question yet is hubris (and salesmanship) on the side of IBM. Trust me, there will be several boom-bust cycles before AI becomes 'intelligent'.
Custom electronics and digital signage for your business: www.evcircuits.com
From the 2017 Amazon review it appears that all of the URLs in the text are now bad, after one year. Not sure how severely that impacts the value of the book but the reviewer only gave it two stars. So that "four star" average rating should be taken with a grain of salt. I certainly would not buy it new, you can get it for a little more than half the (allegedly) discounted Amazon new price.
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
First of all, Jeopardy Watson has almost nothing to do with the Watson product IBM is selling. That system was largely NLP based and the team disbanded afterwards. From what I can tell, the only thing still alive from that project is Apache UIMA. Watson isn't even a single product. It's a collection of about a dozen disjoint products with the word "Watson" in front of their name.
The current iteration of "Watson" is not interesting at all. Their machine learning portion is just SPSS. Their next-gen machine learning is just Apache Spark. Their UI to setup hardware and submit spark jobs is very unreliable. When you get an error, it's generally a "An error has occurred" or something just as useless.
Jeopardy Watson was interesting, but the big players are doing much more interesting things these days. Google and Microsoft have very good public machine learning and AI platforms. Amazon's is OK, nothing special. If you want to work at a lower level, Stanford maintains a set of libraries with implementations of their cutting edge algorithms. Especially their NLP group. Theirs are actually user-friendly compared to many other research level projects.
Most of the complaints are that you have to give Watson the answer first and then it gives you the question. Doctors were hoping for something the other way around.
Search on "oncologists would not have chemotherapy".
Boosting the bodies own defenses against cancer in various ways (including nutrition, intermittent fasting, immune-system tuning, etc.) is another approach at least generally without negative side effects -- wonder if Watson has been fed enough alternative data to recommend it (especially for prevention)?
Example: https://www.drfuhrman.com/lear...
"Cancer screening is promoted as preventive health, and while this may detect early forms of cancer so it can be treated earlier, it does not prevent the development of cancer and has minimal effects on reducing cancer deaths. A Nutritarian diet has the power to repair defects that can lead to cancer, detoxify carcinogens, cause cancer cell death, cut off blood supplies to growing tumors , and stimulate the immune system to recognize, repair abnormalities, and even fight and kill cancer cells. The vitamins, minerals, phytochemicals, and antioxidants found in a diet rich in vegetables, fruits, beans, nuts, and seeds is the key to prevention and even can play an important role in the treatment of various cancers."
Good luck with your own health care choices. It is hard to wade through all the conflicting information and conflict-of-interest. I wanted to make free software to help people make sense of conflicting health information -- but just not enough time given a need to earn money in other ways. What I could do with Watson hardware and that project's budget... (When I was at IBM Research around 2000 I proposed making an interactive display wall powered by an AI-like system to help people make complex decisions and better designs -- but as a contractor the idea did not go that far beyond a proof-of-concept with nine old Thinkpads that looked a lot like a Jeopardy screen, made when my supervisor went on a long vacation...)
https://web.archive.org/web/20...
https://github.com/pdfernhout/...
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.