Slashdot Mirror
Stanford Trains AI To Diagnose Pneumonia Better Than a Radiologist In Just Two Months (qz.com)
A new paper from Stanford University reveals how artificial intelligence algorithms can be quickly trained to diagnose pneumonia better than a radiologist. "Using 100,000 x-ray images released by the National Institutes of Health on Sept. 27, the research published Nov. 14 (without peer review) on the website ArXiv claims its AI can detect pneumonia from x-rays with similar accuracy to four trained radiologists," reports Quartz. From the report: That's not all -- the AI was trained to analyze x-rays for 14 diseases NIH included in the dataset, including fibrosis, hernias, and cell masses. The AI's results for each of the 14 diseases had fewer false positives and false negatives than the benchmark research from the NIH team that was released with the data. The paper includes Google Brain founder Andrew Ng as a co-author, who also served as chief scientist at Baidu and recently founded Deeplearning.ai. He's often been publicly bullish on AI's use in healthcare. These algorithms will undoubtedly get better -- accuracy on the ImageNet challenge rose from 75% to 95% in just five years -- but this research shows the speed at which these systems are built is increasing as well.
From what I have heard, most radiologists review large stacks of MRIs, CTs, or Xrays. They miss stuff all the time. AI wouldn't get tired, or be in a hurry.
SLOWER TRAFFIC KEEP RIGHT
It's becoming increasingly clear that no human can possibly have a functional grasp of all the knowledge required to make accurate diagnosis across all possible conditions. In the current model patients hope that they have something simple or obvious, and if not that their doctor can send them to the correct expert. This has far too many false positives and false negatives built in.
AI systems able to access comprehensive libraries of information are better at this type of work. Sure, I'd want an expert who can tailor search terms, accurately describe symptoms in a consistent manner, but for a number of years now I've been cheering every AI advance in clinical diagnosis. Can't come soon enough.
Volume also plays a role in training too.
To be considered trained, the radiologist usually have to go through several dozen of hundreds of MRIs, CTs and Xrays.
(They are not litteraly counted one by one. It's just accepted estimation that by the time the medical doctor finishes 5 years intership, he'll have seen enough example to be considered trained enough to have his radiologist certification).
The big advantage of the machine, is that instead of taking 5 years of internship training, you can have the neural net train by going through the 100'000 in one big computational jobs on the cluster.
There's this folk saying (Started by Malcolm Gladwell) that you need 10'000 hours of practice to become a master of anything.
The big benefits of AI is that these 10'000 hours don't need to happen in real-time anymore but can be simulated in a computer.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
They've had this sort of tech to read digital images for decades. Papnet is used to read pap smears since the mid-1990s.
http://www.lightparty.com/Heal...
It's becoming increasingly clear that no human can possibly have a functional grasp of all the knowledge required to make accurate diagnosis across all possible conditions.
No, it's much more subtle than that.
The things that they teach you at med school, are mostly rule of thumb. Simple algorithms that you can learn so you can get through your job without killing too many patients.
To diagnose pneumonia, there's a check list of things that you learn to look for and which allow you to say with some relative certainty whether or not the thing you're seeing is pneumonia.
Then there's the "clinical" experience. After seeing things for thousands of times, you start recognizing them automatically.
Like looking at an X-Ray picture and immediately "feeling" that there's "something funny" without even needing to start going through any checklist.
It's your old family doctor who can automatically guess the problem just be looking at how you walk like entering his office, or just based on the noises he hears outside, from the waiting room.
(Of course, there's some part of Sherlockian lightning fast thinking and deducting going on and a strong focus on very small otherwise imperceptible details.
But there's also some part of instinctive almost-sub-conscious gut feeling - how else would you know on *which* of the thousands of small imperceptible details to focus your inner Sherlock on ?)
That not something that you can learn by rote memorization during medical studies. That's something that comes slowly over time with practice.
The big advantage of neural nets, is that you can simulate all this experience inside a computer, by "simply" throwing hundreds of thousands of pictures at the neural into in a huge computational batches on the cluster.
AI systems able to access comprehensive libraries of information are better at this type of work. Sure, I'd want an expert who can tailor search terms, accurately describe symptoms in a consistent manner, but for a number of years now I've been cheering every AI advance in clinical diagnosis. Can't come soon enough.
In the end, as any other advances in the artificial intelligence field, you'll still need human oversight in the foreseeable future.
AI currently isn't replacing the job of actual doctors, as it is in providing more information faster to help taking the decision while taking all other informations on the way.
(Just like ECG able to propose diagnostic didn't cause the cardiologist specialist to disappear over-night).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
It's becoming increasingly clear that no human can possibly have a functional grasp of all the knowledge required to make accurate diagnosis across all possible conditions. In the current model patients hope that they have something simple or obvious, and if not that their doctor can send them to the correct expert. This has far too many false positives and false negatives built in.
AI systems able to access comprehensive libraries of information are better at this type of work. Sure, I'd want an expert who can tailor search terms, accurately describe symptoms in a consistent manner, but for a number of years now I've been cheering every AI advance in clinical diagnosis. Can't come soon enough.
The AI can also triage the scans into sets based on diagnosis which would allow the radiologist to verify the most serious cases or positive results first, and then validate the negative results. This would be better use of their time while still providing a check on the AI. Better still, negative results could be reviewed by a trained NP or PA and the questionable ones sent to an MD; saving money in the process without reducing the quality of care.
I'm a consultant - I convert gibberish into cash-flow.
The things that they teach you at med school, are mostly rule of thumb. Simple algorithms that you can learn so you can get through your job without killing too many patients.
Which, by the way, is going to be an irony completely lost on the "replace occurence of 'AI' with 'algorithm' and complain loudly" trolls that invariably pop-up on each machine learning article.
In, this case, it's the fresh med-school graduate who's using "an algorithm", and the AI which is most definitely relying on the intrinsic pattern-finding properties of any neural network / brain (be it in a biological real-world brain or a simulated one).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Doctors in radiologist help them program AI by doing all the work over the last half-century
I got taken to the emergency room with pneumonia, years ago. They thought I was faking.
Who wants to wait 2 months to be diagnosed with Pneumonia?
Pnumonia is pretty easy to see in XR or CT - this shouldnt suprise anyone - but it is an advance a tired overworked resident can miss even the blatantly obvious
I had unnecessary lung surgery 12 years ago because I was misdiagnosed with lung-cancer when all I had a mild fungal infection common in my area at the time called histoplasmosis. I’ve no doubt AI’s would prevent these kinds of mistakes. I also remember being told my chance of having lung cancer was 97% by the radiologist. I’m pretty sure as a young non-smoker he didn’t factor in any kind of bayesian statistics to arrive at this answer, but just looked a chart based on signal return from a PET scan. Hopefully these days at a minimum there would be some app that has them plug in relevant variables and does the bayesian analysis for them.
Trust me, recovery from invasive lung surgery is no picnic.
We don't get bent out of shape when machines read bar codes to price our grocery items instead of a human reading the price sticker. The machine is doing essentially the same thing, but far more accurately. The only difference is we take the price and refactor into a visual form easier for the machine to read. Imagine the accuracy if the Human had to read the bar code instead and remember the item associated with it.
Letter To Iran
There are no limits to science. Think how many jobs we will be able to eliminate in the next years! We may well be able to reach 99% human redundancy within two decades!
Is the AI allowed to know what medical insurance the patient has, in order to extend the recommended treatment just enough to keep all the hospital beds occupied? I swear, I've run into this. When I turned up with chest pain, and good insurance, they took three days of running me through all the expensive tests, including the machine that goes "ping!"
My own non-specialist explained it in 30 seconds with a good listen to my chest when I got to him. Pericarditis: there's a very characteristic rubbing sound, which the first nurse *heard* and, on reviewing their behavior, diagnosed correctly, but was not allowed to discuss with me, the pain can be quite alarming and feel like a heart attack, but it's aggravated by breathing: it's very characteristic and easy to diagnose *if you can hear it without gasping in pain yourself*. As best I can tell, they kept that nurse the hell away from me while a cardiac specialist eventually arrived to blow every possible resource and keep all the beds and fancy equipment busy with the "cardiac" bogyman.
The unwillingness of clinicians to provide feedback is a horrible, horrible sin of modern medicine. And I don't expect it to be improved by putting an AI in the mix, because the clinicians can and will refuse to provide that information.
It's becoming increasingly clear that no human can possibly have a functional grasp of all the knowledge required to make accurate diagnosis across all possible conditions. In the current model patients hope that they have something simple or obvious, and if not that their doctor can send them to the correct expert. This has far too many false positives and false negatives built in.
AI systems able to access comprehensive libraries of information are better at this type of work. Sure, I'd want an expert who can tailor search terms, accurately describe symptoms in a consistent manner, but for a number of years now I've been cheering every AI advance in clinical diagnosis. Can't come soon enough.
The thing is, it's not necessary for a medical AI to know everything or have a 99.99999% success rate, unlike an autonomous car which has to make split second decisions based on incomplete information.
A medical AI analyses a sample and says "Hey doc, this looks like Cancer", the doctor then has a look and determines if it looks like cancer and determines what extra tests are needed or if it's clear enough, goes straight to recommending a treatment. At the very least, it provides an extra set of eyes.
I understand most posters here are from the US and are used to paying through the nose for basic medical services, but for the rest of us having a medical AI in front of trained doctors will make things a hell of a lot easier and cheaper for the NHS. Simple diagnosis and treatments can be managed by the AI, things like a case of the sniffles or getting a sick note. Advanced cases or ambiguous things can be forwarded through to a human.
Calling someone a "hater" only means you can not rationally rebut their argument.
... two whole months for the AI to diagnose pneumonia, you'll be lucky to still be alive by then.
File under 'M' for 'Manic ranting'
Doctors are history
I doubt if any diagnosis is simple. I mean, the same Gub'mnt structure that approves the med-AI also approved removing guns while allowing rafts of Muzzi-wog/kfir/bantu/slant immigrants into Limeyland ... for all the evil done that ancient-now-corrupt kulture ... imagine the equivalent harm done to medical patient care.
You have to be careful that you haven't inadvertently over-trained the AI so that it's basically memorizing the individual data points rather than reaching generalizations. You can leave some out to test against, but this can inevitably be trained against in an evolutionary sort of way.
The only way to know for sure is to take the "finished product" and run it against brand new stuff it has never seen before, once.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
I've long said that the Turing test will be passed by machines not because machines get smarter but because people get stupider.
These days, radiologists spend something like 15 seconds reviewing an x-ray. No one hardly ever holds them accountable for doing a good job, only for how many tests they can read in a day.
Meanwhile, neural net algorithms (such as the ones used in the paper) are opaque as to HOW they made a decision.
Suppose you go in, and the "AI" says you have lung cancer. Your doc says "You gotta get half your lung removed within a week!" You say, "How does it know that?" Doc says "No one knows, but AI says so, and it was trained on some data, perhaps not from the genetic segment you are in."
The way to use such a machine is to screen AFTER the radiologist, and if it returns 'positive,' then the radiologist takes a LONG look at the data. Even then, radiologist will have confirmation bias.
Thanks for clarifying that it was not peer reviewed, but the appropriate term is definitely not "published" but rather "posted a preprint."
False negatives are dangerous if this was to be used to minimize human's effort (ie first line of analysis). ...and this is likely of how it was trained as they used known database of Xrays and correlate it with known positives.
Can they tell the probability of false negatives (ie missing a problem) given the training set?
So likely it could only be used in addition to human's analysis.
4wdloop
False positive and false negatives are both dangerous and hence need to be reviewed. I do not see how the AI can be used to reduce cost. It can increase accuracy and speed of positive detection when used in parallel with trained processionals. It's like having a second opinion.
4wdloop
Don't don't don't let's start...
False positive and false negatives are both dangerous and hence need to be reviewed. I do not see how the AI can be used to reduce cost. It can increase accuracy and speed of positive detection when used in parallel with trained processionals. It's like having a second opinion.
It's a matter of who review what; having an NP or PA review all the negative results first and only forwarding the questionable ones to an MD would be far cheaper than having an MD review all of them. Part of the problem, however, is the notion you must always use an MD rather than an NP or PA for the entry point into care or to handle cases. NP already do various types of care that MDs do, such as anesthesiology and psychiatry, and sometimes independently; it's a matter of training them to perform specific functions very well.
I'm a consultant - I convert gibberish into cash-flow.
How would we *know* whether the AI is better than humans? How do you *know* the AI's false positive and false negative rate without some kind of oracle to compare the result to? Can an AI algorithm somehow outperform the human-made classifications it is trained on, and if it appears to do so should we trust that result?
I'm not doing the usual Slashdot thing of assuming that experts are too dumb to see the objections that immediately occurred to me; I just think that people take a headline like above for granted when you can't really even understand what it is saying without a lot more information.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Sooooo...... apparently better that 9 people undergo unnecessary surgery, because the computer would make a mistake on the 10th also. If it's not perfect, then we'd better rely on the humans.
Remember, do not let the Perfect be the enemy of the Good
Same comment would apply to Self Driving Cars, which I'd bet you are also against.
Letter To Iran
Can I test this software anywhere?
I've got x-rayed to rule out pneumonia just two few weeks ago and got the image on CD. I'd love to give this thing a test run and see what it comes up with.
Orly?
The AI book that everyone should get is available for pre-order (April 23, 2018). "Artificial Intelligence For Dummies" by John Paul Mueller and Luca Massaron.
than "A" radiologist. How about "every" radiologist? Can it do that?
Even more complex you may not know what rules the AI is using to evaluate something.
Just as the old doctor with 30 years of experience, won't actually know how/why/which neuron are firing up when he gets his hunch.
But he can then Sherlock it to gets a reasonable justification why he should trust his hunch.
The same here : the AI give its best idea about the X-ray.
Then the doctor who's seeing the patient will combine this information with what the rest (patient's complaints, etc.) and make a diagnostic and take a therapeutic decision.
Maybe the AI's diagnostic makes 0% clinical sense, in which case the doctor will ignore it and switch to something else.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]