Slashdot Mirror
AI Better Than Dermatologists At Detecting Skin Cancer, Study Finds (cbsnews.com)
An anonymous reader quotes a report from CBS News: For the first time, new research suggests artificial intelligence may be better than highly-trained humans at detecting skin cancer. A study conducted by an international team of researchers pitted experienced dermatologists against a machine learning system, known as a deep learning convolutional neural network, or CNN, to see which was more effective at detecting malignant melanomas. The results? 'Most dermatologists were outperformed by the CNN,' the researchers wrote in their report, published in the journal Annals of Oncology. Fifty-eight dermatologists from 17 countries around the world participated in the study. More than half of the doctors were considered expert level with more than five years' experience. Nineteen percent said they had between two to five years' experience, and 29 percent had less than two years' experience. At first look, dermatologists correctly detected an average of 87 percent of melanomas, and accurately identified an average of 73 percent of lesions that were not malignant. Conversely, the CNN correctly detected 95 percent of melanomas. The study has been published in the journal Annals of Oncology.
How did CNN compare in bedside manner and comforting during the examination or biopsy?
Great minds think alike; fools seldom differ.
When a human doctor makes a mistake, they might learn from it and know better for next time.
When an AI makes a mistake, every single system connected to the network might learn from it and know better for next time.
And once an AI reaches superhuman levels of performance, it's safe to assume it will stay better.
I am hopeful there will be some rapid advancement in this field.
-I only code in BASIC.-
Years of schooling only to be replaced by a tech and AI (if a tech is even needed for imaging)...
Where can I send my dick pics... I mean epidermal selfies?
"AI better at something" is likely to become very common., in many fields.
Slashdot, fix the reply notifications... You won't get away with it...
Oddly, the original publication calls out the use of Google Inception v4 CNN in the Methods, but the CBS News article doesn't mention it at all.
Ummm, Jon, aren't you supposed to be dead...? - Otter(3800)
If my father-in-law's MD would have had such a system handy, then maybe he would have been diagnosed much earlier for melanoma on his foot sole. As it was, the doctor thought it was something else, and he wasn't treated until it really was too late.
The advancement is not that it is better than a dermatologist, but that it can be added to a system that is easier to access than a dermatologist. We really need the dermatologist after the diagnosis for therapy and treatment.
The obvious conclusion is therefore not to train more doctors to correctly diagnose melanomas but to have them photograph parts of your body and submit the pictures to an IT system which will in turn deliver a diagnosis. While this may be beneficial for a health system in general (not necessarily the US, where the CNN diagnosis will of course be an order of magnitutde more expensive than traditional methods...) it may lead to less well trained physicians in front of the patient. Which may sound like technophobia is in fact happening, or rather has been happening for quite some time. Many orthopedic and trauma surgeons solely rely on imaging systems for diagnosis, while older clinicians or those trained in less advanced health systems could perform reliable first diagnosis with conventional means. You can stilll witness this in parts of Germany with physicians trained in the GDR.
In theory, modern technology should supplement experience when in fact it far too often replaces it, increasing the overall financial burden to the system.
I feel so sig.
Conversely, CNN detects truth in the news 0% of the time...
When an AI makes a mistake, every single system connected to the network might learn from it and know better for next time.
You know the converse is true as well. If they make a mistake ALL the machines will make the same mistake until it is corrected. With a human doctor that isn't generally true.
Years of schooling only to be replaced by a tech and AI (if a tech is even needed for imaging)...
That's like arguing that a code library and compiler can replace a programmer. It might change the tasks they have to deal with but it doesn't eliminate the job. If you think something like this is going to replace dermatologists you have no idea what they actually do.
Where can I send my dick pics... I mean epidermal selfies?
I think this says everything we need to know about you.
The obvious conclusion is therefore not to train more doctors to correctly diagnose melanomas but to have them photograph parts of your body and submit the pictures to an IT system which will in turn deliver a diagnosis.
No visual IT system nor any doctor can definitively diagnose a melanoma on the skin. They can have a strong suspicion and a good dermatologist will be right most of the time but they cannot be certain in all but the most obvious of cases. Even then the tissue has to be biopsied and sent to pathology for any sort of definitive diagnosis. There they have stains, genetic markers and other tools to figure out what is growing on the patient. Furthermore a definitive diagnosis is not always possible because we have no unambiguous test for all forms of melanoma nor do we even have an unambiguous set of criteria in some cases. There is an alarming amount of gestalt in the process. Some are fairly straightforward and others are nigh impossible to diagnose. Melanoma is challenging because it can appear in a variety of forms because of it's genetic origins. It can look near indistinguishable from many other types of lesions and the genetic tests and stains and other tools we have don't always give a clear answer.
The simple fact is that in many cases a diagnosis is really just an informed guess based on the probabilities. We're saying effectively that there is an X% chance that this is melanoma so we should treat it as if it is just to be safe.
Source: my wife is a dematopathologist so I get to hear about all this stuff daily.
The thing with diseases that get regularly checked, is that it doesn't make *that much* importance if they are missed on the first check.
5% is a large percentage, but some of these could be picked up by the dermatologist supervising the exam (if there's one on the premice)
and some of those 5% will eventually be picked up during next year's check, or the year after that.
(so 12 or 24 months later, which is still within the 28 months median time before metastasis, at which point the disease turn fatal.
Meaning that we're considering 0.05 ^ 3 [people missed after 3 tests in a row spread over 24 months] * 0.5 [portion of them who potentially developped metastasis] = 6.25 per million.
That is still a lot of potentially future dead cancer patients, but that's a lot better than no testing at all).
---
NOTE:
unlike benign skin features (birth marks, whatever),
malignant lesion change gradually over time (that, per se, is one of the criteria used by dermatologists).
so even if you run then exact same CNN on the skin picture the year after that, that CNN will definitely not see the exact same skin picture and might actually detect the cancer this time.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Dermatologists don't diagnose melanoma on the skin. They might suspect melanoma but what they do is biopsy the sample and send it to pathology where they can run a variety of tests (genetic markers, stains, etc) and look at the tissue morphology carefully under a microscope. The pathologist then gives an evaluation of the probability that this is indeed a malignancy. Melanoma's are sneaky and can resemble a wide variety of other conditions, many of them benign and there is no dermatologists or AI system that can get them right 100% of the time. Why? There is NO definitive test for all forms of melanoma nor is there any definitive criteria. In many cases it is just a probabilistic guess as to how likely this is to be melanoma or any other lesion. There is an alarming amount of gestalt in many of the diagnosis.
So the dermatologist doesn't have to be right, they just have to not have a false negative and cut out an adequate amount of tissue to give the pathologist a chance to make a diagnosis. They SHOULD be cutting out lesions that turn out to not be melanoma. So it doesn't matter if the AI is slightly better at guessing whether a lesion is melanoma or not on the skin because if the doctor even suspects it might be melanoma he/she should be biopsying the lesion anyway. Neither the doctor nor the AI actually knows what it is until pathology has a chance to look at it.
(Source: my wife is a dermatopathologist and does this stuff for a living)
So, how do they call doctors with 20-30 years of experience?
"AI better at something" is likely to become very common., in many fields.
But that doesn't always mean what people think it means. In this case for instance it doesn't matter if the AI is marginally better at guessing whether a lesion is melanoma from a picture because that isn't how melanoma is actually diagnosed. If the doctor even suspects a small chance of melanoma they are going to biopsy the tissue and send it to pathology where the pathologist tells them what it really is to the best of out knowledge. No dermatologist is going to treat a melanoma without a biopsy. The only important detail here is that the number of false negatives be as small as possible. When it doubt, cut it out. There should be some amount of false positives if the doctor is doing his job correctly. Obviously we want as few of these as possible but it's not the critical issue.
Technology like this isn't going to replace dermatologists. It's just going to become a supplemental tool to help ensure consistency of care and to help ensure more accurate results.
I think that it's the canary in the coal mine for many medical fields.
I think you are worrying about it more than is justified. My wife is a pathologist so I'm watching this issue closely but so far the net benefit seems to be positive.
Image processing neural nets are getting more powerful and more accessible for hospitals and (more importantly) large hospital networks.
I see little evidence we are danger of them getting powerful enough any time soon to start seriously denting the number of doctors needed. I think they will impact how the doctors do their job (and that's probably good) but mostly in the sense of removing a lot of needless busywork and improving quality of care.
This expands well beyond dermatologists. No reason why similar image processing techniques can't be used in radiology, reducing a health system's need to hire more radiologists. Or echocardiogram and electrocardiogram interpretation, freeing up the time of cardiologists (so less cardiologists need to be hired in the future).
I worked in a radiology clinic about 15 years ago where they were using this sort of tech to help with diagnosis. It helps the radiologist do their job better. It doesn't replace the radiologist. (or pathologist or cardiologist etc) To reduce head count you need to have it have enough of an impact to be a step function. I don't think we are in any real danger of that happening any time soon. Even if we did though it will just change some jobs. My wife is an AP/CP pathologist so instead of looking through a microscope most of the day she might end up looking at a monitor or even reading reports more like a clinical pathologist. That's not a bad thing, it's just different.
If you read the published research paper, this is what you will learn: When it comes to diagnosing the cancer based on images alone, AI outperforms all doctors, except for experienced experts (there humans has a slight advantage). When you give the human doctors additional information that they'd normally have in a clinical setting (e.g. patient's age, sex, etc) then all doctors' performance improves and they become comparable to the AI in diagnosing *if* you have skin cancer. HOWEVER, even then AI holds a significant advantage in figuring out exactly what kind of skin cancer you have.
Since the doctor will send sample biopsies to a lab for further analysis, this latter advantage may not be a big deal. But either way, this is a great tool to lower the costs and allow for telemedicine.
Think of the sci fi script "tax" side on this for saving a nations "free" health budget.
A doctor uses the scan to see of the patient qualifies for tests.
Computer always says no. The patient as presented is in good health.
Thats no free referral to the needed expert.
No appointment is made to see an expert.
No free pathology is done. No free pathologist to look at the results.
No results to send back to the doctor.
The patient goes home after a full "medical" thinking they had an actual expert look at their condition.
That the scan was presented as a real teleconference with an expert.
A pathologist starts doing reviews and notices the drop in expensive tests and use of stains.
The AI reports the peer review as outside hacking and alerts the government.
The AI is further expanded to offer a simple yes/no to doctors for further investigation into many other conditions. The once expanding heath budget is saved.
That would make a good fictional sci fi movie/series.
Some form of score on citizens to see if they get to see an expert?
Domestic spying is now "Benign Information Gathering"
Check out figure 3 in the article. They report that their CNN's AUC was third when compared with other algorithms from the 2016 International Symposium on Biomedical Imaging (ISBI) challenge. As they acknowledge in the Results: "The CNN scored results close to the top three algorithms of the ISBI 2016 challenge." While the differences in AUC were in the 2nd or 3rd decimal places, another headline for the article is "CNN developed in 2018 almost matches performance of algorithms developed two years ago." Not as good on the clickbait scale, but hey...
After all, a number of the pathology steps involve looking at things closely and pattern matching, which are the same types of things being done by the AI being discussed.
Vision based expert systems (I wouldn't really call them AI) will certainly get used someday though there are a lot of technical challenges to get through before this is possible. There is much more to pathology than just pattern matching however.
I don't expect it to happen any time soon but I can certainly envision a system in 15-20 years that would have a nurse practitioner scan a patient, take biopsies of flagged spots and feed the biopsies into a "pathologist in a box" machine that would do all the checks required - probably down to the DNA level.
This already happens today. It's called Clinical Pathology. Every time you have blood drawn (for example) that tissue gets sent to the clinical pathology department. The tissue is processed through a machine which spits out a report. The pathologist then evaluates the report and communicates results to the clinician. Most of these reports are fairly uninteresting but some will be require detailed scrutiny. The job of the pathologist is still to make diagnosis but also to manage the laboratory. Anatomic Pathology (the folks who look through microscopes) will to some degree begin to resemble Clinical Pathology over time. No it won't be a nurse practitioner but rather a specialized pathology assistant or histotechnologist or similar doing the job. A lot of pathologists are dual certified in both AP and CP.
A lot of diagnosis also eventually won't require pattern matching but rather will rely on DNA tests or similar. A lot of stains and other tests they use today do this.
Moving the bulk of the cost of diagnosis to a capital expense rather than a labor expense could eventually drive down medical costs tremendously - once we have machines that can do most of the work, the relentless pace of technology and process improvement will make the machines cheaper and cheaper.
In theory yes but in practice it will be harder to drive out the labor costs than you think. There is a LOT of expertise that goes into slide preparation, much of which is challenging to automate beyond a certain point. The people that do this generally have a 4 year college degree specific to this work. Then turning that into a machine readable form is another big challenge. The main pathologists still use microscopes instead of computer screens isn't because they cannot scan the slides but rather because it is not economic to do so in most cases. There definitely is room to improve costs though one should be careful about this because it's really easy to prioritize speed and volume over quality. (many pathology labs do this unfortunately)
Those elites who get their reputation by having read the book‘ unlike the unwashed masses, like Doctors, Lawyers, etc
One huge difference between dermatologists and all medical professionals, and other workers, is that they will still be pulling in huge salaries and working when we're automated out and roasting rats over garbage cans in the street.
Doctors and other health professions were smart and formed a professional organization. They're set for life - the professional body limits the supply of new entrants, ensures they're trained properly and buys legislation/regulations needed to ensure they're highly compensated and have secure jobs.
If you need to see a stark example, look at pharmacists. Pharmacists are highly trained and probably know more about drugs and delivery mechanisms than any doctor. Those that work in hospitals and specialty settings put their knowledge to work all the time, but there are also tons of retail pharmacists who do very basic tasks. Even while pharmacy techs do the basic jobs, every pharmacy needs a licensed pharmacist on staff. If they didn't have a professional organization paying to keep regulations in place, you can bet CVS, Walgreens and the health insurance companies would funnel as many paper bags of cash as necessary to lawmakers to make that requirement go away.
I know many people who don't even go to any doctor in a few years.
The challenge word for this post is: urinate
You are absolutely right of course on how proper and final diagnoses is established. The problem is, and your wife will probably agree, is that the initial diagnoses, you may even call it suspicion, is the most important because it determines whether follow-up tests will be conducted at all. My point is, actually two points are that 1) fewer first patient facing physicians (the front line if you will) will be taught and gain the experience to recognise melanomas (or the likelihood of it being one) in the first place and 2) automated testing will be more expensive in some health systems leading to diminished access to proper diagnoses and eventually treatment for many, thus exacerbating the original problem.
Technology is far more useful when it complements expertise and experience not when it merely replaces it.
I feel so sig.