Slashdot Mirror
AI Better Than Dermatologists At Detecting Skin Cancer, Study Finds (cbsnews.com)
An anonymous reader quotes a report from CBS News: For the first time, new research suggests artificial intelligence may be better than highly-trained humans at detecting skin cancer. A study conducted by an international team of researchers pitted experienced dermatologists against a machine learning system, known as a deep learning convolutional neural network, or CNN, to see which was more effective at detecting malignant melanomas. The results? 'Most dermatologists were outperformed by the CNN,' the researchers wrote in their report, published in the journal Annals of Oncology. Fifty-eight dermatologists from 17 countries around the world participated in the study. More than half of the doctors were considered expert level with more than five years' experience. Nineteen percent said they had between two to five years' experience, and 29 percent had less than two years' experience. At first look, dermatologists correctly detected an average of 87 percent of melanomas, and accurately identified an average of 73 percent of lesions that were not malignant. Conversely, the CNN correctly detected 95 percent of melanomas. The study has been published in the journal Annals of Oncology.
When a human doctor makes a mistake, they might learn from it and know better for next time.
When an AI makes a mistake, every single system connected to the network might learn from it and know better for next time.
And once an AI reaches superhuman levels of performance, it's safe to assume it will stay better.
I am hopeful there will be some rapid advancement in this field.
-I only code in BASIC.-
And this is why healthcare in the US is so expensive.
Who cares? I want an accurate fast cheap diagnosis. Not a happy ending massage.
Oddly, the original publication calls out the use of Google Inception v4 CNN in the Methods, but the CBS News article doesn't mention it at all.
Ummm, Jon, aren't you supposed to be dead...? - Otter(3800)
I'm more wondering how it did on the identification of non malignant lesions.
I could identify 100% with an algorithm that just kicked out yes, but if I got 0% of the non malignant ones, it's not very useful.
It looks like doctors got 87% of the bad ones and falsely identified 21% of non malignant as bad, the computer getting 95% of the bad ones.
Or I completely misunderstood the summary because I don't know medicine.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
If my father-in-law's MD would have had such a system handy, then maybe he would have been diagnosed much earlier for melanoma on his foot sole. As it was, the doctor thought it was something else, and he wasn't treated until it really was too late.
The advancement is not that it is better than a dermatologist, but that it can be added to a system that is easier to access than a dermatologist. We really need the dermatologist after the diagnosis for therapy and treatment.
The obvious conclusion is therefore not to train more doctors to correctly diagnose melanomas but to have them photograph parts of your body and submit the pictures to an IT system which will in turn deliver a diagnosis. While this may be beneficial for a health system in general (not necessarily the US, where the CNN diagnosis will of course be an order of magnitutde more expensive than traditional methods...) it may lead to less well trained physicians in front of the patient. Which may sound like technophobia is in fact happening, or rather has been happening for quite some time. Many orthopedic and trauma surgeons solely rely on imaging systems for diagnosis, while older clinicians or those trained in less advanced health systems could perform reliable first diagnosis with conventional means. You can stilll witness this in parts of Germany with physicians trained in the GDR.
In theory, modern technology should supplement experience when in fact it far too often replaces it, increasing the overall financial burden to the system.
I feel so sig.
Who cares? I want an accurate fast cheap diagnosis.
Go read Yelp reviews for doctors. 95% of the bad reviews are because of rude office staff. The competence of the doctor is irrelevant to their rating.
Or I completely misunderstood the summary because I don't know medicine.
TFA is not much better. It is horrible journalism. It is unclear if the "AI" is actually better, with false positives and false negatives rates mingled together. It also seems to say that the humans and AI were shown DIFFERENT IMAGES, and that the humans were given additional information that the AI did not have. So the comparison of results may be meaningless.
The only thing that can be said with certainty is that CBS produces some garbage journalism.
Years of schooling only to be replaced by a tech and AI (if a tech is even needed for imaging)...
That's like arguing that a code library and compiler can replace a programmer. It might change the tasks they have to deal with but it doesn't eliminate the job. If you think something like this is going to replace dermatologists you have no idea what they actually do.
Where can I send my dick pics... I mean epidermal selfies?
I think this says everything we need to know about you.
The obvious conclusion is therefore not to train more doctors to correctly diagnose melanomas but to have them photograph parts of your body and submit the pictures to an IT system which will in turn deliver a diagnosis.
No visual IT system nor any doctor can definitively diagnose a melanoma on the skin. They can have a strong suspicion and a good dermatologist will be right most of the time but they cannot be certain in all but the most obvious of cases. Even then the tissue has to be biopsied and sent to pathology for any sort of definitive diagnosis. There they have stains, genetic markers and other tools to figure out what is growing on the patient. Furthermore a definitive diagnosis is not always possible because we have no unambiguous test for all forms of melanoma nor do we even have an unambiguous set of criteria in some cases. There is an alarming amount of gestalt in the process. Some are fairly straightforward and others are nigh impossible to diagnose. Melanoma is challenging because it can appear in a variety of forms because of it's genetic origins. It can look near indistinguishable from many other types of lesions and the genetic tests and stains and other tools we have don't always give a clear answer.
The simple fact is that in many cases a diagnosis is really just an informed guess based on the probabilities. We're saying effectively that there is an X% chance that this is melanoma so we should treat it as if it is just to be safe.
Source: my wife is a dematopathologist so I get to hear about all this stuff daily.
The thing with diseases that get regularly checked, is that it doesn't make *that much* importance if they are missed on the first check.
5% is a large percentage, but some of these could be picked up by the dermatologist supervising the exam (if there's one on the premice)
and some of those 5% will eventually be picked up during next year's check, or the year after that.
(so 12 or 24 months later, which is still within the 28 months median time before metastasis, at which point the disease turn fatal.
Meaning that we're considering 0.05 ^ 3 [people missed after 3 tests in a row spread over 24 months] * 0.5 [portion of them who potentially developped metastasis] = 6.25 per million.
That is still a lot of potentially future dead cancer patients, but that's a lot better than no testing at all).
---
NOTE:
unlike benign skin features (birth marks, whatever),
malignant lesion change gradually over time (that, per se, is one of the criteria used by dermatologists).
so even if you run then exact same CNN on the skin picture the year after that, that CNN will definitely not see the exact same skin picture and might actually detect the cancer this time.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
"AI better at something" is likely to become very common., in many fields.
But that doesn't always mean what people think it means. In this case for instance it doesn't matter if the AI is marginally better at guessing whether a lesion is melanoma from a picture because that isn't how melanoma is actually diagnosed. If the doctor even suspects a small chance of melanoma they are going to biopsy the tissue and send it to pathology where the pathologist tells them what it really is to the best of out knowledge. No dermatologist is going to treat a melanoma without a biopsy. The only important detail here is that the number of false negatives be as small as possible. When it doubt, cut it out. There should be some amount of false positives if the doctor is doing his job correctly. Obviously we want as few of these as possible but it's not the critical issue.
Technology like this isn't going to replace dermatologists. It's just going to become a supplemental tool to help ensure consistency of care and to help ensure more accurate results.
I think that it's the canary in the coal mine for many medical fields.
I think you are worrying about it more than is justified. My wife is a pathologist so I'm watching this issue closely but so far the net benefit seems to be positive.
Image processing neural nets are getting more powerful and more accessible for hospitals and (more importantly) large hospital networks.
I see little evidence we are danger of them getting powerful enough any time soon to start seriously denting the number of doctors needed. I think they will impact how the doctors do their job (and that's probably good) but mostly in the sense of removing a lot of needless busywork and improving quality of care.
This expands well beyond dermatologists. No reason why similar image processing techniques can't be used in radiology, reducing a health system's need to hire more radiologists. Or echocardiogram and electrocardiogram interpretation, freeing up the time of cardiologists (so less cardiologists need to be hired in the future).
I worked in a radiology clinic about 15 years ago where they were using this sort of tech to help with diagnosis. It helps the radiologist do their job better. It doesn't replace the radiologist. (or pathologist or cardiologist etc) To reduce head count you need to have it have enough of an impact to be a step function. I don't think we are in any real danger of that happening any time soon. Even if we did though it will just change some jobs. My wife is an AP/CP pathologist so instead of looking through a microscope most of the day she might end up looking at a monitor or even reading reports more like a clinical pathologist. That's not a bad thing, it's just different.
Then, after a few years of active training, the systems will probably be ready to work on their own.
That will (probably) never happen. More likely what will happen is that they will work in conjunction with the physician and the physician will gradually concern themselves more with results interpretation, management, and treatment. But no doctor is going to just let the machine go do its thing without any sort of oversight.