Slashdot Mirror
AI Better Than Dermatologists At Detecting Skin Cancer, Study Finds (cbsnews.com)
An anonymous reader quotes a report from CBS News: For the first time, new research suggests artificial intelligence may be better than highly-trained humans at detecting skin cancer. A study conducted by an international team of researchers pitted experienced dermatologists against a machine learning system, known as a deep learning convolutional neural network, or CNN, to see which was more effective at detecting malignant melanomas. The results? 'Most dermatologists were outperformed by the CNN,' the researchers wrote in their report, published in the journal Annals of Oncology. Fifty-eight dermatologists from 17 countries around the world participated in the study. More than half of the doctors were considered expert level with more than five years' experience. Nineteen percent said they had between two to five years' experience, and 29 percent had less than two years' experience. At first look, dermatologists correctly detected an average of 87 percent of melanomas, and accurately identified an average of 73 percent of lesions that were not malignant. Conversely, the CNN correctly detected 95 percent of melanomas. The study has been published in the journal Annals of Oncology.
Who cares? I want an accurate fast cheap diagnosis.
Go read Yelp reviews for doctors. 95% of the bad reviews are because of rude office staff. The competence of the doctor is irrelevant to their rating.
Or I completely misunderstood the summary because I don't know medicine.
TFA is not much better. It is horrible journalism. It is unclear if the "AI" is actually better, with false positives and false negatives rates mingled together. It also seems to say that the humans and AI were shown DIFFERENT IMAGES, and that the humans were given additional information that the AI did not have. So the comparison of results may be meaningless.
The only thing that can be said with certainty is that CBS produces some garbage journalism.
The obvious conclusion is therefore not to train more doctors to correctly diagnose melanomas but to have them photograph parts of your body and submit the pictures to an IT system which will in turn deliver a diagnosis.
No visual IT system nor any doctor can definitively diagnose a melanoma on the skin. They can have a strong suspicion and a good dermatologist will be right most of the time but they cannot be certain in all but the most obvious of cases. Even then the tissue has to be biopsied and sent to pathology for any sort of definitive diagnosis. There they have stains, genetic markers and other tools to figure out what is growing on the patient. Furthermore a definitive diagnosis is not always possible because we have no unambiguous test for all forms of melanoma nor do we even have an unambiguous set of criteria in some cases. There is an alarming amount of gestalt in the process. Some are fairly straightforward and others are nigh impossible to diagnose. Melanoma is challenging because it can appear in a variety of forms because of it's genetic origins. It can look near indistinguishable from many other types of lesions and the genetic tests and stains and other tools we have don't always give a clear answer.
The simple fact is that in many cases a diagnosis is really just an informed guess based on the probabilities. We're saying effectively that there is an X% chance that this is melanoma so we should treat it as if it is just to be safe.
Source: my wife is a dematopathologist so I get to hear about all this stuff daily.
The thing with diseases that get regularly checked, is that it doesn't make *that much* importance if they are missed on the first check.
5% is a large percentage, but some of these could be picked up by the dermatologist supervising the exam (if there's one on the premice)
and some of those 5% will eventually be picked up during next year's check, or the year after that.
(so 12 or 24 months later, which is still within the 28 months median time before metastasis, at which point the disease turn fatal.
Meaning that we're considering 0.05 ^ 3 [people missed after 3 tests in a row spread over 24 months] * 0.5 [portion of them who potentially developped metastasis] = 6.25 per million.
That is still a lot of potentially future dead cancer patients, but that's a lot better than no testing at all).
---
NOTE:
unlike benign skin features (birth marks, whatever),
malignant lesion change gradually over time (that, per se, is one of the criteria used by dermatologists).
so even if you run then exact same CNN on the skin picture the year after that, that CNN will definitely not see the exact same skin picture and might actually detect the cancer this time.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]