Slashdot Mirror


Deep Learning Algorithm Diagnoses Skin Cancer As Well As Seasoned Dermatologists (extremetech.com)

An anonymous reader quotes a report from ExtremeTech: Remember how that Google neural net learned to tell the difference between dogs and cats? It's helping catch skin cancer now, thanks to some scientists at Stanford who trained it up and then loosed it on a huge set of high-quality diagnostic images. During recent tests, the algorithm performed just as well as almost two dozen veteran dermatologists in deciding whether a lesion needed further medical attention. The algorithm is called a deep convolutional neural net. It started out in development as Google Brain, using their prodigious computing capacity to power the algorithm's decision-making capabilities. When the Stanford collaboration began, the neural net was already able to identify 1.28 million images of things from about a thousand different categories. But the researchers needed it to know a malignant carcinoma from a benign seborrheic keratosis. Dermatologists often use an instrument called a dermoscope to closely examine a patient's skin. This provides a roughly consistent level of magnification and a pretty uniform perspective in images taken by medical professionals. Many of the images the researchers gathered from the Internet weren't taken in such a controlled setting, so they varied in terms of angle, zoom, and lighting. But in the end, the researchers amassed about 130,000 images of skin lesions representing over 2,000 different diseases. They used that dataset to create a library of images, which they fed to the algorithm as raw pixels, each pixel labeled with additional data about the disease depicted. Then they asked the algorithm to suss out the patterns: to find the rules that define the appearance of the disease as it spreads through tissue. The researchers tested the algorithm's performance against the diagnoses of 21 dermatologists from the Stanford medical school, on three critical diagnostic tasks: keratinocyte carcinoma classification, melanoma classification, and melanoma classification when viewed using dermoscopy. In their final tests, the team used only high-quality, biopsy-confirmed images of malignant melanomas and malignant carcinomas. When presented with the same image of a lesion and asked whether they would "proceed with biopsy or treatment, or reassure the patient," the algorithm scored 91% as well as the doctors, in terms of sensitivity (catching all the cancerous lesions) and sensitivity (not getting false positives).

44 comments

  1. How are they cooked? by avandesande · · Score: 1, Funny

    With thyme and cumin I suppose.

    --
    love is just extroverted narcissism
  2. Typo by Anonymous Coward · · Score: 0

    Second "sensitivity" in last line should read "selectivity"

    1. Re:Typo by Anonymous Coward · · Score: 1

      "specificity"

    2. Re:Typo by Excelcia · · Score: 1

      Damned auto corral

    3. Re: Typo by Anonymous Coward · · Score: 0

      "synchronicity"

  3. As Well as Seasoned Dermatologists by Anonymous Coward · · Score: 5, Funny

    Wow, both cancer AND seasoned dermatologists!

    I'm sure it can diagnose the heavily seasoned dermatologists, but how does it handle the merely lightly seasoned?

    1. Re: As Well as Seasoned Dermatologists by Anonymous Coward · · Score: 0

      What kind of seasoning? Cajun?

    2. Re:As Well as Seasoned Dermatologists by Anonymous Coward · · Score: 0

      Came here to say this...

      I would think the Dermatologists could speak for themselves, wouldn't need a learning algorithm.

  4. Algorithms see what we see... by Dread_ed · · Score: 4, Interesting

    ...but they aren't programmed by evolution to disregard 95%+ of it. Pretty much the exact opposite actually. We tune, prune, select, and evolve these algorithms to do this one thing really well. Frankly, its a wonder humans can do as well as we do. A testament to our pattern matching skills, adaptability, and lack of immutable hard wiring in the 'ol thinky thinky bits.

    Something I saw that might be able to help humans get a step up on the algorithms, or actually amalgamate humans and computers, is this:
    Video Magnification

    Maybe cancerous skin lesions absorb slightly different light wavelengths? If so, magnification of the minuscule differences could pinpoint it. Fun to ponder.

    --
    When the only tool you have is a claw hammer every problem starts to look like the back of someone's skull.
    1. Re:Algorithms see what we see... by Anonymous Coward · · Score: 0

      Of course they are programmed by evolution to disregard 95%+ of it. What do you think the "deep learning" part of deep learning algorithm describes? The whole idea came about from understanding how the human brain processes information and trying to duplicate it.

    2. Re:Algorithms see what we see... by easyTree · · Score: 1

      Glad I clicked on your link :D
      Here's one of the related links: Detecting Pulse From Head Motions in Video.

    3. Re:Algorithms see what we see... by JoeMerchant · · Score: 1

      I think the point of this article is that the algorithm is at parity, or better, in sensitivity AND specificity.

      What I wonder is how much time and money it is going to take to get this out and helping people. The computer time is virtually free compared to human evaluators - we could have tanning bed like devices where you strip, get in, and get photographed all over with controlled lighting and perhaps some multi-spectral augmentation on the imaging, should be able to beat a trip to the dermatologist in terms of cost and effectiveness of screening already today, should be able to be provided as a free or minimal cost service to anyone who wants a screening the way that blood pressure stations are set up in grocery stores today.

      Sadly, I think it will be decades before this tech rolls out to its full potential.

  5. Lubax app by Anonymous Coward · · Score: 1

    The lubax.com app has been available for years.
    Maybe this will help with the FDA.

  6. Specificity not sensitivity by Anonymous Coward · · Score: 0

    - not getting false positives

  7. But can it cook my meal? by thesjaakspoiler · · Score: 1

    Be sure not to miss the AI-bandwagon!

  8. Re:That's nice; I don't give a fuck by Anonymous Coward · · Score: 0

    clearly AI are able to troll effectively.

  9. Re:That's nice; I don't give a fuck by Anonymous Coward · · Score: 0

    But it doesn't really care about hurting my feelings, so why bother?

  10. Lower health care costs by Anonymous Coward · · Score: 0

    A while back, I wanted to have some moles checked to see if they were cancerous. It took me two months and a couple days of time off work and hours of driving during to finally have a dermatologist spend 5 minutes looking at my moles - despite the fact that I live in the Los Angeles area. Now, maybe my health insurance provider (BlueShield of California) was just really bad. But making it that difficult to access healthcare means that I won't be able to get checked very often.

    If this deep learning actually works, it would be great to just snap a photo with a mobile phone, have the neural net do it's things, and then know whether it's worth another two months of hassle to get to see a dermatologist (again).

    And there are a lot of areas where technology is poised to dramatically improve the quality and cost of healthcare. But healthcare is one area where there's a compelling case to be made that government intervention is significantly driving up the cost and reducing quality and availability of healthcare. Both in terms of medical doctor training and drug patents the federal government imposes artificial and deeply flawed monopolies.

    Most of what Trump says about the problems of government regulation and intervention is utter nonsense. But, with healthcare, there's actually huge room for improvement in the way the US government regulates and intervenes in the healthcare markets.

    1. Re: Lower health care costs by Anonymous Coward · · Score: 0

      You forgot the punchline: were the moles benign or not?

    2. Re:Lower health care costs by BarbaraHudson · · Score: 2

      It's all insurance providers. Hopefully AI can detect them as a cancer on the system and take appropriate action, because human politicians with their eyes on campaign contributions sure as hell can't.

      --
      "Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
  11. Know your own Skin! by Gim+Tom · · Score: 5, Insightful

    I have had both Basal and Squamous skin cancers since the 1990's and keep a close watch on my own skin. If I see anything suspicious I have a note book where I keep a note of what I saw, when and where. In some cases I will take a close up picture of it. Both Basal and Squamous cancers tend to appear and go away when they are very small and by doing this I have a record of "something" reappearing in the same location. Following the old adage that once is happenstance, twice is coincidence, but three times is most likely enemy action I will call for an appointment with my dermatologist and show them my records or pictures. For the last ten years I found every skin cancer well before the dermatologist would have seen it during an annual exam.

    It did not used to be that way since for many years I had the same dermatologist or group and they got to know my skin about as well as I learned to. However, after that with almost yearly shifts in medical networks due to changes in insurance providers where I worked (always either the lowest bidder or highest campaign contributor), it got where I didn't see the same one twice until I got on Medicare. The patient-doctor relationship SHOULD be long term and more than just a diagnostic code and EMR's. I think it is going to get a LOT worse before it gets better so learn to know your own body and be assertive about your care.

    1. Re:Know your own Skin! by BarbaraHudson · · Score: 1

      So, a skin flicks for a cause :-) That's not a bad idea, taking pictures to keep track of whether things are growing or changing shape. Maybe you should submit it as a story on do-it-yourself health monitoring that could save your life.

      --
      "Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
  12. Re:That's nice; I don't give a fuck by BarbaraHudson · · Score: 0

    Doesn't matter if it's AI or not, not if it can take you job away from you. And it will - not just the assembly-line jobs, but the office jobs, the programming jobs, pretty much anything. You are replaceable. You will be replaced. You need to get over it - here, let me connect you with the robopsychiatrist.

    --
    "Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
  13. Great! by WolfgangVL · · Score: 1

    So now we can have phone-booth style human body diagnostic on every street corner.... and I'll still need to go to Mexico to be able to afford it myself.

    --
    You are being ripped off every second of every day, so that advertisers can help rip you off even more tomorrow.
    1. Re:Great! by geezer+nerd · · Score: 1

      Where will you go when the trade war between US and Mexico shuts down travel between the two countries?

  14. Mycin did this in the 1970s by aberglas · · Score: 1

    Could diagnose bacterial infections as well as human experts.

    Trouble was, needed to input the data correctly. Which means recognize symptoms.

    Also, automated essay marking does a better job than human markers, when compared to marks by experts.

    Trouble here is, we are comparing Artificial Intelligence with human stupidity.

    And "Deep Learning" is not a technology. It is a marketing term.

    http://www.computersthink.com/

    for a better assessment of what is real.

    1. Re:Mycin did this in the 1970s by easyTree · · Score: 1

      Why can't computers already think? Why has 60 years of research failed to produce a single intelligent robot? What has been learnt, what are the technically difficult problems, and when are they likely to be solved?

      Why isn't society already self-regulating? Why has 60 years of research failed to produce a single new form of government? What has been learnt, what are the technically difficult problems, and when are they likely to be solved?

    2. Re:Mycin did this in the 1970s by Anonymous Coward · · Score: 1

      Posting AC because I'm an ex-employee of a company that already has a product on the market that does this (though specifically targeting melanoma), with sensitivity and specificity results that rival the top dermatologists in the US. It's available in the US and Germany last I checked. There was news about the fracas they had with the FDA over the regulatory issues they had, if you pay attention to that kind of thing.

      MelaFind http://www.melafind.com/

      I'm goddamn horrified of skin cancer now. Not really because of my personal health, but because the modern diagnosis of it is a joke. The dermatologist's best tool, a dermatoscope, is a glorified camera with a magnifying lens. After you remove a lesion a histopathologist will examine the tissue for evidence of melanoma, but even then the same lesion sent to three different pathologists will get you three different answers. So most people cut off whatever the derm thinks is bad based on the opinion of a guy who might have taken a picture of it, and even after it's off you don't know for sure if it was malignant, but you sure as shit aren't going to risk it.

    3. Re: Mycin did this in the 1970s by Anonymous Coward · · Score: 0

      AI improvement is mostly caused by better computers. It was only recently possible to process enough data to make these things . And note that the best AI is still run on super computers not on desktop computers.

  15. Good Diagnosis by gringer · · Score: 1

    Deep Learning Algorithm Diagnoses Skin Cancer As Well As Seasoned Dermatologists

    "I'm sorry Sam, it looks like you have a potentially fatal condition."

    "What, skin cancer?"

    "No, it's worse than that. You are infected with a seasoned dermatologist".

    --
    Ask me about repetitive DNA
  16. Re: That's nice; I don't give a fuck by Anonymous Coward · · Score: 0

    You seem angry.

  17. Idiocracy by Hylandr · · Score: 1

    Ok, put one of these probes in your mouth, ear, and butt.

    Wait, Wait, wait *this* one goes in your butt.

    --
    ~ People that think they are better than anyone else for any reason are the cause of all the strife in the world.
  18. As well as 24 Dermotologists? by Anonymous Coward · · Score: 0

    Should we be impressed by this small sample size?

  19. Regulatory Approach by Goldsmith · · Score: 2

    Reading an article like this, I can just hear my regulatory affairs officer having a heart attack in my head.

    I realize that very few people here have ever had dealings with the FDA. The FDA regulates the interstate marketing of medical tools (including software) and drugs, everything they do comes from that core mission and authority. Press releases and statements are pretty central to that mission. You should try to limit press on your product to what the FDA agrees you've proven. Depending on your views of the government, medical ethics, and your risk tolerance, "should try to" in that statement might be "must" or "should pretend to."

    Software focusing on "health" isn't really regulated, but "diagnostic" means this is medical. If you think they made a diagnostic tool for skin cancer, then they may have a problem when it comes time to talk with the FDA. They haven't shown that they have a diagnostic yet, that's the point of the last quote in the article, but that quote is in FDA-speak while the rest of the article is less formal sounding. (They've done what's called a retrospective study, which is at most half of what is necessary.)

    Generally, the authors on the paper are smarter than this. Here's an example of an article a Stanford Dermatologist usually contributes to. Note that the Dermatologist quoted in that article is also quoted in TFA. Note the difference in tone of the publication, the whole thing is in FDA-speak. Yes, it's super boring. It's also not going to give anyone at the FDA a reason to hold up an application for marketing prior to approval.

  20. Well done! by SlithyMagister · · Score: 1

    That is truly amazing.
    It is difficult to diagnose a dermatologist.

  21. Sensitivity and sensitivity ? by Bugdanoff · · Score: 1

    the algorithm scored 91% as well as the doctors, in terms of sensitivity (catching all the cancerous lesions) and sensitivity (not getting false positives).

    Obviously, their natural language processing algorithm still needs some tuning.

  22. Online service to analyse the skin pics we take? by Anonymous Coward · · Score: 0

    How long before there's a website that will allow us to upload the skin pics that we take, and return a cancer analysis? I'm prepared to pay real money for that, I'm sure many others would be too.

  23. Black Box Problem by notil · · Score: 1

    Until we know more about how these algorithms make predictions, it'll be tricky integrating them into medicine: "I think you have a melanocytic lesion because I graduated medical school and have trained in dermatology for six years" still carries more weight than "Our highly accurate algorithm said you scored in a particular way on the 21 dimensions that we can't quite correlate to anything tangible, but it suggests you need this invasive surgery".

    1. Re:Black Box Problem by Anonymous Coward · · Score: 0

      I worked for a company that tried to do exactly that. We tried to help the situation by building our product as a tool for a dermatologist to use, not to compete with them. Diagnosis of skin cancer is almost embarrassingly haphazard, and doctors have no real tools to objectively identify malignant lesions before they are already a problem. Patients seemed to actually appreciate that their doctor had a device that could give them a result based on objective analysis instead of just relying on some med school training and a camera. Sure the lab scientists talked to us about the specific details and the hardcore clinical dermatologists were curious, but a lot of people don't care as long as it works and the FDA lets it happen and most docs seemed like they'd welcome anything with confident results.

      It turns out a bigger problem is "I graduated medical school and have trained in dermatology for six years, I can use this camera" carries a lot more weight than "Using this machine is not covered by insurance".

    2. Re:Black Box Problem by notil · · Score: 1

      This is really interesting, thanks!

  24. Bad news for pimple pinchers by nospam007 · · Score: 1

    They'll become obsolete as diagnosticians.

  25. Oh goody! by Sqreater · · Score: 1

    "... the algorithm scored 91% as well as the doctors, in terms of sensitivity (catching all the cancerous lesions)..."

    Great, it only killed 9% of the people! Let's start using this magnificent technology! Think of the money that will be saved!

    A fundamental problem with computers doing medicine is that it reduces the number of people in medicine and therefore the advance of diagnostics and technique. Computers don't push anything forward. Use too many computers and you will freeze medicine in place at some steady level of knowledge.

    --
    E Proelio Veritas.
    1. Re:Oh goody! by CByrd17 · · Score: 1

      Yeah "as well as" is not the same as "91%" as well as.

      How about, "9% worse than seasoned dermatologists?"

      Don't get me wrong, it's a good step forward; but I want to see 9% BETTER than seasoned dermatologists!