Slashdot Mirror

← Back to Stories (view on slashdot.org)

Many Machine Learning Studies Don't Actually Show Anything Meaningful, But They Spread Fear, Uncertainty, and Doubt (theoutline.com)

Posted by msmash on from the reality-check dept.

Michael Byrne, writing for the Outline: Here's what you need to know about every way-cool and-or way-creepy machine learning study that has ever been or will ever be published: Anything that can be represented in some fashion by patterns within data -- any abstract-able thing that exists in the objective world, from online restaurant reviews to geopolitics -- can be "predicted" by machine learning models given sufficient historical data. At the heart of nearly every foaming news article starting with the words "AI knows ..." is some machine learning paper exploiting this basic realization. "AI knows if you have skin cancer." "AI beats doctors at predicting heart attacks." "AI predicts future crime." "AI knows how many calories are in that cookie." There is no real magic behind these findings. The findings themselves are often taken as profound simply for having way-cool concepts like deep learning and artificial intelligence and neural networks attached to them, rather than because they are offering some great insight or utility -- which most of the time, they are not.

5 of 98 comments (clear)

  1. Didn't we all assume that? by nysus · · Score: 3, Insightful

    When AI can teach itself how to use a programming language from documents found on the internet or solve a long unsolved mathematical puzzle, that'll be something to talk about.

    --

    ---Technology will liberate us if it doesn't enslave us first.

    1. Re:Didn't we all assume that? by HiThere · · Score: 4, Insightful

      Please demonstrate that a human can do something that requires actual insight as opposed to statistical calculation. Now prove that this wasn't done via statistical calculation.

      The real problem that most AIs have is lack of grounding and a weak goal structure. But if they have decent grounding and decent ability to manipulate their environment, then you'd better pray that you got the goal structure correct, whether or not they are "strong AI". Cockroaches aren't strong AI, but just try to get rid of them. And the AI will make itself useful to some powerful group of people. (Possibly the group that caused it to be created, but that depends on the goal structure.)

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
  2. It really is like human intelligence. by hey! · · Score: 3, Insightful

    The human brain sees pattern everywhere it looks too.

    I'm retired now but I've been doing a lot of reading and experimentation with decision tree based classification methods. I like these because the produce models you can examine critically, as opposed to so-called "deep learning" algorithms which produce results that you pretty much have to judge by their giving you the result you expect. It's not that that isn't useful in some cases, but I don't find it as interesting.

    --
    Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    1. Re:It really is like human intelligence. by swillden · · Score: 5, Insightful

      The human brain sees pattern everywhere it looks too.

      Yep. Pattern identification system identifies patterns, news at 11.

      OTOH, the ones I find interesting are the cases where ML identifies patterns that humans might not be able to identify. Sometimes this is less interesting for the potential to use a machine to identify these patterns than for the indication that patterns exist where we might think they don't.

      The recent paper on ML "gaydar", where researchers trained a machine to identify sexual orientation from dating site photos is a potentially-fascinating example. In that one I suspect the algorithm was mostly keying on things like hairstyle and other indicators that the people in the photos might be deliberately using to advertise their orientation (this was a dating site), but there seems to be some evidence that facial structure also played a significant role. Of course, given that the most common genders and orientations (cisgendered heterosexuals) are clearly strongly correlated with certain characteristic facial structures, it's certainly reasonable to expect that the same genetic and development processes that determine gender and orientation and clearly affect face structure in the common cases should also affect face structure in the less common cases. But humans can't really see these patterns. Absent behavioral clues, people have very bad "gaydar". But that doesn't mean the patterns aren't there, just that we can't see them. If ML can see strong correlations between facial structure and orientation (and I don't think this one study proves that), that tells us something about the nature of sexual orientation and the degree to which it is expressed in the body.

      --
      Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
    2. Re:It really is like human intelligence. by tomhath · · Score: 3, Insightful

      On one level, this study showed that you can correlate a conclusion (gay/not gay) with some pattern in the data. The problem is that it has only recognized a pattern in that particular data set.

      The article mentions the factors were probably grooming and how the person posed for the picture. Okay, that might be somewhat useful if you are trying to guess sexual preference from a picture on a dating site. But not necessarily, because they threw out sample pictures that didn't provide the clues they were looking for, and (I suspect) selected test pictures that did have the clues.