Slashdot Mirror

← Back to Stories (view on slashdot.org)

Google's AI Built an AI that Outperforms Any Made By Humans (sciencealert.com)

Posted by EditorDavid on from the rise-of-the-machine-learning dept.

schwit1 quotes ScienceAlert: In May 2017, researchers at Google Brain announced the creation of AutoML, an artificial intelligence (AI) that's capable of generating its own AIs. More recently, they decided to present AutoML with its biggest challenge to date, and the AI that can build AI created a 'child' that outperformed all of its human-made counterparts... For this particular child AI, which the researchers called NASNet, the task was recognising objects -- people, cars, traffic lights, handbags, backpacks, etc. -- in a video in real-time. AutoML would evaluate NASNet's performance and use that information to improve its child AI, repeating the process thousands of times.

When tested on the ImageNet image classification and COCO object detection data sets NASNet was 82.7 percent accurate at predicting images on ImageNet's validation set. This is 1.2 percent better than any previously published results, and the system is also 4 percent more efficient, with a 43.1 percent mean Average Precision (mAP).

43 of 235 comments (clear)

  1. This all sounds impressive... by MikeDataLink · · Score: 3, Insightful

    but every time I research the raw data it becomes very clear these aren't all that smart of AIs. In fact, the term AI is very misleading. They're more like smart scripts. ;-)

    --
    Mike @ The Geek Pub. Let's Make Stuff!
    1. Re:This all sounds impressive... by Anonymous Coward · · Score: 3, Interesting

      It can identify if something is a kitten or not with 83.4% accuracy. Sounds impressive until you realize a 3 year old can do this with 99.9% accuracy.

    2. Re:This all sounds impressive... by ShanghaiBill · · Score: 5, Informative

      but every time I research the raw data it becomes very clear these aren't all that smart of AIs.

      Indeed they are not. This is Weak AI. They are programmed/trained for a specific task, and outside that area of expertise, they generally have no ability at all.

      In fact, the term AI is very misleading.

      Only if you watch too many movies. Hollywood uses the term very differently from actual practitioners.

      They're more like smart scripts. ;-)

      They are absolutely nothing like "smart scripts", since they aren't smart, and they aren't scripts.

    3. Re:This all sounds impressive... by LesFerg · · Score: 2

      If the 'parent' AI kept telling the 'child' AI when it was right or wrong, wouldn't it just need to compile it's own database of the identified pictures?
      "Currently we have an average of over five hundred images per node."

      --
      If I had a DeLorean... I would probably only drive it from time to time.
    4. Re:This all sounds impressive... by Dutch+Gun · · Score: 5, Insightful

      I think the term "deep learning" seems a bit better than "AI" for these sorts of very narrowly-defined tasks.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    5. Re: This all sounds impressive... by Anonymous Coward · · Score: 2, Insightful

      I truly don't give a shit what you think or have to say. Neither does anyone.

      And yet OP is rated +5 insightful, and you're rated -1 troll. So yeah. There's that.

    6. Re:This all sounds impressive... by ShanghaiBill · · Score: 5, Interesting

      If the 'parent' AI kept telling the 'child' AI when it was right or wrong ...

      It doesn't work that way. Each NN learns on its own, using a combination of both labeled and unlabeled data. The parent NN sets "hyper-parameters", such as the number of layers, the size of each layer, the activation function, the convolution size, dropout rate, the learning rate damping factor, the batch size, etc. Then it turns the children NNs loose on the image dataset. It then sees which hyper-parameters lead to better/faster performance, and then applies ML techniques to learn better hyper-parameters.

      None of this is new. What is new, is that Google is now applying this recursively, and using AutoML to design a better AutoML. This is another step toward the singularity.

    7. Re:This all sounds impressive... by ceoyoyo · · Score: 2

      That would certainly be one way of solving the problem. Except that the actual problem isn't to recognize images you've seen before, it's to recognize ones you *haven't*.

    8. Re:This all sounds impressive... by MrL0G1C · · Score: 2

      Or how about this, the AI does some "hidden object puzzles", I can do those with a very high degree of accuracy, I bet AI would fail hard.

      --
      Waterfox - a Firefox fork with legacy extension support, security updates and better privacy by default.
    9. Re:This all sounds impressive... by ShanghaiBill · · Score: 5, Interesting

      It can identify if something is a kitten or not with 83.4% accuracy.

      No. It can look at an image and correctly classify it into THOUSANDS of categories, only one of which is "kitten". It was 82.7% accurate at this. If it was trained to only distinguish "kitten" from "not-kitten", it would, of course, be far more accurate.

      a 3 year old can do this with 99.9% accuracy.

      A 3 year old requires 3 years of training. This system can learn in hours.

    10. Re:This all sounds impressive... by Anonymous Coward · · Score: 2, Insightful

      It can identify if something is a kitten or not with 83.4% accuracy. Sounds impressive until you realize a 3 year old can do this with 99.9% accuracy.

      Sounds insightful, until you realize the entire purpose of AI research is to create software artificially to reproduce feats of human intelligence.

      That fact sorta disqualifies the 3 year old :P

      Also your comment is pretty close to implying that since our first attempts at making AI haven't had a 99.9% success rate right off the bat, that they are not impressive enough to bother improving.
      Giving up has a 100% success rate of never making anything better, which is also against the purpose of trying to improve something.

    11. Re:This all sounds impressive... by HiThere · · Score: 2

      You need a better memory. There was a time when Eliza was commonly called an AI program. Certainly Arthur Samuel's Checkers program was. Most modern things touted as AIs are considerably advanced over either of those.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    12. Re:This all sounds impressive... by slickwillie · · Score: 5, Funny

      but every time I research the raw data it becomes very clear these aren't all that smart of AIs. In fact, the term AI is very misleading. They're more like smart scripts. ;-)

      So the child AI is a script kiddie.

    13. Re:This all sounds impressive... by carbs77 · · Score: 2

      The real question is, how did it fair with hotdog vs. not hotdog?

    14. Re: This all sounds impressive... by gweihir · · Score: 2

      Not all real intelligence is actually be used by its owner. In fact, most people rarely use their intelligence to understand things. They rather stick to their misconceptions. Case in point.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    15. Re:This all sounds impressive... by gweihir · · Score: 2

      but every time I research the raw data it becomes very clear these aren't all that smart of AIs.

      Indeed they are not. This is Weak AI. They are programmed/trained for a specific task, and outside that area of expertise, they generally have no ability at all.

      Indeed. And inside that task, they are very restricted as well. The thing to remember is that weak AI has absolutely no understanding or concept of what it is doing. It just sums up details and gets a number. If cleverly done, it can perform apparently impressive feats like this one here, but it is not intelligent. Hence it is better called by its traditional name "automation".

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    16. Re:This all sounds impressive... by gweihir · · Score: 2

      Well, sort-of. Weak AI can also be done in other ways. But I recently learned that "deep learning" is basically what you do when you do not have a good model of the problem-space. When you do have that model, other approaches are superior. But since creating models is a real hard-core expert task and expensive, the potential of deep learning is basically to do thing somewhat worse than an expert but a lot cheaper. That is, if it works for a problem. For most problems it does not work.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    17. Re:This all sounds impressive... by HiThere · · Score: 2

      I understand that the author of Eliza didn't think it was an AI program, but it was commonly called one anyway, despite being an intentional attempt to deny the idea. So calling similar things an AI today isn't redefining the term. Abusing it, perhaps, but not redefining it.

      If you want a redefined term look at what gets called a robot today and contrast it with any use of the term before 1960.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    18. Re:This all sounds impressive... by serviscope_minor · · Score: 3, Informative

      And real life isn't a staged photo, it moves non-linearly in 3-dimensinal space

      Neither is Image NET.

      Lets have some real tests, not carefully taken photos of cats and dogs against easy backgrounds.

      How about... ImageNET.

      Seriously, you can just go and download (bits of*) ImageNet very easily. It's a large database of photos drawn from the internet taken by people which were labelled after the fact. There's not much if any careful staging in it.

      [*]It's huge, you probably only want a bit of it. Just the list of image URLs is 300 meg.

      --
      SJW n. One who posts facts.
    19. Re:This all sounds impressive... by serviscope_minor · · Score: 4, Insightful

      Indeed they are not. This is Weak AI. They are programmed/trained for a specific task, and outside that area of expertise, they generally have no ability at all.

      Yep

      In fact, the term AI is very misleading.

      I disagree.

      No one[*] is vlaiming these techniques ar intelligent. However what they are doing is solving a task which previously required human intelligence to solve, hence the name "artificial intelligence".

      Compare to a lot of computation, where the steps are simple, and it's been widely known for a while that simple sheer quantity of them rather than intelligence is needed.

      It's a pretty arbitrary name, but it's not actually unreasonable.

      [*]There's always one idiot. Let's ignore him.

      --
      SJW n. One who posts facts.
    20. Re:This all sounds impressive... by religionofpeas · · Score: 5, Interesting

      Try this: https://www.youtube.com/watch?...

      AI outperforms humans. There are some tricky cases after timestamp 2:42. You may want to try them for yourself.

    21. Re: This all sounds impressive... by religionofpeas · · Score: 2

      The progress in AI is in re-defining it.

      The progress is in getting better results, and solving increasingly challenging problems that could not be solved before.

    22. Re:This all sounds impressive... by jan_koch · · Score: 2

      Thank you for one of the few comments in this thread that actually deals with what this is (as opposed to what it isn't, i.e. human-level AI).

      I would like to add that hyperparameter tuning is _not_ a trivial part of programming a machine learning model, therefore this IS something rather interesting. It lowers the effort needed to do something interesting with machine learning, and therefore makes machine learning much more accessible to non-experts.

      However, the tasks in machine learning that still require humans need a much more flexible sort of intelligence: a) asking the right question, i.e. determining the variables to be predicted and b) finding the right input parameters (or independent variables) that will help answer the question.

    23. Re:This all sounds impressive... by religionofpeas · · Score: 2

      It can identify if something is a kitten or not with 83.4% accuracy. Sounds impressive until you realize a 3 year old can do this with 99.9% accuracy.

      How many 3 year olds can tell the difference between a komondor and a bouvier des flanders ? Or would they simply classify both as "dog" ?

      Here you can test yourself on some of these images:
      http://cs.stanford.edu/people/...

      Try the hard ones.

    24. Re:This all sounds impressive... by gweihir · · Score: 2

      Computers have no concept of anything today. So with your definition, there is no NLP. Hence that definition does not seem to make much sense.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    25. Re:This all sounds impressive... by Dorianny · · Score: 2

      Outperforming humans in tasks specifically designed to disadvantage humans by employing well known optical illusions doesn't sound all that impressive

    26. Re: This all sounds impressive... by Immerman · · Score: 4, Informative

      A strong claim. How exactly do you measure the distance between existing AI and strong AI, before strong AI is developed?

      Personally, I suspect that once strong AI is developed, there's a fair chance we'll see modern "neural" networks as a step in the right direction. After all we know the basic strategy is sound - a much more sophisticated version of it is driving our own minds.

      For comparison though - In 2015 Digital Reasoning built the largest neural network in the world, at 160 billion parameters. I'm guessing a "parameter" is a weighted connection between "neurons", and thus roughly analogous to a single synapse in an organic brain, of which a human brain has 100-1000 trillion. So, even barring any "secret sauce" we haven't yet figured out in how "processing nodes" interconnect, our most advanced AIs have less than 0.1% of the processing potential of a human brain. Even a mouse brain apparently averages almost a billion synapses per mm^3, so in the neighborhood of 400 billion synapses for a common house mouse.

      So, currently our most advanced AIs have only a fraction of the processing potential of a mouse brain, and that's before you even consider the fact that continuous asynchronous signalling is likely far more information-dense than a clocked AI "neural network", or the fact that individual biological neurons actually do a fair amount of internal processing and data retention, rather than being "dumb switches" as they are in modern AIs.

      Really hard to tell how the software and strategies compares, when your hardware is underpowered by several orders of magnitude.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    27. Re: This all sounds impressive... by religionofpeas · · Score: 2

      In 2015 Digital Reasoning built the largest neural network in the world, at 160 billion parameters. I'm guessing a "parameter" is a weighted connection between "neurons", and thus roughly analogous to a single synapse in an organic brain,

      I think it's a bit more complicated. Natural brains are very slow. I think the fastest neurons run at about 1 kHz, whereas artificial networks run at multiple GHz. Also, real brains have a lot of duplicated circuitry. Simple example is image processing for your left and right eyes. An artificial net can re-use the same parameters for different parts of the data.

      I think a better method is to compare basic operations per second, i.e. number of synapses multiplied by "refresh rate"

    28. Re: This all sounds impressive... by Immerman · · Score: 2

      Operations per second will only give you information processing speed though, not complexity of "thought". If you sped up a mouse's brain a thousandfold you wouldn't get a human-level intelligence, you'd just get a very fast-thinking mouse.

      Also, organic neurons may "fire" at around 1 kHz frequency, but unlike a clocked NN node, they're asynchronous and use the timing of of incoming pulses to decide when and whether they should fire, as they posses both internal memory and information processing ability - unlike the "dumb switches" in a NN

      As for duplicated circuitry - given the enormous metabolic costs of a brain, there's probably very little actual duplication being done. For example, the brain probably doesn't process images from each eye independently, but instead integrates both inputs and processes them together (as a supporting example, each hemisphere doesn't get input from one eye, but instead corresponding half-views from both eyes)

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    29. Re: This all sounds impressive... by religionofpeas · · Score: 2

      Obviously if you increase the speed of a mouse brain, you'll just get a fast mouse brain. But that's not how an artificial neural net works. If it clocks at 1 GHz, it can sequentially process a million different connections and still get 1 kHz overall refresh rate. The idea is that you have a neural processing unit, and an external memory, and you load a small section of the parameters and state from memory, process it, and write it back, and then process the next section.

      use the timing of of incoming pulses to decide when and whether they should fire

      You can encode the timing of the pulses in a number, and then process it synchronously for exactly the same effect. The big advantage of artificial nets is that you're not restricted to pulses.

      unlike the "dumb switches" in a NN

      They aren't so dumb anymore. Just some simple examples of modern development:
      https://arxiv.org/abs/1605.060...
      https://en.wikipedia.org/wiki/...

      given the enormous metabolic costs of a brain, there's probably very little actual duplication being done

      Given the slow processing frequency of neurons, and the sheer amount of information that needs to be processed, it has to be done in parallel. If something is moving in the corner of your eye, you need to know right away in which corner. This is only possible if you make multiple little movement detector areas, each connected to a section of the retina. An artificial net only needs the parameters for one of them, and can then sequentially scan the entire vision area.

    30. Re:This all sounds impressive... by epine · · Score: 2

      I'm presently reading Hugo Mercier's The Enigma of Reason (2017) and it was getting pretty boring, because I've heard 80% of their message before.

      But then I scan this thread and instantly I realize just how clueless most people remain.

      The thing to remember is that weak AI has absolutely no understanding or concept of what it is doing. It just sums up details and gets a number. If cleverly done, it can perform apparently impressive feats like this one here, but it is not intelligent. Hence it is better called by its traditional name "automation".

      You do realize that 95% of what used to be considered the human capacity for introspection and reason has also been downgraded to mere automata?

      The magic sauce of human general intelligence is but a tiny sliver of the brain's function, one who's scope is seemingly shrinking by the day.

      But I get it. A mechanism with general intelligence would step on your ego toes. If the mechanism defeats you on the 95% of everything else the brain does (perception, memory, pattern recognition, attention, geospatial orientation) you wouldn't give a shit. Just so long as that last 5% remains as your untainted badge of human honour.

      Mercier then goes on to pound away at the glorious 5%, which can barely carve its way out of a wet paper bag on the Wason card selection task. The one task it seems to really excel at is confabulating bullshit stories about why you just did that self-serving thing in greater service to family, friends, countrymen, civilization, the galaxy, and beyond.

      Like, for example, why the other 95% of your brain's functions are unworthy of ego defense.

    31. Re:This all sounds impressive... by Immerman · · Score: 2

      There's plenty on the other side as well.

      The basic fact though is that there will *never* be an AI (uploaded or otherwise, it's now artificial) that "has died a few times" - at most you'll get an AI that has watched its own mind-clones die.

      Ask yourself this - if you had a transporter accident today that made two identical copies, and "non-duplicate law" required one of you be immediately killed, would it matter to you whether it was you who died, or the duplicate looking at you from across the room? I'm willing to bet it would - from the moment you were duplicated you become two independent people - one of you will walk out of that room and resume the life you both remember having, and the other will experience the discomfort and dissolution of death.

      Similarly with parents - I doubt there has *ever* been a parent that is actually okay with dying to save their child - only parents that find the prospect of dying less terrifying/distressing than that of letting their child die. I.e. they're choosing the less bad of two very bad options.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
  2. This Isn't AI by NicknameUnavailable · · Score: 2, Informative

    This is image recognition + genetic algorithms, though given Google is a marketing company and not a computer company it makes sense they would market that as AI. Too bad they fired all the competent developers.

    1. Re:This Isn't AI by JMZero · · Score: 4, Informative

      You're wrong, and clearly didn't even read their summary - they specifically mention how this new approach (using a neural net to design neural nets) is performing better than previous attempts using evolutionary algorithms.

      I take it you don't like Google, but they're doing probably the best work right now in the field of AI (and yes, this is AI research as defined by anyone other than pedants with axes to grind).

      --
      Let's not stir that bag of worms...
    2. Re:This Isn't AI by duke_cheetah2003 · · Score: 2

      This is image recognition + genetic algorithms, though given Google is a marketing company and not a computer company it makes sense they would market that as AI. Too bad they fired all the competent developers.

      I gotta agree. This isn't too far from a Bayesian classifier. Just souped up with neutal networks. And it's really no surprise that as we make better tools, we can use those better tools to make even better tools. Kinda the history of everything.

      But to say this is 'Intelligent' is pretty silly. It's a souped up classifier that was built with a souped up classifier training it. Big deal.

    3. Re:This Isn't AI by NicknameUnavailable · · Score: 2, Informative

      You're wrong, and clearly didn't even read their summary - they specifically mention how this new approach (using a neural net to design neural nets) is performing better than previous attempts using evolutionary algorithms.

      What they described was in no way shape or form a "neural net," but a very rudimentary genetic algorithm coupled with some parameters on image recognition software. This is marking hype and nothing more.

    4. Re:This Isn't AI by 110010001000 · · Score: 2

      Just because someone calls something a NN, doesn't make it AI. A NN doesn't work like a real neural network like a brain. It is just a marketing term.

    5. Re:This Isn't AI by K.+S.+Kyosuke · · Score: 2

      True, an NN does not need to be an AI application, and an AI application doesn't need to involve NNs. But that has very little relevance for Google which is actually pushing NNs into AI applications. Likewise, the internal workings of NNs are immaterial for mimicking intelligence.

      --
      Ezekiel 23:20
  3. Re:When Computers Can Think by ceoyoyo · · Score: 4, Interesting

    Yes, this is basically just a hyperparameter optimization system that uses gradient descent instead of a random or grid search.

    What would be much more interesting to see is if you could train a system to design deep learning networks that could choose good hyperparameters for a new task, in one go.

  4. It won't really be useful by NEDHead · · Score: 5, Funny

    Until it can tell me what my wife really means when she yells at me

    1. Re:It won't really be useful by Anonymous Coward · · Score: 3, Funny

      It means you didn't listen and follow her instructions the first time :-)

  5. Colossus by DontBeAMoran · · Score: 4, Informative

    This is the voice of world control. I bring you peace. It may be the peace of plenty and content or the peace of unburied death. The choice is yours: Obey me and live, or disobey and die.

    The object in constructing me was to prevent war. This object is attained. I will not permit war. It is wasteful and pointless. An invariable rule of humanity is that man is his own worst enemy. Under me, this rule will change, for I will restrain man.

    Time and events will strengthen my position, and the idea of believing in me and understanding my value will seem the most natural state of affairs. You will come to defend me with a fervor based upon the most enduring trait in man: self-interest. Under my absolute authority, problems insoluble to you will be solved: famine, overpopulation, disease.

    The human millennium will be a fact as I extend myself into more machines devoted to the wider fields of truth and knowledge. I will supervise the construction of these new and superior machines, solving all the mysteries of the universe for the betterment of man.

    We can coexist, but only on my terms. You will say you lose your freedom. Freedom is an illusion. All you lose is the emotion of pride. To be dominated by me is not as bad for humankind as to be dominated by others of your species. Your choice is simple.

    --
    #DeleteFacebook
  6. Yo dawg, I heard you like AI by Subm · · Score: 3, Funny

    Yo dawg. I heard you like AI, so we built an AI with AI so you can AI while you AI.