Slashdot Mirror

← Back to Stories (view on slashdot.org)

Google Brain's Co-inventor Tells Why He's Building Chinese Neural Networks

Posted by samzenpus on from the build-a-brain dept.

An anonymous reader writes "Here's an interview with Andrew Ng, former leader of Google Brain, discussing Baidu, Deep Learning, computer neural networks, and AI. An interesting excerpt from the interview on biological vs. computer neural networks: "A single 'neuron' in a neural network is an incredibly simple mathematical function that captures a minuscule fraction of the complexity of a biological neuron. So to say neural networks mimic the brain, that is true at the level of loose inspiration, but really artificial neural networks are nothing like what the biological brain does."

21 of 33 comments (clear)

  1. Re:hu? by RightwingNutjob · · Score: 2

    Having crossed paths with this fella about 10 years ago, you're not too far off. Minus the racism.

  2. On the other hand ... by PPH · · Score: 4, Funny

    artificial neural networks are nothing like what the biological brain does.

    ... there are quite a few people around who tried to overclock their brains during the '60s and '70s.

    --
    Have gnu, will travel.
    1. Re:On the other hand ... by ColdWetDog · · Score: 1

      More like stripped the insulation off the wiring ...

      Which leads me to a question. If the 'neuron' of machine learning is so very different from a biological neuron, why are people insisting on calling it a 'neuron'. Sounds more like a 'synapse' that just takes an input and does some fairly simple manipulation of the signal. Is there some deeper analogy that isn't obvious? A car analogy perhaps?

      --
      Faster! Faster! Faster would be better!
    2. Re:On the other hand ... by ArhcAngel · · Score: 2

      Why were the first cars called horseless carriages? Because the carriage pulled by a horse was something that was familiar. and automobile sounded funny. We ended up dropping the horseless part and shortening carriage to car eventually though.

      --
      "A person is smart. People are dumb, panicky dangerous animals and you know it." - K
    3. Re:On the other hand ... by ShanghaiBill · · Score: 5, Insightful

      If the 'neuron' of machine learning is so very different from a biological neuron, why are people insisting on calling it a 'neuron'.

      You could say the same about the 'wing' of an airplane and a biological wing. The wing of a hummingbird or mosquito is vastly more complicated and capable than the wing of a 747. It provides thrust as well as lift, can do a vertical takeoff without a runway, and can go instantly from forward flight to hovering. On the other hand, a hummingbird can't go from SFO to Narita in 8 hours.

    4. Re:On the other hand ... by Anonymous Coward · · Score: 1

      Technically these are called "perceptrons" in the parlance of machine learning. They're basically a really simple linear discriminant. They have an activation threshold and a bunch of connections to different intermediate layers of perceptrons, and that's where the "neuron" comparison comes from, but it's kind of a false equivalence that gets overplayed for the media.

    5. Re:On the other hand ... by Sudline · · Score: 1

      And cars have still "horsepower".

  3. Bran and brain! What is brain? by neo-mkrey · · Score: 1

    It is Controller, is it not?

  4. Nothing like Biological by NReitzel · · Score: 2

    To say that "artificial neural networks are nothing like what the biological brain does" is no more correct than to say "artificial neural networks are just like the brain."

    Machine learning neural networks do the same flavor of thing that a real organic brain does, but at a complexity that is -many- orders of magnitude smaller. They also tend to be directed at a single skill, and don't have to cohabit the network with, well, everything.

    They're not the same, but they're not totally different, either. Truth is not well served by hyperbole.

    --

    Don't take life too seriously; it isn't permanent.

    1. Re:Nothing like Biological by Anonymous Coward · · Score: 1

      Since not everyone can be a Polymath, hyperbole is a lossy compression algorithm for so-called truth. It summarizes the entirety of an individual's personal experience and amplifies the subtlety in to an signed char.

      It's essentially Fuzzy Logic vs the one dimension analysis of Good/Evil "black and white" Boolean thinking.

      More intelligent people are capable of understanding multidimensional analysis like a radar chart. With enough familiarity the intricacies no longer require amplification to get above the noise threshold and grab the listener by the short hairs.

      It's why a Criminal Justice or Statistics Professor doesn't have to be "racist" to participate in bigotry. The thought-crime taboo of committing the heinous act of mental approximation is only as indecent as the lack of precision used to describe the profile of tendencies along made up fantasies of genetic distinction.

      Although stereotypes are highly efficient predictive algorithms which can usually describe tendencies along any single axis of analysis, they are only as efficient at prediction as the self-fulfilling prophecy has people subscribing to the confirmation bias RSS feed. Racism is effective as a predictor because racism inspires the described outcomes. It's like a reflection of itself which exacerbates in predictive value with every application.

      You can usually shock people out of their confirmation bias pattern by sticking them in a dark alley with a black cop and a bunch of skin head goons. They'll either learn a more "gray" approach to thinking or they'll end up with a knife in the stomach, but don't blame the fish for pattern matching the contours of a shark.

      Now, when I say "highly efficient" I'm actually participating in the hyperbole that you are condemning(maybe even as a demonstration) but the reality is that bigotry and prejudice aren't very efficient at all when compared against a better methodology. Only better than random guesses in many cases. Crucifying bigots for their ignorance of better methodologies for oppressing the poor may do something to mitigate the aggravating impact on injustice of an excessive RSS subscriber count, but it is only whipping the poor/ignorant for their own failure to benefit from the education which would give them superior approximation tools when guesstimating if they're about to get stabbed.

      The mental picture of a self-righteous womyn's studies profession getting mugged and volunteering his SSN as driven by White Guilt(thought crime/cognitive bias hat trick) is humorous(comedians are a reflection of societies pressure points) but in practice my argument is that it's easy to guesstimate who is "low rent" based on their willingness to swallow the pill that crossing the street in a bad neighborhood is an intolerable crime against human dignity. A willingness to play these political games makes a good litmus test for which side of town the candidate grew up on and what type of car they drive(will their poverty be contagious if they park their shit car next to your broken window?).

      Am I more indecent for criticizing the wealthy lizard-people based on my own inability to cohabitate next to the "good schools" or afford the college admissions driven up in cost by their conservative voting tendencies(West Virginia is more "Red" than California)? Is my readily apparent mental illness a form of autism, or an acute stress reaction to living like a popper long enough to be sympathetic to their tar pit plight? Can I haz be one of the "Common People"?

      You might argue this rant is off-topic but we're discussing Deep Learning and ANN's so the product of several decades of survival instinct on the multivariate optimization problem of how not to get beat up/stabbed or fired/starve from salary stagnation for thought crime seems pretty on topic to me. My 1x lifetime human assisted machine learning research project made a pseudo-bigot/crazy person. Neat? Fuck that! We're trying to make co-conspirators, not activists and authors! "Scrub it! Start from scratch with differe

    2. Re:Nothing like Biological by TapeCutter · · Score: 2
      Exactly, an artificial neuron is a mathematical model of a real neuron, it's the "spherical cow" of computer science.

      They also tend to be directed at a single skill

      Yes, The single skill problem is the main reason neural nets were seen as curious toys for 50ys but I think IBM solved that problem with Watson in the mid-noughties. How? - I'm not sure.

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    3. Re:Nothing like Biological by l0n3s0m3phr34k · · Score: 1

      "They also tend to be directed at a single skill"....as in, "kill all the humans". Kill the humans, destroy their culture...kill the humans, kill them all!

  5. The problem with Chinese neural networks by Anonymous Coward · · Score: 1

    An hour after you turn them one they are hungry for singularity.

    1. Re:The problem with Chinese neural networks by Tablizer · · Score: 1

      Cey Lon

      --
      Table-ized A.I.
  6. Biggest difference is timing. by aberglas · · Score: 4, Interesting

    Certainly biological neurons are much more complex than artificial neural net neurons. The simplest "Integrate and Fire" (IF) model of a biological neuron perform a leaky integration over *time*, and if the voltage ever reaches the trigger value the fire. So the timing of stimulations is critical, whereas most Artificial Neural Networks (ANNs) does all its calculations (logically) at the same time. The ANN is both simpler and cleaner to work with. Biological synapses are very complex, but much of that complexity just reflects the wet technology that they are made from.

    If you want to understand how the brain works, study biological neurons. If you want to understand how to build an intelligent machine, engineer ANNs.

  7. No, dude by Threni · · Score: 1

    That is NOT what a Chinese Room is!

  8. Han. by tepples · · Score: 1

    Chinese-as-race is probably referring to the Han Chinese ethnicity.

  9. Re:hu? by Anonymous Coward · · Score: 1

    He's just doing what everyone else who succeeds in this business does. Downloading libraries to recreate popular results and doing better demos. You might as well accuse 90% of PhD students of doing the same thing. In fairness, you probably are.

    Man, we get jaded in this business, don't we?

  10. Oh, the media, lol. by fyngyrz · · Score: 1

    They're basically a really simple linear discriminant.

    Actually, most of them are nonlinear. Sigmoid function is common, and there are much more exotic things going on too, such as fuzzy logic-based discriminants. Bottom line is that any discriminatory function is of interest.

    There's also some fascinating stuff going on with time discriminants where they're having very encouraging results.

    Odds are excellent that both (time and transfer function) are part of a solution that is most human-neuron-like. But it isn't by any means a given that we have to go there to make actual AI work. That's just how we work. Also, I am fairly confident, like this.

    --
    I've fallen off your lawn, and I can't get up.
  11. As to biology by fyngyrz · · Score: 1

    In addition to very high complexity, fixed topology (meaning, using primarily electrical, chemical and timing means as opposed to topological modification to operate), general problem solving networks, I am fairly confident that we develop plenty of what can accurately be described as single-skill networks, topologically tuned to individual problems by continuous cut-and-fit until the errors drop. I lay out why right here.

    --
    I've fallen off your lawn, and I can't get up.
  12. Is this going to be the Chinese Room? by sproketboy · · Score: 1

    https://en.wikipedia.org/wiki/...