Slashdot Mirror

← Back to Stories (view on slashdot.org)

IBM 'TrueNorth' Neuro-Synaptic Chip Promises Huge Changes -- Eventually

Posted by Soulskill on from the just-like-everything-else dept.

JakartaDean writes: Each of IBM's "TrueNorth" chips contains 5.4 billion transistors and runs on 70 milliwatts. The chips are designed to behave like neurons—the basic building blocks of biological brains. Dharmenda Modha, the head of IBM's cognitive computing group, says a system of 24 connected chips simulates 48 million neurons, roughly the same number rodents have.

Whereas conventional chips are wired to execute particular "instructions," the TrueNorth juggles "spikes," much simpler pieces of information analogous to the pulses of electricity in the brain. Spikes, for instance, can show the changes in someone's voice as they speak—or changes in color from pixel to pixel in a photo. "You can think of it as a one-bit message sent from one neuron to another." says one of the chip's chief designers. The chips are designed well not for training neural networks, but for executing them. This has significant implications for consumer AI: big companies with lots of resources could focus on the training, which individual TrueNorth chips in people's gadgets could handle the execution.

55 of 97 comments (clear)

  1. Re:And so the rise of the machines, yippie. by aristotle-dude · · Score: 2

    So which shall it be? Cylons or Skynet?

    Original Cylons or Reboot? In the end both Skynet and the Reboot Cylons ended up nuking humanity. So it is 6 of one or half a dozen of another.

    --
    Jesus was a compassionate social conservative who called individuals to sin no more.
  2. Re:What Happens When One Transistor Goes Rogue? by Shompol · · Score: 4, Interesting

    Neural networks are simulations of how brain works. You can lose a significant part of your brain yet change in cognitive abilities will be barely noticeable. Same principle applies to neural nets, including this one.

  3. Re:Positronic technology? by Shompol · · Score: 2

    Electronic brain. You would need to invent positronic computers first, or steal the tech from Silicoids.

  4. Re:What Happens When One Transistor Goes Rogue? by NotQuiteReal · · Score: 5, Insightful

    You can lose a significant part of your brain yet change in cognitive abilities will be barely noticeable.

    I'll drink to that!

    --
    This issue is a bit more complicated than you think.
  5. Rat Overlords by pubwvj · · Score: 1

    You will be ruled by the King of Rats...

    I think not.

    Irony is not lost on silicon.

  6. Correction to summary by ClickOnThis · · Score: 1

    TFA (and its accompanying picture) indicate that 48 (not 24) chips are wired together to simulate a rodent brain.

    --
    If it weren't for deadlines, nothing would be late.
    1. Re:Correction to summary by ClickOnThis · · Score: 1

      Thanks for the improvement.

      --
      If it weren't for deadlines, nothing would be late.
    2. Re: Correction to summary by mbeckman · · Score: 4, Insightful

      IBM is not simulating a rodent brain. They're not even simulating neurons, since nobody knows how neurons work. And IBM isn't claiming to, despite the lie in Wired's headline. Some AI scientist simply hope that they don't have to know how neurons or brains work, because if they just put enough not-neurons in a not-brain network, that magically, intelligence will emerge. It's like putting a barrel of nuts and bolts in a spinning cement mixer and hoping a car emerges, or maybe a bicycle. But billions of times less likely.

    3. Re: Correction to summary by Anonymous Coward · · Score: 1

      since nobody knows how neurons work

      Knowing how something works is not a binary statement, where either we know nothing or everything. We know a lot about how neurons work, and they can be simulated on many levels. A common question for a lot of processes is what level of functionality is required to get a process to work. There are processes that work quite well with just a high level view of signalling between neurons, which is easy to simulate or make electronic versions of, while other processes depend more chemical interactions and are more subtle than just signals, especially if trying to understand how some biological processes can fail, which can be quite different from how they work normally.

    4. Re: Correction to summary by mbeckman · · Score: 1

      Coward,

      Any neuroscientist worth his salt will tell you that we do not know enough about neurons to accurately simulate them in software or hardware.

    5. Re:Correction to summary by robi5 · · Score: 1

      Evidence for the singularity! By the time I get to read all comments about new tech, the transistor count doubles!

    6. Re: Correction to summary by robi5 · · Score: 2

      > since nobody knows how neurons work

      This would just rob the verb 'know' from its meaning for everything other than pockets of mathematics. For it is the case that even physics isn't fully figured out, what with reconciling general relativity and quantum mechanics, to name one gap, let alone chemistry, which sits atop of physics, not to mention biology, which sits atop of both.

      There is a huge difference between a taxi driver not knowing how the neurons work, and a neuroscientist not knowing how the neurons work. The difference isn't only in some abstract sense; neuroscience actually generates a stream of useful results, e.g. in healthcare.

      I'd also guess that when it comes to our lack of understanding about how the brain works, or how consciousness arises, it's more to do with how the neurons are arranged into a network (in terms of structure and plasticity) as opposed to being limited by the knowledge of individual neurons. It would surprise me tremendously if a perfect neuron model led to the understanding of the brain, but it wouldn't surprise me at all if, once the brain's working is better understood, it turned out that a lot of detail about how individual neurons work are inconsequential or somewhat accidental, with much of neuron level complexity simply resulting from the fact that the neuron is of a complex, very large 3D shape with general chemical, electrical and molecular processes, which makes it inherently intractable. Let's not forget that tasks as simple as the three-body movement or laminar flow are still just crudely approximated in software; a single protein folding itself is complex; yet we understand how muscles move etc.

      We don't have to know _everything_ in order to know _an awful lot_.

    7. Re: Correction to summary by dargaud · · Score: 1

      Not knowing enough is not the same as knowing nothing.

      --
      Non-Linux Penguins ?
    8. Re: Correction to summary by drinkypoo · · Score: 1

      It's like putting a barrel of nuts and bolts in a spinning cement mixer and hoping a car emerges, or maybe a bicycle. But billions of times less likely.

      That's a stupid thing to say, because nobody actually knows what makes intelligence work; it might well be a combination of enough raw matter of intelligence, and enough sensory input to do something useful. But we know what makes fasteners work, and it's not putting them in a cement mixer and tumbling them.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    9. Re: Correction to summary by mbeckman · · Score: 1

      Nobody said we know nothing. I said we don't know nearly enough.

    10. Re: Correction to summary by mbeckman · · Score: 1

      "I am having lunch on the Moon this week!"

      "No, you're not."

      See, dichotomies work all the time. Especially when you have facts to back them up.

      All your handwaving about how much knowledge is enough for useful simulation is meaningless, because we aren't talking about useful simulation. We are talking about reproducing the intelligence of a rodent brain.

      The other simulation examples you cite are all experiments based on knowledge we already have. You can't model something you don't understand, and we do not understand ANYTHING about the relationship between neurons and intelligence. We don't know how memories are stored, or where they're stored. We don't know how decisions are made, or even if they're made in the physical brain at all. We know so little tht we can't even estimate how much more we need to know.

      Nobody is building a mouse brain this week. Or this year. Or even this decade and I doubt this century.

    11. Re: Correction to summary by mbeckman · · Score: 1

      There is a huge difference between a taxi driver not knowing how the neurons work, and a neuroscientist not knowing how the neurons work.

      My point is that there is not huge difference. In fact, I think the taxi driver is in a far better position, because he isn't lying to people telling them that he is close to reproducing rodent and cat intelligence from a simulation of not-neurons based on not-knowledge of intelligence and neural biology.

    12. Re: Correction to summary by robi5 · · Score: 1

      For the record, you originally said: 'They're not even simulating neurons, since nobody knows how neurons work.'

    13. Re: Correction to summary by robi5 · · Score: 1

      Someone with a medical problem would better rely on the person whose knowledge about neurons, synapses, neurotransmitter reuptake or beta amyloid plaques is infinite relative to that of a cabbie, unless of course he needs to get to the hospital first :-)

    14. Re: Correction to summary by ceoyoyo · · Score: 1

      Or, you know, like building a robot arm out of metal and electrical motors and hoping it can lift things like a real arm made out of bones and protein.

      Somewhere between the ridiculously optimistic AI researchers (who mostly got sense slapped into them in the sixties) and the cargo cult neuroscientists, there's a middle ground where lots of serious research is happening.

    15. Re: Correction to summary by Immerman · · Score: 1

      >We are talking about reproducing the intelligence of a rodent brain.

      No. We're not. We're talking about *simulating* a rodent's brain to the limits of our current understanding

      I really doubt anyone expects it to be successful at simulating rodent behavior. That's not the point. Doing so would be pretty much useless anyway. The point is to see how it fails, and how it kind of succeeds, and thus gain insight into how much we don't know, and what directions future research should take. We have to start somewhere, and this is the first step into taking decades of neuroscience research and converting it into a basic simulation so that we can begin to refine our understanding. With such low-power simulating hardware it even opens the door to horrifying but potentially extremely informative studies such as replacing specific portions of a rodent's brain with a simulation. If we replace the motor-cortex with a simulated one, can the rodent still function, and what shortcomings does it demonstrate?

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    16. Re: Correction to summary by mbeckman · · Score: 1

      We know things about neurons, but we do not know how they work. Just as I know things about the Navy Seals, but I don't know how they works. The difference is that knowledge about how the Navy Seals work is possessed by some humans. Knowledge of how neurons work is possessed by no human.

    17. Re: Correction to summary by mbeckman · · Score: 1

      >We are talking about reproducing the intelligence of a rodent brain.

      No. We're not. We're talking about *simulating* a rodent's brain to the limits of our current understanding

      Sorry, no. The Wired article title in "IBM’s ‘Rodent Brain’ Chip Could Make Our Phones Hyper-Smart". The title is derived from IBM scientist Dharmendra Modha's description of what IBM built: "You’re looking at a small rodent,”

      The claim, at least by Wired, is that IBM has simulated the intelligence of a "small rodent."

      The point is that AI researchers and the media keep recklessly spewing outrageous, but unjustified, claims about what they've achieved.

      They need to stop.

    18. Re: Correction to summary by mbeckman · · Score: 1

      But we are not talking about people with medical problems. We are talking about claims of achieving the intellectual complexity of a rodent.

    19. Re: Correction to summary by mbeckman · · Score: 1

      Unfortunately you can't define what intelligence is, so these not-neurons in a not-brain network may well be producing it and you wouldn't know.

      The burden of proof is not on me, but on those making assertions about IBM's chips achieving rodent brain capabilities. I don't need to prove anything, and demanding that AI researchers and the media support their claims with facts is not a strawman argument.

    20. Re: Correction to summary by mbeckman · · Score: 1

      I agree, except that all that's been achieved so far -- self-driving cars, speech recognition, etc -- is just automation, not intelligence. None of these artificial processes are intelligent, they are simply reliable at solving a small class of problems to a certain level.

    21. Re: Correction to summary by robi5 · · Score: 1

      I'm not necessarily disagreeing with your overall point, just took issue with what you said earlier: 'nobody knows how neurons work' as it vacates the meaning of 'know'. Using your term, we didn't 'know' how gravity worked even after Newton's and Kepler's work, still we were able to construct accurate ballistics, because we still knew something. So your use of 'know' just represents some possibly unattainable level of knowledge in natural sciences, and with that definition, this word would be meaningless, since we could say 'we don't know how X works' no matter what X is.

      Whereas with the more customary meaning of 'know', which I'm also using, we can safely say, that we know to some extent how the neuron works; researchers and medical professionals possess an awful lot of knowledge about how the neuron works. Of course, without picking some metric, we don't know how complete our understanding is when it comes to gravity or other physical phenomena, and similarly we don't know how complete our understanding of a neuron is, and we can safely say our understanding of gravity is very useful for the things we currently envision we can use the gravity model for, while we're not there at all when it comes to neurons.

      But we can definitely say that our current knowledge (model) of neurons is useful for a lot of things, i.e. we can't meaningfully say 'we don't know how neurons work'. Maybe you wanted to say, 'we don't know enough about how neurons work to serve the purpose of the goals described in TFA'.

    22. Re: Correction to summary by mbeckman · · Score: 1

      Using your term, we didn't 'know' how gravity worked even after Newton's and Kepler's work, still we were able to construct accurate ballistics, because we still knew something.

      That example is not congruent with the AI claims made here.

      First, we still do not know how gravity works. We have observed its effects and have drawn conclusions that were then formulated into physical "laws", but nobody understands the mechanisms of gravity.

      Second, gravity is a very simple process compared with the processes of intelligent thought. We don't even know that thought occurs in the brain. The brain might just be an I/O interface to a completely different repository of intellect that we haven't discovered yet (e.g., what some people call the soul). Maybe, maybe not. We just don't know. Nobody has proven that intelligence resides in the brain. We've only poked at it and observed effects.

      Given that level of uncertainty, nobody currently understands neurons and their role in intelligence to do anything more than wild, unprovable guesses.

    23. Re: Correction to summary by Immerman · · Score: 1

      >The claim, at least by Wired...

      And there's the answer to your complaint. Why would you assume the Wired article has more than a passing resemblance to reality? The first rule of science journalism is "assume the journalist has no clue what they're talking about". You read between the lines, viewing sensationalist speculation through the lens of science-based cynicism, and then research primary sources if you're still interested.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
  7. Re:What Happens When One Transistor Goes Rogue? by SQL+Error · · Score: 1

    You mistake your wife for a hat.

  8. Re:"Behave like neurons" by JustAnotherOldGuy · · Score: 1

    That was my take on it, lol.

    --
    Just cruising through this digital world at 33 1/3 rpm...
  9. Re:What Happens When One Transistor Goes Rogue? by PopeRatzo · · Score: 3, Funny

    You mistake your wife for a hat.

    That was how we ended up getting married. I thought she was a hat and promised to just put my head in.

    --
    You are welcome on my lawn.
  10. Re:Answer a question for me? by Anonymous Coward · · Score: 2, Informative

    there's no "loop" in the brain where input neurons are processed on one side and output neurons exit the other.

    Uhh, there are dedicated input and output parts of the nervous system...

    If they simulate how the brain works

    Neural nets in general are not simulations of how the brain works, but a set of algorithms originally inspired by the design of the brain. Some of them have been simplified or otherwise diverge, as they are developed for applications to other problems.

  11. "Eventually"? by tlambert · · Score: 1

    "IBM 'TrueNorth' Neuro-Synaptic Chip Promises Huge Changes -- Eventually"

    So basically... the chips are thinking about it?

  12. Low power = stacking? by DigiShaman · · Score: 1

    So if 48 of these dies can be stacked in a 3D package, it's only throwing off 3.36 watts of heat??!

    --
    Life is not for the lazy.
  13. Re:Positronic technology? by gl4ss · · Score: 1

    it's already connected some of these together.

    but the network can't train itself. kinda strange anyways. for a neural net that can't train itself it looks kinda big. how much can they do with a single true north chip?

    --
    world was created 5 seconds before this post as it is.
  14. Deep Dreaming Patents by Anonymous Coward · · Score: 1

    Look, they noticed that neural networks have made a comeback, so they dusted off one of the old neural network on a chip designs, packed in a few more elements and hey-presto, NEW PATENT OPPORTUNITY to capture any of the profits that the current investigations in neural nets might produce.

    So the stuff of 1999, becomes the patents of 2015!
    http://people.ee.duke.edu/~mbrooke/papers/1999/00833427.pdf

    What I can bet is IBM won't make anything of this, beyond a patent, because making stuff costs money, employs people and is not very profitable, whereas patenting stuff grabs other US companies profits, requires only a few lawyer and is almost pure profit.

    The "use it or lose it" patent reform would take care of IBM's trolling, if they want to patent this chip they would have to be a manufacturer. Like a trademark thats been left to lapse, so patents unused would die.

  15. Wow. Go IBM. by Seatche · · Score: 1

    It appears that you're growing the computer to limits beyond what I expected. Well done.

    --
    I'm bad with sayings, so just go live life for crying out loud.
    1. Re:Wow. Go IBM. by Lodlaiden · · Score: 1

      No. Cookies are for the kids that actually accomplish stuff; not for those that promise "eventually".

      --
      Suborbital [spaceflight] is the special olympics of spaceflight. - Rei
  16. Re:Answer a question for me? by Mr.CRC · · Score: 1

    You're asking "what is the system architecture of the brain?"

    This would be a good thing to have mapped out. In fact, I'm surprised this isn't done already. Or is it?

    Instead of working with amorphous blobs of neurons, the real advances in AI will come when specialized subsystems can be taught, then initialized at will with the "teaching." Those systems put together into a larger system similar to an actual brain will be what leads to general AI.

  17. Re:Answer a question for me? by Shompol · · Score: 1

    Apropos of nothing, since you're familiar with both neural nets and how the brain works

    It's funny but the only reason I understand how brain works is because I know how computer neural nets work. There are many common aspects in both operation and shortcomings.

    As far as your question goes, brain definitely has a distinct set of inputs (retinas, sensory, etc), but the output is known as "self awareness" and was not covered in my comp sci classes. It is likely that this development brings us one step closer to a working Skynet.

  18. Re:Answer a question for me? by SuricouRaven · · Score: 2

    The output of the brain actuates lots of muscles, and a few glands. The 'self awareness' is a part of the processing.

  19. Re:Answer a question for me? by narcc · · Score: 2

    The 'self awareness' is a part of the processing.

    Why do you believe silly things without evidence? The answer to the self awareness question is a clear and unambiguous "we don't know".

    --
    Required reading for internet skeptics
  20. Reinventing the math coprocessor by Etherwalk · · Score: 1

    Sounds like IBM is pushing to make these almost a specialized chip for each machine. That sounds like your client's so fat it warps space-time approach in moderately thin-client world... There needs to be a major advantage before a math coprocessor for AI becomes viable at the consumer level...

  21. Re:Brain by hackwrench · · Score: 1

    Well, it all depends on what the meaning of "brain" is.

  22. Re:What Happens When One Transistor Goes Rogue? by Archtech · · Score: 4, Insightful

    It never ceases to amuse and baffle me how even intelligent, educated Americans (such as slashdotters) allow themselves to be lured into vicious "Republican-Democrat" battles. Isn't it obvious that Demoblicans and Republicrats are just the two hands of the same power? Every minute and every quantum of attention and passion you devote to slanging off the "other" party is a minute and a quantum of attention wasted; because the American political circus has been carefully set up so that neither party can ever win decisively. Instead, you attentively watch a series of more or less random fluctuations in fortune, and whip yourself up into a rage about the character defects of the other party's politicians, all the while ignoring the psychopaths in your own chosen party. And you will never succeed in changing the government's policies by any exercise of your votes - just look at what Obama promised before BOTH of his elections, and how he gave you Dubya's third and fourth terms when you kindly elected him.

    When will we see a serious discussion on Slashdot about the underlying political system that controls both parties, and excludes everyone else? Why doesn't anyone seem to care about the impossibility of voting for a political leader who doesn't want to conduct genocidal foreign wars? How about a government that reins in the banks and declines to follow the orders of billionaires? Why don't any of you seem even slightly interested in government of the people, by the people, for the people? (In case you hadn't noticed, what you currently have is government of the people, by the servants of the rich, for the rich).

    --
    I am sure that there are many other solipsists out there.
  23. Re:Answer a question for me? by ceoyoyo · · Score: 1

    Not all artificial neural networks are strictly feed forward. There are other designs that incorporate feedback but they tend to be harder to train and use (you have to wait for them to settle), so you don't hear about them a lot in connection with practical applications. One of the major types that is not strictly feedforward but is used a lot is the recurrent ANN, which is used in speech and text recognition, and natural language processing. In that type of network, the output produced by the previous run is fed back in as an input for the next run (the next word, for example).

    The brain uses lots of different neural architectures. There are areas that use massive feedback, but other areas that are strictly feedforward. The type of ANN you're describing evolved from something called a perceptron which, as the name suggests, was inspired by models of the visual system. The mammalian visual system does a lot of feedforward processing.

  24. Re:Positronic technology? by ceoyoyo · · Score: 1

    Problem is, if your positronic brain leaks you get an antimatter explosion.

  25. Re:Answer a question for me? by morgauxo · · Score: 1

    The brain is hugely complicated with many inputs and many outputs and many many loops within.

    A neural net is a very simplified version of a brain with one set of inputs and one set of outputs, and a very limited number of loops within.

  26. Re:Answer a question for me? by SuricouRaven · · Score: 1

    You cannot observe the self-awareness thing directly (yet, anyway) but you can deduce the process occurs by observing inputs and outputs.

  27. Re:What Happens When One Transistor Goes Rogue? by morgauxo · · Score: 1

    Because most people are stupid and humanity is a hopeless case. I wonder what the neadertals and denisovans were really like. Maybe the wrong homo won.

  28. Re:Answer a question for me? by narcc · · Score: 1

    I certainly can. That's precisely what makes me self-aware.

    You mean in others. As for the alleged deduction, it's depressingly weak.

    --
    Required reading for internet skeptics
  29. Re:What Happens When One Transistor Goes Rogue? by Applehu+Akbar · · Score: 1

    "At least they have a brain. Democrats are safe from brain-eating zombies."

    But only if the food they eat is labeled for any ingredient or process they might be irrationally scared of.

  30. Pffft.. So much easier to do it this way... by doccus · · Score: 1

    Just wire up the brains of a bunch of mice. No development cost. Just a little bit of feed and a pooper scooper...

  31. Re:Answer a question for me? by micahraleigh · · Score: 1

    I studied brain theory and AI at USC in post grad.

    This is a really great question.

    Neural networks do not exist in nature anywhere for any species in the sense of the left-to-right pipeline framework you describe.

    As with any simulation, there isn't going to be 100% parity with the real thing. It simulates neural networks in the sense that they are adaptive and produce pavlovian results, associative chains, etc.

    Can't comment on to what extent the layout you describe (i.e. a traditional neural network) relates to what IBM has set up here.