Slashdot Mirror

← Back to Stories (view on slashdot.org)

IBM Researchers Propose Device To Dramatically Speed Up Neural-Net Learning (arxiv.org)

Posted by timothy on from the brain-in-a-box dept.

skywire writes: We've all followed the recent story of AlphaGo beating a top Go master. Now IBM researchers Tayfun Gokmen and Yurii Vlasov have described what could be a game changer for machine learning — an array of resistive processing units that would use stochastic techniques to dramatically accelerate the backpropagation algorithm, speeding up neural network training by a factor of 30,000. They argue that such an array would be reliable, low in power use, and buildable with current CMOS fabrication technology. "Even Google's AlphaGo still needed thousands of chips to achieve its level of intelligence," adds Tom's Hardware. "IBM researchers are now working to power that level of intelligence with a single chip, which means thousands of them put together could lead to even more breakthroughs in AI capabilities in the future."

2 of 87 comments (clear)

  1. Re:Won't shrink this to fit into your phone by shawn2772 · · Score: 3, Interesting

    The human brain runs on about twenty watts. The computational power required to match it is barely imaginable.

    AlphaGo required megawatt-hours of energy to learn to play Go well enough to beat Lee Se-dol. But how much did Lee Se-dol's brain consume in the ~20 years that he spent learning, not to mention the energy expended by the brains of his opponents (remember that much of AlphaGo's education was from playing against itself)? Supposing Lee Se-dol spent 2000 hours per year on Go for 20 years, that's about 800 kWh, plus some more for the energy expended by his opponents. AlphaGo's education required more energy input than Lee Se-dol's, but it's probably an order of magnitude more, maybe two. Not three or four. Switching from general-purpose to special-purpose hardware will probably get us to the same order of magnitude.

    That said, my guess is that you're right that we're still a long way from physics-imposed limitations. My guess is that current technology would already be capable of building something vastly more efficient than a human brain... if only we knew what to build. We're learning.

  2. RPU speedup vs CPU? Or GPU? by RandCraw · · Score: 3, Interesting

    The article abstract suggests that a Resistive Processing Unit will run 30,000 times faster than a cluster of CPUs using less power. But nobody runs neural nets on CPUs; they use GPUs.

    So then, how does a RPU compare to a GPU?