IBM Researchers Propose Device To Dramatically Speed Up Neural-Net Learning (arxiv.org)

skywire writes: We've all followed the recent story of AlphaGo beating a top Go master. Now IBM researchers Tayfun Gokmen and Yurii Vlasov have described what could be a game changer for machine learning — an array of resistive processing units that would use stochastic techniques to dramatically accelerate the backpropagation algorithm, speeding up neural network training by a factor of 30,000. They argue that such an array would be reliable, low in power use, and buildable with current CMOS fabrication technology. "Even Google's AlphaGo still needed thousands of chips to achieve its level of intelligence," adds Tom's Hardware. "IBM researchers are now working to power that level of intelligence with a single chip, which means thousands of them put together could lead to even more breakthroughs in AI capabilities in the future."

Comments

Re:Won't shrink this to fit into your phone

[m00sh]

There are some major limitations with the design they have gone with for deep learning. You may think that thousands of chips will soon shrink to fit in a phone. (~15 years if moore's law holds). But thermodynamics won't let this happen, you can't flip an arbitrary number of bits for zero energy. There is a minimum amount of energy necessary for a register to perform a simple operation, and the amount needed for a deep learning system of this scale is more than you would want to comfortably power in your pocket.

This is for training. Once the training is done, the model can be used in a cell phone.

Case in point, voice recognition.