Deep Learning Pioneer On the Next Generation of Hardware For Neural Networks

An anonymous reader writes: While many recognize Yann LeCun as the father of convolutional neural networks, the momentum of which has ignited artificial intelligence at companies like Google, Facebook and beyond, LeCun has not been strictly rooted in algorithms. Like others who have developed completely new approaches to computing, he has an extensive background in hardware, specifically chip design and this recognition of specialization of hardware, movement of data around complex problems, and ultimately core performance, has proven handy. He talks in depth this week about why FPGAs are coming onto the scene as companies like Google and Facebook seek a move away from "proprietary hardware" and look to "programmable devices" to do things like, oh, say, pick out a single face of one's choosing from an 800,000 strong population in under five seconds.

Comments

The problem with neural networks

[Viol8]

Is that *in theory* you could understand why they come to a particular result, but in practice it could be potentially very hard with a large network for any person to get their head around the processes leading up to the output. This means that unless safety rules are changed we won't be seeing these things driving cars or flying aircraft anytime soon since the software needs to be verifiable and neural networks are not.

Re:The problem with neural networks

[Sneeka2]

Or, arguably, we need to change our definition of "verifiable"... For complex activities such as driving cars, we're reaching the limits of traditionally programmed computers. A human programmer cannot possibly think of every possible situation a car might encounter on the street and pre-program an appropriate response into the car. Neural networks and "artificial intelligence" doesn't have a pre-programmed response, but could come up with one based on patterns it knows. So it becomes more about giving the machine a robust basis to work on and then training it... just like a human gathers experiences and then applies them to new situations.

Re:The problem with neural networks

[ziggystarsky]

Fortunately we can understand the processes within real people that lead to their actions. This is the reason that we safely let them drive cars, trains or fly planes.

Re:The problem with neural networks

[fuzzyfuzzyfungus]

Well, I think that the standards for driving tests could use some modification; but I was actually aiming at exactly the opposite point: There isn't any particular reason to believe that we need to, or will, demand that machines that control vehicles be submitted to some sort of profound understanding and formal verification, given that we accept black-box testing(and pretty shoddy testing at that) for human operators.

The initial ,lobbying might be a fairly ghastly pain; but I see no reason why there would be any long-term resistance to complex systems(neural network or otherwise) that are effectively beyond human understanding; so long as they pass black box tests of their abilities. I say this both because that's what we do when dealing with people; and because, in practice, even today's tech is complex enough that effectively nobody outside of very specialized software dev and test outfits knows what the hell is going on; and people basically accept that, because the alternative involves being restricted to radically simpler technology or radically more expensive tech support.

Re:The problem with neural networks

[Sneeka2]

Sure, it's all extremely difficult. I'd think with neural networks you can use an evolutionary approach and eventually choose the program which has evolved and performed best over a series of X million of tests. The question "when is the program done" doesn't mean "when has the programmer thought of every last possibility" anymore, but rather "when are we satisfied enough with the statistics that we trust this program enough?"

Re:The problem with neural networks

[orasio]

You make a very interesting point.
With automation, it's a lot easier for us to accept a given amount of understandable failure, than a much smaller amount of inexplicable failure. That might be a roadblock against some forms of automation.

In any case, there's also economics, which do like statistics, and will make you choose the strategy that fails less, overall. For example, insurance companies might favour driving algorithms that crash less often vs ones that crash a bit more often, but for better known causes.

Re:Can this bubble burst already?

[lorinc]

I don't know. There is a part of me that says "yeah this is a bubble that's going to burst soon", and another part that says "wait, you've never seen that much improvement on such complex tasks before". Probably the future is in between, and parts of the deep conv nets are here to stay, while some others parts will rapidly be forgotten. But frankly, I don't know, which is a bit scary.