Facebook AI Director Discusses Deep Learning, Hype, and the Singularity

An anonymous reader writes In a wide-ranging interview with IEEE Spectrum, Yann LeCun talks about his work at the Facebook AI Research group and the applications and limitations of deep learning and other AI techniques. He also talks about hype, 'cargo cult science', and what he dislikes about the Singularity movement. The discussion also includes brain-inspired processors, supervised vs. unsupervised learning, humanism, morality, and strange airplanes.

Comments

Hope not

[nospam007]

It would be really cruel if Skynet awakes and wants us to 'LIKE' it.

Re:"Singularity" is a horrible term.

[topology]

Math has prior art on the use of the term. In fact, Physics' use of the term is just plain derivative.

In AI, the term singularity refers to the point where an AI can sustain its own learning and that learning outclasses what humans are capable of comprehending/predicting. Right now AIs are dependent on and limited by human instruction and guidance. I'm not talking about quantity of knowledge, but what the AI is capable of doing with that knowledge. The kind of reasoning and complexity of interaction the AI is capable of.

If the function f(x) represents the learning potential of the AI, then at the point that AI is able to teach itself and learn higher order concepts, metaphors, and thought patterns, its potential will have outclassed human potential and the function is effectively a singularity. (Might as well be infinite, since its beyond human comprehension).

On a personal note, I don't think a singularity is achievable without somehow embuing the AI with various forms of visceral sensation. Less symbolic reasoning (Chineses Room) and more experiential primitives. While human intelligence has been greatly advanced by language and formal conception, the underpinings of our concepts and understanding is still our primitive and direct experiences. We draw pictures on the board and ask our students to visualize when they are learning math. We use pictures to ground the meaning of the symbolic order.

Very informative article

[gurps_npc]

Glad to hear from an intelligent person, rather than an obsessed 'futurist' that has mistaken wishful/paranoid thinking for scientific projections.

I would have added that the concept of the 'singularity' assumes multiple 'facts' that are extremely unlikely. In part because if they were true, science would already have been much farther along. Also in part because they confabulate different definitions of words, most often 'intelligence'. When AI people are talking about intelligence they are generally not using the word in the same way that a biologist, or worse, a priest. would.

Re:Very informative article

[Tyler Durden]

2) Observe that technological progress seems to be accelerating. So, the reach of any predictive models we have today will be even shorter when used tomorrow.

I think LeCun covers this quite well when he quotes, "the first part of a sigmoid looks a lot like an exponential." There's nothing that says the acceleration of technological progress has to continue as it has.

Re:Very informative article

[gurps_npc]

One of the major mistakes is that technology is not universally disruptive. It's main disruption arises from unexpected forms of technology, not the ones we pay attention to. I.E. we get cellphones, not flying cars/jetpacks.

The progress tends to be in areas that were not gaining progress before,

In general the Singularity people believe the progress will entirely be in AI. Specifically, they think that our advancements in computer technology will continue to be in complexity etc. along the SAME lines it has already done. I hereby propose that AI will NOT have any major disruptive changes in the future. Instead it might be in something dramatically different. Maybe shoes, soap, or some other commonplace item - kind of like the phone underwent a dramatic and unpredicted change.

The major issues with the AI people is that they think all the progress in making computers have faster processing of mathematical equations will somehow create a thinking computer. We see it all the time in all the fiction. They confuse good at math for "have a soul".

Most importantly, while the Singularity people talk about unable to predict, they then go ahead and make a bunch of crappy predictions - mainly based on junk science that we know is wrong.

You want a realistic story of the creation of the first AI. AI gets created, learns to talk, explores the internet then writes a horrible, "emo" suicide note before it kills itself.

THAT would be far more likely than the crappy "humans uploads the entire race and stops having kids" junk that Singularity people like to fantasize about.

Re:Very informative article

[dpidcoe]

The big error is assuming the the accelleration will continue at the same rate it currently is. It won't.

Look at the curve for other technologies now considered "mature" fields. When they were initially discovered there were huge leaps and bounds made, then it all started to dry up once the low hanging fruit was picked. Now there's little new development except for highly specialized breakthroughs that effect some niche uses as the technology starts to encounter hard limits oh physics or limitations from other fields (e.g. manufacturing technology)

we'll see the same thing happen with computers. Eventually transistors hit the smallest physical size possible, and that's the end of moors law. Most of the really interesting things in computer science (such as these learning algorithms) are very non-linear in their computing requirements (usually some O(2^n) or worse), so all the work to increase computing power isn't going to be as much of a payoff as it's historically been. Quantum computing is only fast at certain kinds of things and so isn't going to be the savior a lot of people think it is.

Cargo Cult Science

[Kunedog]

If you've never read it before, Feynman's original essay is more worth your time (especially the part about the lab rats).
http://neurotheory.columbia.ed...

Re:Cargo Cult Science

[gweihir]

Thanks for linking that article, it is very, very insightful. I found that this term is so very descriptive about what is mostly going on in AI research that is publicly visible, it is staggering. The problem seems to be that many people cannot recognize more than the shape of a thing and are completely unaware that it does in no way describes what the thing is. That scientists fall to the same delusion is rather tragic.

As a scientist myself (now only a very small part of my time), I found that Feynman is exactly on the mark with the other things he says. Publishing full results, and in particular carefully including negative aspects, gets your papers rejected and some reviewers seem positively believe that you are incompetent. Very, very much of CS research works this way: You have to deliver positive results or be ignored. One of the effects for me was that I have best paper awards on about 10% of my papers (and many of them were difficult to get published), but never had any chance to get a permanent scientific position or a professorship. This means that, except for accidents, most CS research is done by people that can give the appearance of having positive results, while their "research" hardly is scientific at all. That is doing a huge disservice to anybody involved and affected. It is also the reason why so much CS research "results" fail when applied in practice. You just need to look at the string of hypes that have failed over the last few decades in CS, time and again. Yet people do not learn.

Re:Priest?

[itzly]

A priest is somebody that tells other people to believe. It's not required that the priest holds these beliefs himself.

Re:AI endpoint is key

[itzly]

You won't get AI by messing with some genetic algorithm for a day, trying to do something completely different. The search space is just too big to stumble upon AI accidentally.

Perhaps Yann LeCun needs an AI to count for him

[DumbSwede]

“machines that learn to represent the world.” That’s eight words.

methinks seven :-)

Re:Priest?

[itzly]

Religion is simply a method to wield power over the weak minded. Actually believing the stuff yourself only gets in the way.

Re:"Singularity" is a horrible term.

[ShanghaiBill]

It would need agreement from John Conner.

Unless the AI was dumb, this is what would happen to John Conner.

What about the surface learning of listening?

[Blain]

Like listening to the preferences users have selected about silly things like what order they want items in their feed listed? I know you love these whiz-bang prediction algorithms, but they suck at predicting what I want. I'm really good at asking for what I want, and changing those settings to what you want will never ever do a better job than letting me pick. I promise.