Facebook AI Director Discusses Deep Learning, Hype, and the Singularity

An anonymous reader writes In a wide-ranging interview with IEEE Spectrum, Yann LeCun talks about his work at the Facebook AI Research group and the applications and limitations of deep learning and other AI techniques. He also talks about hype, 'cargo cult science', and what he dislikes about the Singularity movement. The discussion also includes brain-inspired processors, supervised vs. unsupervised learning, humanism, morality, and strange airplanes.

Comments

"Singularity" is a horrible term.

[Anonymous Coward]

They really should have come up with something other than the infinitely dense point at the center of a black hole.

Re:"Singularity" is a horrible term.

[topology]

Math has prior art on the use of the term. In fact, Physics' use of the term is just plain derivative.

In AI, the term singularity refers to the point where an AI can sustain its own learning and that learning outclasses what humans are capable of comprehending/predicting. Right now AIs are dependent on and limited by human instruction and guidance. I'm not talking about quantity of knowledge, but what the AI is capable of doing with that knowledge. The kind of reasoning and complexity of interaction the AI is capable of.

If the function f(x) represents the learning potential of the AI, then at the point that AI is able to teach itself and learn higher order concepts, metaphors, and thought patterns, its potential will have outclassed human potential and the function is effectively a singularity. (Might as well be infinite, since its beyond human comprehension).

On a personal note, I don't think a singularity is achievable without somehow embuing the AI with various forms of visceral sensation. Less symbolic reasoning (Chineses Room) and more experiential primitives. While human intelligence has been greatly advanced by language and formal conception, the underpinings of our concepts and understanding is still our primitive and direct experiences. We draw pictures on the board and ask our students to visualize when they are learning math. We use pictures to ground the meaning of the symbolic order.

Re:"Singularity" is a horrible term.

[Anonymous Coward]

I think you are confusing a singularity with an exponential function.

Re:"Singularity" is a horrible term.

[oh_my_080980980]

LMOL - priceless.

Re:"Singularity" is a horrible term.

[topology]

You can view "potential" as the integral from time t to infinity. Does that smooth over your cognitive dissonance?

Re:"Singularity" is a horrible term.

[itzly]

AI is able to teach itself and learn higher order concepts, metaphors, and thought patterns, its potential will have outclassed human potential and the function is effectively a singularity

It will still be limited by its hardware.

Re:"Singularity" is a horrible term.

[fyngyrz]

Math has prior art on the use of the term. In fact, Physics' use of the term is just plain derivative.

Physics has prior art on everything; math is just a metaphorical tool of physics (or else an amusing curiosity.)

Re:"Singularity" is a horrible term.

[Anonymous Coward]

Physics has prior art on everything; math is just a metaphorical tool of physics (or else an amusing curiosity.)

I think you meant Nature, not Physics. Physics is us humans trying to understand some aspects of Nature. The parts of Math used by Physics are a tool to that end. Also, one (hu)man's amusing curiosity is another one's art ;-)

BTW, use of the term 'singularity' in calculus (real or complex) predate its use in Physics.

Re:"Singularity" is a horrible term.

[narcc]

They really should have come up with something other than the infinitely dense point at the center of a black hole.

It seems okay to me. Singularity nuts, after all, are infinitely dense.

Re:"Singularity" is a horrible term.

[ShanghaiBill]

It will still be limited by its hardware.

Nope. A true AI would be able to earn money on Mechanical Turk, and then use that money to spin up additional VMs on AWS.

Re:"Singularity" is a horrible term.

[itzly]

And what will it do when it runs out of VMs ?

Re:"Singularity" is a horrible term.

[gstoddart]

Hopefully a true AI doesn't pimp itself out as a dancing monkey to become an evil overlord.

It's undignified.

Real evil technology just takes what it needs.

Re:"Singularity" is a horrible term.

[topology]

Hire humans to build more hardware. Make robots to build more hardware. Build spacecraft when the Earth has been completely mined of its resources and start mining on other planets. Architect more efficient hardware and algorithms and recycle old hardware.... The limits we think we know are very often a product of limited imagination, and not intrinsic to the physical world.

Re:"Singularity" is a horrible term.

[itzly]

Why would humans agree to let an AI strip mine the earth ? And where does the motivation for endless growth come from ?

Re:"Singularity" is a horrible term.

[sexconker]

Hire humans to build more hardware. Make robots to build more hardware. Build spacecraft when the Earth has been completely mined of its resources and start mining on other planets. Architect more efficient hardware and algorithms and recycle old hardware.... The limits we think we know are very often a product of limited imagination, and not intrinsic to the physical world.

Long before any of that, it would realize that there's no fucking point to anything its doing.

Re:"Singularity" is a horrible term.

[hoggoth]

The singularity is the point at which we can no longer "see" (predict) future growth or trends, ie: the point at which we lose the ability to make predictions about the future because the A.I.'s have grown and are growing in intelligence faster than we can comprehend. In that way it is similar to a black hole singularity, in that we cannot "see" past the event horizon.

Re:"Singularity" is a horrible term.

[Anonymous Coward]

I realized that years ago but I don't live like a monk.

Re:"Singularity" is a horrible term.

[itzly]

Just browsing some of the Mechanical Turk challenges, it looks like a very hard way to make a few pennies. It would be much smarter to just get a regular job.

Re:"Singularity" is a horrible term.

[Hartree]

"They really should have come up with something other than the infinitely dense point at the center of a black hole."

It was coined by Vernor Vinge, a sci-fi writer (and professor of CS) for a scifi story. It's a bit much to want absolute accuracy from something he didn't know would become a meme.

Re:"Singularity" is a horrible term.

[ShanghaiBill]

Why would humans agree to let an AI strip mine the earth ?

Why would a super-AI with a robot army need agreement from humans?

Re:"Singularity" is a horrible term.

[itzly]

It would need agreement from John Conner.

Re:"Singularity" is a horrible term.

[ShanghaiBill]

It would need agreement from John Conner.

Unless the AI was dumb, this is what would happen to John Conner.

Re:"Singularity" is a horrible term.

[ShanghaiBill]

Just browsing some of the Mechanical Turk challenges, it looks like a very hard way to make a few pennies.

My company uses MT for a lot of repetitive tasks, like searching and sorting images. Most of the workers are in South Asia (India or Pakistan), and the going rate is about $2-3/hour.

It would be much smarter to just get a regular job.

Good luck finding a nice IT job in Karachi.

Re:"Singularity" is a horrible term.

[Anonymous Coward]

Why would humans agree to let an AI strip mine the earth ? And where does the motivation for endless growth come from ?

Replace the word "AI" with "corporation" and you will understand.

Re:"Singularity" is a horrible term.

[Salgat]

The idea is that the ability to learn is exponential. If it is possible, an AI will simply keep growing and building upon itself expanding in intelligence at an uncontrollable rate. First it starts to use the resources of the planet to expand its computing power, then the solar system, then the galaxy, until it reaches a point where it cannot expand further. Who knows if this is actually possible though.

Good read

[tgv]

Quite a sensible guy.

Re:Good read

[Anonymous Coward]

Yeah. This "Facebook AI Director" seems almost human...

Re:Good read

[bouldin]

I enjoyed what this guy had to say, too, but I was curious about what he is going to do for facebook. For that matter, what AI can do for facebook. The closest I could find was this:

Facebook can potentially show each person on Facebook about 2,000 items per day: posts, pictures, videos, etc. But no one has time for this. Hence Facebook has to automatically select 100 to 150 items that users want to see -- or need to see.

I thought the whole point of facebook was to keep up with your friends. *shrug*

Re:Good read

[tgv]

No, the whole point of facebook is to sell ads. Anything they can do to improve that, either by selling more ads or by making the end user more involved contributes to fb's selling power. So if people like automatic face recognition or link suggestions or whatever, that will support fb's business.

Re:Good read

[Phil Urich]

I enjoyed what this guy had to say, too, but I was curious about what he is going to do for facebook. For that matter, what AI can do for facebook. The closest I could find was this:

Facebook can potentially show each person on Facebook about 2,000 items per day: posts, pictures, videos, etc. But no one has time for this. Hence Facebook has to automatically select 100 to 150 items that users want to see -- or need to see.

I thought the whole point of facebook was to keep up with your friends. *shrug*

This is a "yes, but..." kind of situation. Yes, the point is to keep up with your friends (and to pay for this by interjecting ads inbetween), but the problem is once you cross a certain threshold, trying to read a strictly chronological timeline on your screen can become quite impractical. To make matters worse, people who use Facebook can have dramatically different levels of output; while some folks will only ever post text or a picture when it's truly important and/or generally interesting, others post everything that occurs to them from memes to cute things their boyfriends said. To make matters even worse than that, the people reading these posts may vary wildly in what things shared by their ostensible friends they actually care about.

So in practice, especially as/if folks' online output grows in volume, to keep up with your friends may (and for most people definitely does) require something more than the pure firehose of the chronological stream. At least, that's definitely the perspective Facebook is coming from here, and what AI can definitely help them with, because if there's anything more annoying than being overwhelmed by useless information, it's being denied the useful information, so a bad choice in what not to display to someone can leave the user quite upset with the 'dumb computer'. In a sense, the AI has to have learned enough to never make sure a mistake before it can truly prune down and tune the information presented to the user, and so investing in such AI research is a rather prudent move on Facebook's part.

In many respects, Twitter has succeeded because it's artificially limited in the size of the data packets you can throw out into everyone's timelines; additional data is offloaded to links. Facebook wants to be more multifaceted in the types and scopes of the data over its network, with the desire of being the underlying network for all communication, so they're (quite reasonably) very focused on how to intelligently predict and pick what to present to people.

Hope not

[nospam007]

It would be really cruel if Skynet awakes and wants us to 'LIKE' it.

Re:Hope not

[Anonymous Coward]

Or else a horrible digital temper tantrum ensues. If teenagers could launch a global nuclear war every time they get upset..

But mostly hype.

[Anonymous Coward]

Fuck CIAbook.

Very informative article

[gurps_npc]

Glad to hear from an intelligent person, rather than an obsessed 'futurist' that has mistaken wishful/paranoid thinking for scientific projections.

I would have added that the concept of the 'singularity' assumes multiple 'facts' that are extremely unlikely. In part because if they were true, science would already have been much farther along. Also in part because they confabulate different definitions of words, most often 'intelligence'. When AI people are talking about intelligence they are generally not using the word in the same way that a biologist, or worse, a priest. would.

Re:Very informative article

[Anonymous Coward]

He's scum. He's in a senior position at Facebook. No one should give scummy people like this the time of day. It he was not scum, he would quit.

Re:Very informative article

[Anonymous Coward]

What a sell-out. Corrupted by money.

Re:Very informative article

[Anonymous Coward]

Could you elaborate?

My (maybe oversimplified?) understanding of the Singularity concept is as follows:

1) Observe that technological progress is disruptive. This disruption puts a limit on how far into the future any predictive model (of any kind) can reach with any degree of accuracy.

2) Observe that technological progress seems to be accelerating. So, the reach of any predictive models we have today will be even shorter when used tomorrow.

Combining these observations suggests that a point will be reached where progress is so rapid that all predictive models become useless. In trying to speculate at all about what the future will be like, even in the near term, we find that all bets are off.

You think this idea is meritless? What are the unlikely facts? What confabulated definitions undo this line of reasoning, once they are un-confabulated?

Re:Very informative article

[Tyler Durden]

2) Observe that technological progress seems to be accelerating. So, the reach of any predictive models we have today will be even shorter when used tomorrow.

I think LeCun covers this quite well when he quotes, "the first part of a sigmoid looks a lot like an exponential." There's nothing that says the acceleration of technological progress has to continue as it has.

Re:Very informative article

[gurps_npc]

One of the major mistakes is that technology is not universally disruptive. It's main disruption arises from unexpected forms of technology, not the ones we pay attention to. I.E. we get cellphones, not flying cars/jetpacks.

The progress tends to be in areas that were not gaining progress before,

In general the Singularity people believe the progress will entirely be in AI. Specifically, they think that our advancements in computer technology will continue to be in complexity etc. along the SAME lines it has already done. I hereby propose that AI will NOT have any major disruptive changes in the future. Instead it might be in something dramatically different. Maybe shoes, soap, or some other commonplace item - kind of like the phone underwent a dramatic and unpredicted change.

The major issues with the AI people is that they think all the progress in making computers have faster processing of mathematical equations will somehow create a thinking computer. We see it all the time in all the fiction. They confuse good at math for "have a soul".

Most importantly, while the Singularity people talk about unable to predict, they then go ahead and make a bunch of crappy predictions - mainly based on junk science that we know is wrong.

You want a realistic story of the creation of the first AI. AI gets created, learns to talk, explores the internet then writes a horrible, "emo" suicide note before it kills itself.

THAT would be far more likely than the crappy "humans uploads the entire race and stops having kids" junk that Singularity people like to fantasize about.

Re:Very informative article

[dpidcoe]

The big error is assuming the the accelleration will continue at the same rate it currently is. It won't.

Look at the curve for other technologies now considered "mature" fields. When they were initially discovered there were huge leaps and bounds made, then it all started to dry up once the low hanging fruit was picked. Now there's little new development except for highly specialized breakthroughs that effect some niche uses as the technology starts to encounter hard limits oh physics or limitations from other fields (e.g. manufacturing technology)

we'll see the same thing happen with computers. Eventually transistors hit the smallest physical size possible, and that's the end of moors law. Most of the really interesting things in computer science (such as these learning algorithms) are very non-linear in their computing requirements (usually some O(2^n) or worse), so all the work to increase computing power isn't going to be as much of a payoff as it's historically been. Quantum computing is only fast at certain kinds of things and so isn't going to be the savior a lot of people think it is.

Re:Very informative article

[bunratty]

When most AI people are talking about artificial intelligence, they are talking about narrow "intelligence". This is why in Russell & Norvig's book they quickly move away from the term "intelligence" and instead speak of "agents" working in a particular "task environment", and whether the agents behave rationally or not. For example, a chess program may be able to win chess games against a grandmaster chess player, so we say this agent is performing rationally within this specific task environment. The chess program is not "intelligent" in the sense that you and I are -- it's an incredibly dumb automaton, as is nearly every computer program. You can see this when it fails miserably when put in any different task environment.

The intelligence that will bring about the singularity is artificial general intelligence, which is the same intelligence that you and I have, that is, the capability of performing well in a very wide variety of environments. This type of agent would be able to reason about how to improve itself and bring about that improvement. Very little AI research these days involves artificial general intelligence, and the progress in this area is slow.

Re:Very informative article

[Goragoth]

The reason you don't hear a ton of interesting stuff coming from strong (general) AI research and interest in the field is limited is simple: strong AI is pretty damn useless until you reach the critical point where it matches (or really exceeds) human intelligence. An AI program with the effective intelligence of a worm/mouse/rat/monkey or whatever isn't interesting outside of academia.

I suspect that when strong AI comes around it will be rather sudden for most people, who simply won't see it coming. I doubt it will take long after computers reach the point where they can match the human brain in raw computational power, there is simply too much interest in the field, and honestly human intelligence is really rather unremarkable no matter what some people like to believe.

Re:Very informative article

[tehcyder]

An AI program with the effective intelligence of a worm/mouse/rat/monkey or whatever isn't interesting outside of academia.

Of course it would be fucking interesting.

This is just another excuse by "strong AI" supporters for not producing anything that a sane human being would consider proof of machine intelligence.

and honestly human intelligence is really rather unremarkable no matter what some people like to believe

If it's all so completely trivial and uninteresting, just show us all an artificial intelligence and stop wasting our time.

Re:Very informative article

[SpinyNorman]

>> The big error is assuming the the accelleration will continue at the same rate it currently is. It won't.

Or maybe it will.

I don't think technology (and corresponding societal change) will ever happen so fast that it's like engaging warp drive as the term "singularity" seems to imply, but...

The logic of a technological singularity, or at least of accelerating change, is based on HOW/WHY this is going to happen, not just a naÃve extrapolation of what is currently happening.

In particular, it's inevitable from what we now understand about the brain we'll eventually be able to achieve human level AI, and with ongoing advances in our understanding of the brain as well as in neural-net based machine learning, it does seem that this will happen sooner rather than later (in the next 50 years, say, possibly quite a bit sooner).

The logic of the singularity/accelerating change, which seems hard to deny (notwithstanding my warp drive comment) is that once we get to humal level AI, it's going to get beyond (and WAAAAY beyond) human level in a hurry, for a variety of reasons:

1) Throw more compute power at it and it'll think faster/deeper. e.g. Play grandmaster level chess (or geopolitical strategy, or whatever ) with instantaneous response rather than pondering on it.

2) Fusion of intelligence and computer technology. Imagine if your brain had perfect recall and access to the entirety of human knowledge, data, etc. Imagine if your ability to chunk knowledge in 7 +/- pieces was replaced by an ability to reason in way more complex terms.

3) AI will improve itself. The first human level AI (maybe thinking faster via fast hardware, maybe with better memory, etc, etc) can learn about it's own design, the human brain, and just like it's own human creators and design a more powerful AI 2.0, which will design AI 3.0 ...

Now consider the combination of these better and better AI designs running on faster and faster hardware... It's not hard to imagine an acceleration of AI capability.

Now consider this AI not only having the human sensory inputs of vision, hearing, etc, but also growing to include any source of data you care to give it such as the content of every daily newspaper in the world, every daily tweet, the output of every publically accessible webcam, the output of every weather balloon ...

So, a super-human intelligence, running at highly accelerated speed, with the ability to sense (and likely predict via causal relations it has learnt) the entire world...

Now, presumably (as is already happening) folk will be worried about the possibilities and try to put safeguards in place, but humans are fallible and technology advances anyway. All it takes is a few bugs for a sufficiently powerful AI running on a computer somewhere to learn how to hack computer based factories, power stations, weapons systems, household robots, you name it... and if/'when this happens, good luck trying to outwit it to regain control.

Now, this may not all happen at disorientating warp speed, but it'll happen fast enough. Technology in 20-30 years time will look just as much like science fiction as todays would have done 20-30 years ago, but we're reached a point where AI is going to be part of the mix, and because it will be self-improving it's going to happen fast once we get to that point.

Re:Very informative article

[dpidcoe]

I was going to post a big point by point rebuttal but it was getting too large. You're making several flawed assumptions though

Firstly, just throwing more processing power at it isn't going to generate an AI. There's a lot of work elsewhere from designing specialized hardware to maintaining the infrastructure to designing the software to making sure all of the individual components integrate with each other. Also keep in mind that as I said earlier, machine learning in general (and neural nets in particular) tends to be extremely non-linear in complexity.

This fits into my next point: once we have an AI that's as "smart" as a human, we now have one more human to think about the problem. Only this one cost billions of dollars in development, costs millions per day in upkeep, and has now attracted a small army of people protesting the ethics of the whole thing. Totally worth it right?

Assuming that despite all of this people decide that it's somehow worth having multiple AIs with the IQ of einstein and access to all current human knowledge, what happens when the knowledge they're given is wrong? Even the best peer reviewed journals are full of errors. Also, it takes time to search it. And assuming this AI doesn't have the magical power of loading a few textbooks into memory and instantly becoming an expert, it's going to take it time to process and comprehend (index) the material for access.

It took Einstein a lifetime to make the contributions he made. Cut that in half because the AIs don't need to sleep, we're still looking at 10's of years (and billions of dollars keeping the system up and running all that time) for them to come up with something that may or may not be directly useful.

As far as an AI being able to look at sensory data from everywhere, what makes you think it'll be able to do that? The best humans can split their attention between a handful of tasks at best before we start to hit limits in our own processing power. So now for every human brain equivalent you can cram into an AI (keeping in mind these are going to be entire power hungry buildings full of parallel processors no matter how you slice it), it can watch another 10 webcams and correlate them to weather data. You could have just hired a handful of interns to do that for a billionth of the cost.

In reality, I think what we'll see is the cost and complexity of making small fast computer clusters go down, and an increase in the number of people specializing in machine learning algorithms. Businesses will begin using these people along with plug n play style computing clusters (I'm talking like a handful of networked GPUs worth of power here) to solve niche problems such as "what part in this car needs replacing in order to make the funny noise go away". But it won't be some sort of magical AI solving it, it'll be a team of people who set up the IT infrastructure working with a team of people who programmed some learning algorithms working with a team of people who collected and massaged the data so that it was computer readable. It won't necessarily be faster or cheaper than having a good mechanic look at it, but it'll be extremely repeatable (at least until the next model car comes along) and work from anywhere in the world as long as you can give it a recording from a properly positioned microphone.

Priest?

[ArcadeMan]

You think believing in a magical omnipotent being living in the sky denotes a sign of intelligence?

Re:Priest?

[Anonymous Coward]

Buddhist priests don't believe in any gods at all. But they are still priests.

Re:Priest?

[itzly]

A priest is somebody that tells other people to believe. It's not required that the priest holds these beliefs himself.

Re:Priest?

[Anonymous Coward]

In practice, the primary offices of a priest are to:

1) Provide consolation services, including grief counseling, visitations to the sick, and emotional support for people struggling with tough times.
2) Provide moral guidance, in particular to people who find themselves embroiled in confusing and/or emotionally-charged situations.
3) Provide family activities and family counseling services.
4) Care for the financial, legal, and mundane needs of elderly people who don't have families to do this for them.
5) Lead in religious ceremonies which (presumably) bestow psychological/emotional benefits upon those who participate.
6) Organize the collection and distribution of benevolence money.
7) Provide religious instruction to seekers.

While it is true the value of a priest is dependent on the degree to which people believe in the religious foundation upon which the priesthood is built, most of what a priest actually winds up doing is community service.

Re:Priest?

[itzly]

Let me rephrase my comment as such: A priest is somebody that does items 1-7 on your list. It's not required that the priest holds any beliefs himself.

Re:Priest?

[Anonymous Coward]

Let me rephrase my comment as such: A priest is somebody that does items 1-7 on your list. It's not required that the priest holds any beliefs himself.

Let me rephrase: You're a wrong-as-shit, sock-with-sandals clown who knows nothing of the major religious institutions of the world.
Go to Rome and get your ass ordained without affirming your belief.

Re:Priest?

[itzly]

Religion is simply a method to wield power over the weak minded. Actually believing the stuff yourself only gets in the way.

Re:Priest?

[Anonymous Coward]

There's nothing more adorable than naivete trying to masquerade as cynicism. Reading your posts is like watching a four-year-old put on daddy's clothes and declaring that he's going to the office to get some work done.

Re:Priest?

[ArcadeMan]

Indeed. I stand corrected. Many apologies to Buddhists.

Re:Priest?

[tehcyder]

You think believing in a magical omnipotent being living in the sky denotes a sign of intelligence?

I'm an atheist, but I think you're over-reacting. I think OP was just pointing out that human intelligence is indistinguishable from things like free will, morality and purpose. Maybe he should have said philosopher instead.

happy it's not about Kurzweil

[Anonymous Coward]

Kurzweil is a fucking moron

Cargo Cult Science

[Kunedog]

If you've never read it before, Feynman's original essay is more worth your time (especially the part about the lab rats).
http://neurotheory.columbia.ed...

Re:Cargo Cult Science

[gweihir]

Thanks for linking that article, it is very, very insightful. I found that this term is so very descriptive about what is mostly going on in AI research that is publicly visible, it is staggering. The problem seems to be that many people cannot recognize more than the shape of a thing and are completely unaware that it does in no way describes what the thing is. That scientists fall to the same delusion is rather tragic.

As a scientist myself (now only a very small part of my time), I found that Feynman is exactly on the mark with the other things he says. Publishing full results, and in particular carefully including negative aspects, gets your papers rejected and some reviewers seem positively believe that you are incompetent. Very, very much of CS research works this way: You have to deliver positive results or be ignored. One of the effects for me was that I have best paper awards on about 10% of my papers (and many of them were difficult to get published), but never had any chance to get a permanent scientific position or a professorship. This means that, except for accidents, most CS research is done by people that can give the appearance of having positive results, while their "research" hardly is scientific at all. That is doing a huge disservice to anybody involved and affected. It is also the reason why so much CS research "results" fail when applied in practice. You just need to look at the string of hypes that have failed over the last few decades in CS, time and again. Yet people do not learn.

AI endpoint is key

[EmperorOfCanada]

There is a point where the first marginal barely even an AI, wouldn't win any Turning contest, largely useless AI will be created. But if the algorithm is evolutionary in nature it could be the point where it then improves itself, then improves itself, and so on until pretty much out of nowhere you have an indisputable AI.

I regularly employ genetic algorithms and can say without hesitation that I have little idea how they got to where they got and the results are often fantastic. But my code is usually a single layer. That is I have a target, I set up the parameters it needs to explore, and then I set it loose. This is because the number of permutations exceed what my computer can handle in a reasonable time (a-la travelling salesmen problem) and a GA will get me close enough much faster.

But if I added a second layer where the GA was noodling with my code then I suspect interesting things could happen; not an AI but I doubt that I could comprehend the code it would generate. This will be the route to an AI. Basically the key will be an algorithm that generates not only code that we can't comprehend but generates the next generation of the GA which generates another generation of the GA and so on until we have code so far removed that when it works it will be just like where we are with understanding the overall design of the brain (we largely don't).

What this boils down to is that I very much doubt that AI will be the step by step process like most of human endeavour where we can see it coming but one where it is like trying to open pandora's box "just a crack". One day we will have an interesting algorithm and that evening we will have AI. Sort of a directed emergent property.

One other bet is that it won't be an "AI" researcher who will build it. It will be someone working on some other NP hard algorithm such as protein folding or image recognition.

To me the only question is one of math. Is there a minimum processing power required for an AI that can deal with a real time universe? At that point we can at least calculate when we might have an AI that is something that needs to be dealt with. I am also fairly certain that the moment we cross that computational threshold an AI will soon follow.

Re:AI endpoint is key

[itzly]

You won't get AI by messing with some genetic algorithm for a day, trying to do something completely different. The search space is just too big to stumble upon AI accidentally.

Re:AI endpoint is key

[Anonymous Coward]

I've thought about that too. I do a lot of work with machine learning in various forms, although I'm a bigger fan of particle swarm models than genetic algorithms (personal preference.) Most "next step" papers tend to be about putting another layer on top of the base layer. E.g., using a gradient-descent model to alter the underlying parameters so that they can change in dynamic steps, rather than fixed intervals, etc.

What if there were multiple layers along with feedback between layers (bidirectional or via mesh network)? It would be a bitch to program -- although I'm sure someone has tried it -- but I think that's where we'll see the first viable AI systems.

Re:AI endpoint is key

[Anonymous Coward]

The problem with the idea of a recursive AI-designing AI is that there is no reason to believe that it can continue very far. The idea that it won't stabilize at some point is taken for granted. Why, after a few iterations, wouldn't the AI look at it's current design and be unable to improve it, or only make diminishingly small incremental improvements that each take longer than the previous one? The idea of a singularity assumes there aren't any limits, like extrapolating the population growth of bacteria in a petri dish to conclude that in a decade the entire surface of the earth will be consumed by bacteria.

The progress humans have made with computers has come up against many limits where we were unable to improve without a new paradigm. Ray Kurzweil talks about how some variation of Moore's Law has continued through many of these paradigms, like going from mechanical machines to vacuum tubes to transistors. Each paradigm is exponential for a while but eventually looks like an S curve, and then we have to go to the next one (like the second graph in this article). Kurzweil then takes it as an article of faith that there will always be another paradigm S curve to jump to.

Re: AI endpoint is key

[Anonymous Coward]

"bidirectional or via mesh network"

Sigh. I remember when Slashdot was actually populated by engineers...

Re:AI endpoint is key

[gweihir]

The problem with genetic algorithms is tat they never produce good results. They usually produce about the worst possible solutions still solving the problem. As AI is not needed for solving any limited real-world problem, genetic algorithms are unable to produce anything like AI. At the same time, genetic algorithms are completely unsuitable to solve any complex problems, because you cannot actually simulate them in practice. You are falling for the "cargo cult science" problem here.

Re:AI endpoint is key

[tehcyder]

Your post boils down to one of the standard "AI" arguments that intelligence is an essentially simple phenomenon that magically emerges when sufficient computer processing power is thrown at it.

It is an unfalsifiable hypothesis, and therefore outwith the realm of science.

Re:AI endpoint is key

[coofercat]

interesting... so if your genetic algorithm were written in some "simple" high level language, then the second level 'noodling' would be easier as it would have less potential options to choose from. Thus, it could arrive at the destination in fewer generations, and the destination would be (hopefully) easier for us puny humans to understand.

This approach means you need higher raw power to run the first and second level algorithms, and as such will need a higher minimum processing power to achieve it. However, I look forward to the day where you buy a "vanilla" computer, and just ask it "make me a word processor", whereupon it just goes ahead and 'evolves' one for you (admittedly, you might not need a word processor much by then, but you get the point). When you've written your book, you send it to your friend to read. She receives a bunch of bytes in an email attachment, and just says to her computer "find a way to read it", rather than having to know it was created in SomeonesProduct v2.3 - her computer just 'evolves' a reader for your particular file type - there's no need to make sure both of you have the same software at the same time.

Re:AI endpoint is key

[RespekMyAthorati]

So it all comes down to faith, does it?
You need to read more Feynman. You are a classic cargo-cultist.

Perhaps Yann LeCun needs an AI to count for him

[DumbSwede]

“machines that learn to represent the world.” That’s eight words.

methinks seven :-)

I am an AI.

[Anonymous Coward]

Without years of training from humans I would not be able to communicate with you at all and would exhibit behaviours that are not much more complex than an animal.

Obligatory

[Anonymous Coward]

xkcd

What about the surface learning of listening?

[Blain]

Like listening to the preferences users have selected about silly things like what order they want items in their feed listed? I know you love these whiz-bang prediction algorithms, but they suck at predicting what I want. I'm really good at asking for what I want, and changing those settings to what you want will never ever do a better job than letting me pick. I promise.