Marvin Minsky: It's 2001. Where is HAL?

ZigZag writes: "Marvin Minsky speaks about everything important (MUDs, education, AI, N(atural) I, immortality) while fighting with his MS Word for Mac presentation slides at the Game Developers Conference. Transcript, audio and video are available from Dr. Dobbs. It was in part a preview of his upcoming book The Emotion Machine. Some quotes from the talk will give you a feel: "Whenever you see a number, you should say `how sad'"; "Have you heard the theory that to learn something you should do it in little bits and not stay up all night working on it? If that were true, there would be no computer games"; "robotics people treasure their videos - because it won't work tomorrow.""

Comments

Re:It's Time for Dr. Minsky to Retire

[Jonathan]

The part that literally floored me is "where you're hoping you won't have to figure anything out,"

I'm no fan of old-school AI, but Minsky has a point -- people use genetic algorithms and neural nets to "learn" from examples, but such pattern matching tells us *nothing* about how learning really happens. They are just generic black boxes that people throw at data in the hope that something useful comes out.

Slashback two weeks ago

[QuantumG]

here. Thanks for playing.

Re:Slashback two weeks ago

[1010011010]

Slashdot: Reuse, Reduce, Regret. Er, Recycle.

- - - - -

Re:Slashback two weeks ago

[the_other_one]

Re-news for nerds. Stuff that matters and matters and matters...

Revenge of Slashback

[1010011010]

I thought that article looked familiar. And yep, posted on Slashdot not too long ago.

http://slashdot.org/article.pl?sid=01/05/27/172422 6

- - - - -

It's Time for Dr. Minsky to Retire

[Louis Savain]

It is clear that AI hasn't delivered on the promises made over thirty years ago. What happened? In a preview of his upcoming book, The Emotion Machine, Marvin Minsky examines the failures of AI research and lays out directions for future development in the field.

I used to be a Minsky fan (I still have a copy of his "Society of Mind") but not anymore. Marvin Minsky is one of the reasons that AI still has not delivered on its promises. He is part of the old symbolic school of AI. He was the guy who, with Seymour Papert, wrote a scathing criticism of the then embryonic field of neural networks, effectively strangling research in neural networks for the better part of a decade. I am sure Dr. Minsky has had occasions to change his views since but I don't think he has anything to offer that will lead us to HAL. The following is a quote from a Scientific American article on Arthur C. Clarke's HAL.

The novel of 2001 explains how the HAL 9000 series developed out of work by Marvin Minsky of the Massachusetts Institute of Technology and another researcher in the 1980s that showed how "neural networks could be generated automatically--self-replicated--in accordance with an arbitrary learning program. Artificial brains could be grown by a process strikingly analogous to the development of the human brain." Ironically, Minsky, one of the pioneers of neural networks who was also an adviser to the filmmakers (and who almost got killed by a falling wrench on the set), says today that this approach should be relegated to a minor role in modeling intelligence, while criticizing the amount of research devoted to it. "There's only been a tiny bit of work on commonsense reasoning, and I could almost characterize the rest as various sorts of get-rich-quick schemes, like genetic algorithms [and neural networks] where you're hoping you won't have to
figure anything out," Minsky says.

The part that literally floored me is "where you're hoping you won't have to figure anything out,". All along I'm thinking that intelligence is so complex and intractable that the most plausible solution to the problem of making a human-level AI is one where we let the AI emerge, grow and learn. IOW, what we really need to understand is the learning process, which encompasses perceptual, motivational and motor learning.

But here comes Marvin Minsky, a luminary in the AI community, insisting that figuring everything out is precisely what needs to be done. Haysoos Martinez! This is the main reason why we still don't have human-level AI! I think Minsky's stance is a disservice to computational neuroscience and ANN researchers everywhere.

The man has had his day in the sun. Now it's time for the younger generation of AI researchers to come in and say "hold it! we're taking a different approach from now on. The unkept promises of AI were made by the old symbolic AI crowd. There is a new school in town. The new AI neural, it's emergent, and it's gonna to kick ass!"

Re:It's Time for Dr. Minsky to Retire

[rgmoore]

The part that literally floored me is "where you're hoping you won't have to figure anything out,". All along I'm thinking that intelligence is so complex and intractable that the most plausible solution to the problem of making a human-level AI is one where we let the AI emerge, grow and learn. IOW, what we really need to understand is the learning process, which encompasses perceptual, motivational and motor learning.

Well, to some extent that depends on whether your goal is to make human level AI or to use AI to help your understanding of NI. A lot of AI researchers are really more interested in the second than they are in the first, so it's understandable that they're not too happy with neural nets. You don't get very far in understanding the mind if you simply copy it in silicon instead of carbon. It would be much more interesting and instructive to make an AI that didn't use neural nets, because doing so would necessarily imply that it was possible to abstract some deeper core of how intelligence works. Of course it's entirely possible that doing so is a false dream, and that what we think of as intelligence is deeply dependent on the structure of the medium in which it is encoded. But we can't find that out without trying to build minds in some way that is fundamentally different from neural nets.

Re:It's Time for Dr. Minsky to Retire

[sv0f]

Minsky was just arguing that a perceptron could not compute an XOR function, and the reason he was wrong is simply because he didn't consider that you might connnect one perceptron to the output of another.

For Minsky to not even consider connected perceptrons was a humungous brain fart for which he should rightly be ridiculed, particularly given the influential position the way he was in at the time, and the effect it had stifling all ANN funding and research for a long time.

I think you have your facts wrong. Minsky and Papert proved that single perceptrons could only represent linearly separable mappings, and thus couldn't represent XOR for example. They also proved (or perhaps it was already known at the time since it's kind of obvious) that networks of units with linear activation functions collapse to single perceptrons, and thus suffered the same representational limitation.

They went on to express skepticism that an algorithm for training networks of units with nonlinear units to represent linearly inseparable mappings (such as the XOR function) would be discovered. On this conjecture, they were wrong.

That is, training algorithms such as backpropogation allow networks with at least one layer of sufficiently many hidden units, where each unit has a nonlinear activation function, to usually represent arbitrary mappings. I say 'usually' because gradient descent techniques like back propogation are not guaranteed immunity from local minima, although in practice this does not appear to be a problem.

There are other training algorithms for neural networks for which I am less knowledgable -- they may not suffer even in principle from local minima. There are also proofs (like Judd's) about the intractability of optimally 'loading' nontrivial neural networks. In other words, I don't know the current state of the art.

But you are clearly wrong in what you think Minsky proved and what he conjectured. That his conjecture, built on his own research, has proven largely incorrect is just a testament to the unpredictability of science. Connectionists need to stop with their conspiracy theory malarky. Minsky's no bogeyman either.

Re:It's Time for Dr. Minsky to Retire

[sv0f]

Marvin Minsky is one of the reasons that AI still has not delivered on its promises.

Are you trolling or just delusional? AI has not delivered in its promises in part because it's a damned hard problem. Philosophers, psychologists, linguists, and others have pursued it for 100s of years, and for most of its 50 year existence in computer science departments, it has been pursued on pathetically underpowered machines. The field's in its embryonic stage. Hell, ANNs have come and gone twice already (i.e., with McCulloch/Pitts/Hebb and then Rosenblatt); will you sour on ANN-AI if the third time is not a charm? I hope not, oh faithless one.

He is part of the old symbolic school of AI.

Is the 'old' or the 'symbolic' part your rhetorical way of discrediting him? First of all, as I pointed out in an earlier comment, it is plain wrong to put Minsky in the same bin as McCarthy, Chomsky, and other 'neats' who believe that logic is the royal road to AI. Minsky is a scruffy, an "anything goes" type. Second, do you know what he Minsky did for his dissertation circa 1950 or so? Built the first physical ANN out of primitive hardware in the basement of the math building at Princeton. That's right, he's an old skool ANN researcher. He also came by his skepticism the old fashioned way -- he eeeaaarrrned it. He doesn't simply oppose ANNs for philosophical reasons, brain boy.

He was the guy who, with Seymour Papert, wrote a scathing criticism of the then embryonic field of neural networks, effectively strangling research in neural networks for the better part of a decade.

He cowrote a book that formally disproved claims that irresponsible ANN researchers of the time were making about the representational capacities of their nets and inductive powers of their learning algorithms. He did express skepticism that these could be overcome. He was proven wrong on this count 15 years later (not counting Werbos's work). Again, you make it seem like he was an irrational crusader against ANNs. He was not. From experience and mathematical analysis, he made a conjecture. He was wrong. Such is the scientific life.

Frankly, I've never understood why ANN types cry and complain about the lack of ANN funding in the 1970s. Given analyses like those of Minsky and Papert, and the successes of the symbolic approaches of the time (ever heard of HEARSAY and how Rumelhart and Hofstadter claim it as a primary inspiration?), it appeared that ANN research would not be as fruitful as symbolic approaches. Hindsight is 20-20 on this one of course -- ANNs are better than was thought then. But hindsight always works this way. Why exactly did ANN researchers give up? "Oh boo hoo, not as much money for my ANN work -- must follow other fads." Their lack of resiliency was cowardly. (I feel the same way about symbolic folks who lie shamed in the shadows right now.)

IOW, what we really need to understand is the learning process, which encompasses perceptual, motivational and motor learning.

This is the main reason why we still don't have human-level AI! I think Minsky's stance is a disservice to computational neuroscience and ANN researchers everywhere.

We need to understand learning and development, perceptual and motor activity (not just learning, but performance too), how all of this is coupled with cognition (unless you think, for example, Cantor's Continuum Hypothesis is best represented perceptual-motor-ly), etc. We also need to understand how all of this is embodied in the body more generally, the ecological environment, and the culture at large. ANNs show little if any promise at tackling this last set of questions. (Journal article we'll never see: "An ANN analysis of female circumcision"). So get off your high horse about how ANN folks are gonna crack the whole cognitive nut. Frankly, it's not at all clear that ANNs are the right formalism for investigating even learning and development. A number of developmental neurosciences favor other approaches (e.g., dynamical systems).

The man has had his day in the sun. Now it's time for the younger generation of AI researchers to come in and say "hold it! we're taking a different approach from now on. The unkept promises of AI were made by the old symbolic AI crowd. There is a new school in town. The new AI neural, it's emergent, and it's gonna to kick ass!"

Oh, you're just the AI equivalent of a script kiddie. Why did I bother?

I don't know how useful this really was...

[hypermanng]

...for game programming, but I'm always happy to have people read Minsky because he tends to crack peoples' preconceptions about what is "obvious" about consciousness and AI and etc. Even better might be Daniel Dennet, author of "Consciousness Explained". Less philosophically sound (and ultimately less satisfying) but still very interesting is Steven Pinker.

I think it's very important to understand that there's no magic to consciousness. It's not something shrouded in mystery about which we know nothing. In fact, we know an amazing amount about individual areas. The topology of the information processing membranes are more complex than we can sort out just yet, but there's nothing about the structure of the brain that's not duplicable by silicon hardware. We just have a lot more mapping to do.

Also important to notice is that to implement the human mind in hardware (as opposed to wetware), we'd need something on the order of a 10 teraflop supercomputer. We just don't have the hardware to pull that off yet. The AI-related optimism of yesteryear was fueled by the misconception that computers are faster than humans. What's really true is that the "programming" that underlies the various gadgets in the mind is the product of millions of years of specialization at small tasks. We have fantastic motor-control gadgets and unparalleled pattern-recognition wetware, for example. Figuring out exactly how many animals are in 15,342 groups of 967 animals each was never all that important, so we never evolved any gadgets to carry out high-speed arithmetic. On the other hand, we're good at seeing how things divide out and how games might be played to our advantage. Idiot savants have been known to find extremely large prime numbers as if by magic - probably the same hardware put to an exotic use.

So in 20 years (or so), we'll have the hardware, and maybe we'll have the information processing topology as well. Some intrepid researcher will put all that in a state-of-the-art cybernetic body. Then it'll be a matter of watching the first hardware human child grow up and meet the world.

PS- make some pretty bold claims here, and also cite a number or two that one might be expected to view with suspicion. I can back it up, just ask.

Nato

The truth about neural nets

[hypermanng]

Neural nets, on their own, are not very smart in many ways. Sure, there are some problems it seems that only they can solve. But complex, multi-stage problems generally baffle them nearly indefinitely.

The brain is not a big neural amalgam that gets to some critical mass and then suddenly starts doing stuff. It's wired. It's got gadgets. It's really a big collection of them. Some of them are damned complex, composed of sheets of neurons talking to each other in intricate, bewildering arrays.

And modern Connectionists understand that. Certainly the symbolic logic guys were wrong as wrong as those who thought neural nets would solve everything. But that's people like Fodor and Chomsky. The Minksy "Agent" model is very much on the "Connectionist" side of the map. That's not to say that I agree with everything he says, but I think you're unfairly blaming him for the mistakes of others.

Symbolic logic, by itself is no panacea, but neither is the neural net. I'm willing to bet that a lot of the interactions of various neural nets in the brain form very formal symbolic logic gadgets. Also, in the end, it is the formal logic of virtual-neuron microcode in a computer is what will generate Strong AI.

Don't look now, but HAL is crawling up your leg.

[xmark]

A fail-proof method for creating intelligence has already been developed...by Nature. Intelligence is now thought to be an emergent property - it arises naturally in certain kinds of self-organizing systems (like life) in which the ability to acquire and process information increases survival, and natural selection sorts out the best ways of doing it. That you are reading this and understanding it is proof that this mechanism can deliver the goods.

About ten years ago, Rodney Brooks (also of MIT) flipped AI on its head with his "insect bots," which took a bottom-up (instead of Minskyesque top-down) approach. Brooks put a cheap microprocessor and servo motor on each of six "legs" of a lowly bot, and programmed each leg unit to do extremely simple things like check whether the leg was bumping against something, and if so, to lift it. Repertoires of behavior learned from the environment were then stored and re-used when similar stimuli presented themselves again. What happened after a short time was that far more complex behaviors than were programmed "emerged" from the collection of puny processors and actuators. With just a few lines of code, the damned things could navigate complex environments (like a back yard) that completely foiled Minsky-style bots run by minicomputers and millions of lines of instructions. (Brooks coined the phrase "fast, cheap, and out of control" to describe not only his bots, but the behaviors they "invented" by walking around.)

George Dyson (Freeman's son) wrote a book a couple of years ago called Darwin among the Machines that is as good an explanation of machine-evolved intelligence as I've seen. It's packed with illustrative stories from both within and without the discipline. Look here for Dyson's own commentary and some good links. Hans Moravec, director of Carnegie-Mellon's Field Robotics Lab, also writes very convincingly, if speculatively, about the evolution of machine intelligence, in his recent book Robot: Mere Machine to Transcendent Mind . It's a fascinating read.

After what's been learned in the past decade about how machines can become intelligent, Minsky seems to me a bit like Lord Kelvin. Kelvin made tremendous contributions to science, especially in the fields of heat theory and thermodynamics, but in his later years, became mired in defending some pet theories that were way past their prime. He railed bitterly against Darwin, claimed the Earth was only a few million years old, and refused to accept radioactivity. One of his biographers observed that for the first half of his career, he could no wrong, and for the second half, he seemingly could do no right. Minsky, alas, has in some ways shared this fate.

Hi, I'm Not Aware

[LionKimbro]

Hi, I'm Minsk. I'm not aware.

I'm a collection of experiences, memory, and light processing systems, but I don't have this weird pseudo-mystical thing that some morons compute about.

Once, I met a task that said it was aware. I said, "Of course you are processing light patterns." The task replied to me, "Right, I'm processing light patterns, but it's different, I'm actually experiencing it. I said, "Of course you are, my scan of your brain AI is occuring."

My contentment rating increased, because I had helped purify the system. But this beliggerent process would not stop. "No! No! You don't get it!", it said. "The processing is occuring, but there's something else; I'm seeing it- this patterns appear before me." He rambled on for some time, and then got to his crux: "The difference between this thing- which I'll call awareness- and the processing that is going on- is that the processing does not require it, and yet it is still there."

I found his nonsense absurd and disagreeable. I reported to central computing this processes insanity, but only after attempting a little more reasoning, to salvage the rogue process: "Surely you recognize that your 'awareness'- is merely a dangler and a phantom belief. Have you cleared yourself through the Computer Science program? Perhaps a little time within an electric fence will assist? Surely you know that you have some residual data from prior superstitious existance within the random garbage data before your allocation. Your computational appendix, this strange persistance within you, is completely illusory and inconsequential."

But I was not allowed to finish my sentence, for after uttering the word "appendix", the bugged process shouted profanities and said such incoherent nonsense as, "I AM THAT APPENDIX!". The process was clearly delerious, and thus I had him scheduled for termination with the Scheduler.

After all, You Can't Argue With a Zombie.

Coincidence?

[Dancin_Santa]

I was just looking up Mirsky on the web the other day. It was cute, I found the Drunk Browsing Test. It's not the Worst of the Web or anything, but fairly amusing.

Dancin Santa