The Baby Bootstrap?

An anonymous reader asks: "Slashdot recently covered a story that DARPA would significantly cut CS research. When I was completing graduate work in AI, the 'baby bootstrap' was considered the holy grail of military applications. Simply put, the 'baby bootstrap' would empower a computing device to learn like a child with a very good memory. DARPA poured a small fortune into the research. No sensors, servos or video input - it only needed terminal I/O to be effective. Today the internet could provide a developmental database far beyond any testbed that we imagined, yet there has been no significant progress in over 30 years. MindPixels and Cycorp seem typical of poorly funded efforts headed in the wrong direction, and all we hear from DARPA is autonomous robots. NIST seems more interested in industrial applications. Even Google is remarkably void of anything about the 'baby bootstrap'. What went wrong? Has the military really given up on this concept, or has their research moved to other, more classified levels?"

Comments

The Terminator

[hshana]

Maybe they were afraid of Skynet.

Re:The Terminator

[randomErr]

No, they're afraid the computer may ask 'Want to play a game?'

Oh great...

[kwoo]

Just one problem with this kind of research...

For the first year I'll be up every two hours all night, tending to the system.

Actually, that may be better than just being up all night, like I am now.

Re:Oh great...

[un4given]

Actually, that may be better than just being up all night, like I am now.

I guess I'm not the only MCSE on this site after all...

Classified

[pete-classic]

It has moved to more classified levels.

I'd go into more detail, but the C.I.A. and C.I.D are at my door. Ooh, the B.A.T.F. just pulled up in a Mother's Cookies truck!

-Peter

Re:Classified

[sgant]

But they all ran when they saw the RIAA and MPAA moving in...even the IRS is afraid of them.

Tremble as they pass...stare in awe at their mighty power

Re:Classified

[sys$manager]

And you wondered why there was a

Flowers
By
Irene

truck parked on your street all week?

Re:Classified

[PhosterPharms]

Not to be an insensitive clod, but the BATF exists no longer. I work for a winery, so I am certain of this much. The TTB is the new regulatory agency which governs over our side of thinge, and I believe that the Department of Homeland Security deals with the Firearms now.

Regards,

-PhosterPharms

Re:Classified

[Infinite Entropy]

Somwhere, in a deep sub-basement of some unmarked NSA building is a fully sentient AI that spends all its time reading Slashdot. Its creators wonder why it keeps wanting to meet Natalie Portman and asking what hot grits are.

baby bootstrap

[kris_lang]

Sure, that was the engine of thought behind stories such as WarGames and 9x109 names of god. Somehow, unfettered access to data and time with "neural networking" capacity to form links and create linkages to pieces of data ("associative memory") would be all that was needed to create intelligence, and perhaps even sentience.

Minsky came up wrong on the single layer perceptron, AI was wrong on the purely feed-forward neural-network systems, Rumelhart and McLelland got some good promo off of their feed forward net that could learn to pronounce idiosyncracies, and Sejnowski got a great job at the salk from the AI delusions. But no, it appears to not have gone anywhere... thus far.

Later comment will be positive. ...

Re:baby bootstrap

[Al Mutasim]

It seems we can program anything done with conscious thought--algebra, logic, and so forth. It's mostly the things we do unconsciously--recognize objects, interpret terrain, extract meaning from sentences--that can't be put adequately into code. Would the code for these unconscious processes really be complicated, or is it just that we don't have mental access to the techniques?

Re:baby bootstrap

[cynic10508]

Dreyfus commits a whole book to asking why these things don't work. I believe Minsky overestimates the project. It may all boil down to the fact that purely syntactic (symbol manipulation) work isn't going to give you any semantically meaningful output.

Re:baby bootstrap

[man_ls]

I doubt it would be too difficult to code -- if we knew the mechanism by which it proceded.

Its hard to code a procedure to replicate the working of the mind...if you don't know how the mind does it in the first place.

Re:Baby Bootstrap

[hobbesx]

I can assure you.. I am very classified.

Dear Baby Bootstrap computer,

You forgot to check the AC box. Congratulations on becoming Un-Classified!

Re:baby bootstrap

[kris_lang]

Ah, those are exactly the things I was commenting about above...

That's what the "neural network" paradigm was all about. You have an arbitrary and fixed number of input node, you have an arbitrary and fixed number of output nodes. You create linkages between these nodes and "weight" them with some multiplicative factor. In some particular instantiations, you limit all inputs to be [-1... +1] and limit all weights to be within the range [-1 ... +1].

So with A input nodes and B output nodes, you've got a network of AxB interconnections between these input and output layers. The brain analogy is that the A layer is the input layer or receptor layer, the Blayer is the output or motor layer, and it is the interconnections between these neurons, the neural network composed of the axons and dendrites connecting these virtual neurons that does the thinking.

Example: create network as above. Place completely random numbers meeting the criteria of the model (e.g. within the range -1 weight B's output feeds forward to C, etc., and these are called intermediate layers.

Rumelhart and Mcllelland encoded spellings as triplets of letters (26x26x26), had a few (or one, I can't remember this now) intermediate layers, and an output layer corresponding to phonemes to be said. They effectively encoded the temporal aspect of the processing into the triplets, sidestepping a (what I consider the more intersting...) part of the problem. They trained this neural network by feeding it the spelling of words and adjusting the weights of the networks until the outputs were the desired ones.

Note that nowhere in this process do they explicitly tell the system that certain spelling combinations lead to specific pronunciations. They only "trained" the system by telling it if it's right or wrong. The systems weights incorporated this knowledge in these "Hebbian" synapses and neurons.

So this is associative processing, using only feed-forward mechanisms. Feedback, loops, and temporal processing are even more interesting...

alas not enough room in this margin to keep going.

Re:baby bootstrap

[cynic10508]

Ah, philosophy of math. How fickle and unforgiving it is.

True, you can apply meaning to a syntactic structure. But like the mistake Douglas Hofstadter makes in Godel, Escher, Bach: An Eternal Golden Braid, there is nothing that "forces itself upon us." Or, another way of refuting Hofstadter, there's nothing about D:=B|| that makes it "Doug has two brothers" anymore than "Assign B to D, double pipe".

Machine translation is an example of applying semantics to a syntactic structure. It doesn't work because the syntax gives us semantics but rather we structure the syntax in such a way that we can systimatically apply semantics and get meaningful output. Like creating your own algebra.

Re:baby bootstrap

[rgmoore]

I think that a key issue is that not everything in our brains is handled the same way, so not all of it is equally easy to program. Conscious thought is essentially a software process running on the part of our brain that serves as a general-purpose computer. Our unconcious processes are essentially hardware processes running in parts of our brains that are specifically structured to do just that one thing. The fact that unconcious processes are run in hardware means that they're not subject to introspection. I suspect also that many of those processes are the kinds of things that are most efficiently done with custom hardware like DSPs rather than with general purpose CPUs.

Re:baby bootstrap

[thelen]

The system might generate syntactically correct outcomes, but have we really solved the problem if we the observers are still the ones to apply semantic content? Isn't the point of Searle's Chinese Room thought experiment to show that syntactic transformations are not sufficient to imbue the transformer with a semantic understanding of its activity?

Re:baby bootstrap

[nacturation]

Note that nowhere in this process do they explicitly tell the system that certain spelling combinations lead to specific pronunciations. They only "trained" the system by telling it if it's right or wrong.

Right, it's kind of like an implementation of bayesian spam filtering, but for other problem domains. Instead of spam/ham, it's pronounced-correctly/incorrectly. Rinse and repeat.

I dabble in AI now and again so I haven't read up on everything that's out there, but in my limited travels what I haven't yet seen is a neural network implementation which can learn and grow itself. The recently posted /. article about Numenta seems to be heading in the right direction. Most neural networks are incredibly rudimentary, offering a few levels of propogation. In a real brain, there's a hell of a lot more going on.

I did some calculations a while back, and based upon 100 billion neurons in the brain, each capable of firing let's say an average of 1000 times per second, and we'll assume that at any given time a generous 1% of all neurons are actively firing, and that the information firing takes 100 clock cycles to process, then you'd need the equivalent of about a 100 TeraHz processor with oodles of memory to have the same processing power as the human brain. Of course, you'd also need to correctly simulate *how* the brain is wired up to get any kind of beneficial processing.

So as far as the whole 1980's AI winter, it was inevitable. The computing power and storage requirements for any sufficiently advanced AI just wasn't possible. It's only until very recently that it's possible to achieve fairly complex AI.

Re:baby bootstrap

[sgt101]

It's a little while since I looked at Minsky's book, but as I remember it his point was that perceptron learning, which could only be applied one layer at a time could only separate linearly divisable functions. He showed that a multi layer perceptron with particular settings could separate an XOR problem (for example), but at the time there was no algorithm to learn these settings.

Later Rumelhart and Hinton invented back propagation that could over come this issue by learning the kind of classifiers that Minsky was describing - ie. non linearally separable spaces.

A recent revolution has come in the form of the realisation that structural risk minimisation can also be done automatically, as well as statistical risk minimisation in classifier learning. That is that we can not only minimise the error rate of the classifier (statistical risk) but also minimise the risk that we are over fitting to the data set and not the domain theory (structural risk). Algorithms that do this are things like instance based learners, support vector machines and various ensemble learners like boosters and roc learners.

I don't know of much more progress in supervised learning after this point - it's mostly held to be solved I think now. The challenges are more in things like inference based learning, community based learning and unsupervised learning of various types.

And of course the dirty word - applications.

Re:baby bootstrap

[Mr. Slippery]

John Searle advocates a position that symbol manipulation isn't intelligence. Rather that consciousness is an emergent property of patterns in neural firing.

You're equating "intelligence" with "consciousness"...that's problematic, as I can infer your intelligence by your words and actions but I cannot infer your consciousness by anything other than analogy to my own. Intelligence is functional, consciousness is purely experiential.

Searle's view falls down on two counts: it requires us to say that a symbol manipulation system that seems to be presenting intelligent output (like the Chinese room example) isn't intelligent; and it requires some unspecified "magic" property of brains.

I recommend the discussion in Hofstadter and Dennett's The Mind's I .

There is no intentionality for 1 + 1 = 2. But there is for the statement "I believe in the validity of axiomatic mathematicaly systems."

I know that there is intentionality when I make statements about belief. But there's no way that I can know that you're not just a clever but "dark inside" symbol manipulator when similar statements come out of your mouth.

Yes, that way lies solipsism, but we need not go all the way down that road, just enought to establish what we really can and cannot know about other minds.

Re:baby bootstrap

[djfray]

I'd be very interested in seeing information confirming anything close to your generous 1% firing at a time, and how this is integrated with the rest of the system for signal processing, who fires when, etcetera. I think, however, that we need to take into account the fact that more neurons doesn't mean smarter at all. Take a look at whales, for instance, with brains much larger than our own, and thusly, more neurons. A whale can't go on Slashdot and say "OMGZ first post guys" much less something of human level intelligence. (Apologies to creationists for the following...)It took billions of years of evolution, all the way back to the primordial ooze(or whatever) to get to the point of having a species with the genetic mappings to produce the neural networks that allow us to learn, remember, think, and process as we humans do. I think this would add a significant number of zeroes to your processor calculation, even when we incorporate a design based on our own incredible thinking.

Re:baby bootstrap

[Servants]

I doubt it would be too difficult to code -- if we knew the mechanism by which it proceded.

Its hard to code a procedure to replicate the working of the mind...if you don't know how the mind does it in the first place.

On the other hand, it might be that the reason we don't understand how the mind does certain things is that they're actually extremely complicated, and don't reduce very well to a programmable step-by-step algorithm nor to a simple and general mathematical learning structure. It's hard to tell, although I think it's telling that after decades of work, neither psychologists nor computer scientists can understand or replicate much of what babies do.

Sometimes the best way for a computer to learn something may not be the way a baby does it, anyway; c.f. chess.

Re:baby bootstrap

[Servants]

Right, it's kind of like an implementation of bayesian spam filtering, but for other problem domains.

By Bayesian spam filtering, I think you mean general classification problems, in which case, yes, neural networks can implement classification - it's a stretch to say that McClelland and Rumelhart's did, because the possible output included most non-repeating combinations of English phonemes and is thus nearly infinite, but the principle is there.

Of course, you'd also need to correctly simulate *how* the brain is wired up to get any kind of beneficial processing.

I think you're overestimating the importance of processing speed and underestimating the importance of the above.

For starters, a whole lot of parsing of the input has to go on -- retinal images parsed into people and objects, sound streams broken up by source, language identified, relevant phonological boundaries determined, speech separated into words. Then you have to know what your goal is, and approach it at the right time: learning syntax or social conventions won't help a baby who doesn't know words or faces yet. And what's the output of "learning syntax", anyway? A list of rules? A network that can turn... something... into a sentence?

Throwing more nodes into a network doesn't get anywhere with these problems, whether the network "grows itself" or whether the programmer does the work. The big problem is structuring inputs and outputs to be complete, sensible, and not wrongly redundant, and perhaps arranging networks in sequences or graphs to separate information and model psychological findings of dissociations between tasks.

Also, there's a great deal of parallel processing in play. Your 100 THz processor can perform a zillion operations per second in order, which gives it far more flexibility than the brain has. Between neuron firing rate and communication time, I think (can't currently find the reference) the brain is limited to about 100 sequential operations per second.

That's astoundingly few. You can come up with a good chunk of a sentence in a second, and recognize a blurry familiar face in less. Parallel or not, I have difficulty imagining how one does that in so few chunks of time.

Re:baby bootstrap

[polv0]

It is fairly easy to show (see Bishop 1995) that a simple two layer neural network can scale to reproduce arbitrarily complex but smooth functions to any degree of required accuracy, and that a three layer neural network can extend this capapility to functions with discontinuities. While mathematically this is a tantalizing prospect, and only begins to cover the work that has been done to extend the capabilities of neural networks and other machine learning algorithms (such as support vector machines), there remains a fundamental problem. In order for these networks to effectively learn, they must be presented with a tremendous number of high quality and meaningful sequences of input and output.

For example, in text recognition, hundreds of thousands of hand written characters are painstakingly hand labeled with their correct letters and used as a learning database on which the algorithm is trained. The algorithm will then accurately reproduce the correct categorization for a suprisingly high number of the training examples, and any new examples drawn from the same population. But given new examples written in a different script or style, the classifier will fail to generalize

How can we hope to create a training database that is comprehensive enough to cover a topic that, when learned, would demonstrate intelligence? And fundamentally, aren't we just creating a really good mimic?

Re:baby bootstrap

[bluephone]

"Sometimes the best way for a computer to learn something may not be the way a baby does it, anyway; c.f. chess."

Except computers never learned chess; humans programmed complex move analysis routines along with the rules, and many times a database of strategies with statistical weighting. There's a limited capacity to "learn: against opponents, but that's usually just more preprogrammed analysis and pattern matching than actualy spontaneous data linking. And like a poster higher up said, ther ewas a time we thought that was all one needed. It's not. We already have rudimentary AIs in labs that can "learn" in the sense they can create accurate spontaneous data links. The human brain (or the brain of any semi complex organism, really) is a black box with such unimaginable gears inside we're fumbling in the dark. It's hard to reverse engineer a mind becuase unlike reverse engineering a BIOS or widget, we don't really understand how a mind works, is put together, or even what it's really comprised of.

Re:baby bootstrap

[pluggo]

Take a look at whales, for instance, with brains much larger than our own, and thusly, more neurons. A whale can't go on Slashdot and say "OMGZ first post guys" much less something of human level intelligence.

This doesn't necessarily mean lower intelligence, in my opinion. Being underwater prevents most technology (that we know of) from working, from fire and wheels to computers and airplanes.

A whale doesn't have fingers or hands, either, but whales and dolphins could well be as intelligent as (or more so than) us, but simply be less technologically advanced and unable/unwilling to communicate with us in a way we understand.

Sure, they seem dumb at Sea World- but then, if you took a human baby and put it in a cage and threw bananas at it when it did a trick for you, it would probably behave pretty stupidly. Much of our intellect is awakened by our experiences in the early 5 or so years- within limits, the more you are stimulated within this time, the smarter you will end up being. I would simply wonder what a dolphin or whale could be taught to do if stimulated properly.

An interesting and slightly off-topic side note is that whales and dolphins are conscious breathers; i.e., they must consciously surface in order to breathe, so they never go completely to sleep. Instead, half of their brain sleeps at a time- during this time, they're in a groggy half-sleeping state that allows enough consciousness to surface and to wake up if there's danger.

Intelligent and friendly on rye bread with some mayonnaise.

Re:baby bootstrap

[dublin]

So as far as the whole 1980's AI winter, it was inevitable. The computing power and storage requirements for any sufficiently advanced AI just wasn't possible. It's only until very recently that it's possible to achieve fairly complex AI.

Funny, that's the same thing they said back in the 80's. And the 70's. And the 90's.

Sorry, but I don't buy it. Neural nets are not a panacea - I'm a robotics guy by training, and they've been the supposed magic pixie dust technology that was going to give us human-like robot motion in the 1980's. Funny, but the hard problems that need real AI, like voice recognition, handwriting recognition, unled learning, etc. are just as far off today as they were 20-30 years ago.

Faster computers have definitely not been terribly beneficial. As an example, modern speech and voice recognition systems are significantly but not dramatically "better" than they were 20 years ago (perhaps a 10-20x improvement, max) in spite of the fact that computers are roughly a million times faster: ~6 MHz vs. ~4 GHz for high-end desktop PCs. (Not to mention available RAM that's larger than the disk storage in entire mainframe data centers back then...)

Procedural AI has proven itself to be a miserable failure for nearly a half century now, and neural nets have shown that they are anything but self-organizing. Like so many other efforts to copy or explain life, it appears that having the raw materials is simply not enough - life is *different* - it's really, really hard to imitate even poorly, no matter how hard we apply our own intelligent design to the problem.

I sincerely doubt that I will live to see "baby bootstrap" systems, and I'm not all that old. I suspect that only true hardware neural nets hold any hope of mimicking life to any minimally useful degree, but the problems are very, very, hard here, and ther reality is that we know next to nothing with any certainty about how even the simplest brains really function...

Re:baby bootstrap

[coaxial]

I dabble in AI now and again so I haven't read up on everything that's out there, but in my limited travels what I haven't yet seen is a neural network implementation which can learn and grow itself. The recently posted /. article about Numenta seems to be heading in the right direction. Most neural networks are incredibly rudimentary, offering a few levels of propogation. In a real brain, there's a hell of a lot more going on.

I don't know what you mean "grow", since all implentations use a static number of neurons that are connected to each other via a series of preexisting links. In a very real sense there's no real difference between a neuron that's connected another neuron via link of weight 0 and one that isn't connected at all. Fully connect the neurons, and walla. And you a completely abstract network. Of course, now you have cycles, so the propagation algorithms get complicated real fast. Also you can't just throw neurons into a network and expect it to work. Every neuron and every link between neurons adds another degree of freedom to the network, and so stability can become harder to reach.

NNs are kind of neat, but they're far from the end-all-be-all. A single neuron can only divide the search space via a hyperplane. Determining how many neurons in how many hidden layers is a bit of dark art. And to dash the last bit mistique about NNs, backpropagation is nothing more than hill-climbing.

Re:baby bootstrap

[nacturation]

The problem, however, is that no matter how efficient your filters are, they will lack the motivation of a learning, growing human being to learn. They will not notice things as a human would; they won't notice things at all. They'll simply take input and use a pre-determined algorythm to produce output.

Then you need to ask, what is motivation? Unless you believe that people have a soul which AI can never possess, then why can't a sufficiently intelligent AI achieve everything a human can? Put another way, if we were able to take DNA and perfectly simulate its growth just as a fetus would, so that we have a machine duplicate of a human brain, is there any reason to believe (again, given a perfect simulation) that our software AI would operate any differently than our own wetware?

Re:baby bootstrap

[misterpies]

>> By Bayesian spam filtering, I think you mean general classification problems, in which case, yes, neural networks can implement classification - it's a stretch to say that McClelland and Rumelhart's did, because the possible output included most non-repeating combinations of English phonemes and is thus nearly infinite, but the principle is there.

IIRC, mathematically it's been shown that neural nets and bayesian learning systems (such as spam filters) are entirely equivalent. Check out some of the work by David MacKay at the University of Cambridge.

Re:baby bootstrap

[HuguesT]

The human brain is hardwired for complex languages. We're not sure about cetaceans. They definitely communicate, but we don't know at what level of complexity.

We know this because people who have had their speech centre knocked out by a stroke don't recover any form of speech. Other bits of the brain don't take over to compensate.

Now language is pretty important to overall intelligence. Without it no I/O processing, and it's pretty hard to learn.

Re:baby bootstrap

[HuguesT]

The technologies you talk about are not as far off as they were earlier.

Today OCR of printed text is a solved problem. It comes bundled with your $100 scanner, and it's damn useful.

By solved I mean that if you gave a few pages to type up to a person they would make more errors than OCR software make now.

Handwritten OCR will come, it is harder, but not impossibly harder.

Speech recognition is progressing. It comes bundled with MacOS/X, and you've certainly heard of spoken text entry in word processors. It's not accurate enough to replace typing in able-bodied people, but for many diabled people it's a godsend.

We've come to a much greater understanding of what neural nets do, and to duplicate or better their success rate with non-black-box classification systems such as regression trees, support vector machines, kNN and the like.

Progress are slow but constant, the most obvious (and upsetting) progress I know of is in the security area. It has become feasible to trust automated security systems to some extent. You are certainly aware that you can be booked for speeding or running a red light via an automated system (which will read your number plate and send you a fine without much supervision).

Myself I'm not a believer in strong AI, but I witness (in my work as a reseacher in automated image analysis) constant and relentless progress in useable weak AI.

Re:baby bootstrap

[ajs]

"It's hard to reverse engineer a mind becuase unlike reverse engineering a BIOS or widget, we don't really understand how a mind works,"

I would argue (and I could be proven wrong) that today we have a very general understanding of how a mind works... in that we understand the concept of a neural network which does seem to be a decent model for the basic "mind" which makes choices for us... the problem comes about where we attempt to understand HUMAN BEHAVIOR which is the combination of a mind (neural network) and dozens of auxiliary, special-purpose systems ranging from the neurons in the optic nerve that perform a plethora of pre-processing on the retina's image data to the area in the brain that we're just discovering models our "empathy"; it allows us to re-process visual information about others as if we were experiencing what they are.

These special-purpose systems are sometimes inside the brain (the latter example) somtimes outside (the former), but they are not part of what we traditionally expect consciousness to be.

These tools make many of the tasks that we expect AIs to perform nearly impossible. For example, facial recognition seems like it should be easy, but once you sit down with a camera and try to make the computer "see" differences, you find that faces all look very much alike. We are tricked -- by a shockingly sophisticated facial recognition pre-filter in our brain -- into thinking that faces are widely distinct, but they are not (the old "all [race] look alike," is actually true... for all values of [race]).

So, while we might look at an AI and say, "unless it can tell faces apart, it's not 'smart'," it turns out that that's actually a pretty poor measure of pure intelligence.

Other aspects of our instinctive measures of intelligence such as language, managemetn of a human body (e.g. walking), etc. all have one or more of these auxiliary systems at their heart.

So we really have two problems: create a machine that can think; and create a machine that can behave like a human.

The former is either within our grasp, or already possible. The latter is going to have to be the product of an enourmous reverse-engineering effort which has probably only just begun.

The Internet as a Intellect...

[ackthpt]

It has too much fascination with pr0n.

Some tentative approaches towards AI being made

[Anonymous Coward]

From time to time I see individuals talking about adaptive intelligence usually involving the Internet as a basis of information, but the general consensus is still garbage in, garbage out.

These training systems are generally specialized because it's easier to get a practical result out, and I've actually seen some in use as 'knowledgebase' support webpages that will intelligently determine what you want based on what others wanted and syntactic similarities between the pages. I've never heard the term 'baby bootstrap' so maybe different terminology will obtain better results from Google?

It's obvious why the search failed

[exp(pi*sqrt(163))]

Who calls what you describe "baby boostrap"? I haven't worked in AI myself but have a keen interest in it and have friends who worked in the field including one who worked on Cyc (who says it's a scam BTW). Not once have I ever heard the expression "baby bootstrap". But what you've done is cool. Rather than search on precisely that term you've submitted your search to the serach engine known as "/. readership". It's not terribly relaible but it is good at fuzzy searches like yours.

Re:It's obvious why the search failed

[Dun Malg]

Who calls what you describe "baby boostrap"?

I've also noticed that nobody seems to make Horseless Carriages anymore (and after they showed such promise). Likewise, the Difference Engine has been a total flop. I do, however, expect we will see in the future some use made of the Vegetable Lamb of Tartary, though no use has been made of it in the last 1000 years since it was discovered.

Re:It's obvious why the search failed

[quokkapox]

you've submitted your search to the search engine known as "/. readership". It's not terribly reliable but it is good at fuzzy searches like yours.

Good point; however, each query made to the /. readership search engine is quite expensive in terms of all the employer-funded man-hours it consumes. If we all stopped wasting so much time reading/posting here, the world economy would surely take off like a bat out of hell.

Doublethink

[nappingcracker]

q: Has the military really given up on this concept, or has their research moved to other, more classified levels?

a: yes.

Re:Doublethink

[wolenczak]

Regular implementations of knowledge engines in AI use true/false semantics for automated learning, this would be the answer you could expect from such application, just like Mindpixel e.g. Sad you missed the sarcasm.

Stat algos

[Anonymous Coward]

What happened was that research focused
on machine learning models and inference
models for belief networks. The work
in this area since the 80s has been
*spectacular* and has impacted other
areas of research. (E.g., speech
recognition, image processing, computer
vision, algos to process satellite information
faster, stock analysis, etc.)

So, mourn the loss of the tag phrase "baby
bootstrap", and celebrate the *unbelievable*
advanced in belief nets, causal analysis,
join trees, probabilistic inference,
and uncertainty analysis. There are
literally dozens of classes taught at
even non-research oriented Univs (e.g.,
teaching colleges or vocational-oriented
schools) on this very subject.

(As for your concern that the web is not
being mined for ML context, just look at
semantic web research, and other belief
net analysis of text corpuses. Try
scholar.google.com instead of just
plain old google to find relevant
citations.)

The early AI research paid off BIG TIME,
albeit in a direction that nobody could
have predicted. Researchers did not keep
using the phrase "baby bootstrap" so
your googling will give you a different
(and wrong) conclusion.

Re:Stat algos

[phreakmonkey]

You're going to take this answer from someone who enters their comments on a Commodore 64?

Re:Stat algos

[chris_eineke]

slashdot haikuness
you make quite a bad mistress
compared to the moon

Re:Stat algos

[Anonymous Coward]

Yes, but where are the results? Crappy Bayesian spam filters that can be gamed just as well as any other system? Thank you, AI!

Don't be so critical! NI (Natural Intelligence) can be "gamed", too. After all, biological brains aren't all knowing: lots of animals have them, and they still do "stupid" things, because that's just how their brains are wired.

Rabbits get run over on roadways because their minds are programmed to make them dart sideways, then "freeze", so that a predator will miss them in the woods. When they're not in the woods, and not facing a "predator" of the expected kind, they tend to get squashed flat. The rabbit's NI is poorly programmed for dealing with cars.

Moose get run over by trains, because moose brain is programmed to try to run away from predators as fast as it can, along the clearest path it can.

Unfortunately, the clearest path away from a oncoming train is often straight down the train tracks... and so the moose gets run over, because it's brain is poorly programmed for dealing with trains.

An interesting slashdot article mentioned how a certain kind of pet lizard could be caught by slowly lassoing them with a strand of grass: the lizard's brain was incapable of detecting the grass strand as a threat. They'ld dart away from an outstretched hand, but they'ld get caught time and again with the grass trick.

So, yes, AI might not be so bright... but then again, neither are many complicated biological systems. Given that AI has been around for 50 years, and rabbits for thousands, we're still doing rather well: some robots are have nearly rabit-like intelligence, and we've still got a lot more research to explore...
--
AC

Re:Stat algos

[rkrabath]

the moon is a harsh mistress

excellent book

Baby Bootstrap?

[ArcCoyote]

The process that bootstraps a baby is still the Holy Grail for a lot of geeks.

Hardest problem not yet addressed

[RobotWisdom]

You can't expect any system to discover the deep structure of the human psyche on its own-- we humans bear the full responsibility of discovering it. But once we have a finite structure that can handle the most important aspects of human behavior, everything else should fall into place.

My suggestion is that we need to explore all the possible permutations of persons, places, and things, as they're reflected in the full range of literature, and classify these permutations to discover the underlying patterns.

(I've tried to make a start with my AntiMath and fractal-thicket indexing.)

Re:Hardest problem not yet addressed

[swillden]

You can't expect any system to discover the deep structure of the human psyche on its own

An interesting book that relates to this is George Lakoff's "Women, Fire and Dangerous Things". Lakoff analyzes the categories defined by linguistic structures and uses what he learns to deduce some interesting notions about human cognition. In the process, one of the things that becomes very clear is that much (all?) of the way we structure our thinking is fundamentally and inextricably tied to the form and function of our physical bodies.

One of the shallower but easier to explain examples is color: although the color spectrum is a continuous band, with no clear dividing points imposed by physics, the way in which people choose segments of that spectrum to which to assign names is remarkably consistent. Even though different cultures have different numbers of "major" colors (essentially, the set of colors that are identifiable by any member of that culture with basic verbal abilities, consider "green" vs "chartreuse"), the relationships between the major color sets is one of proper subsets. For example, one African (IIRC) culture has only two major color words, which would translate to Western color senses as roughly as "warm" and "cool". Another culture has four color words, two of which fall into the "warm" category and two of which are "cool". Western cultures have seven, and there's a direct correspondance between those color categories and the four and the two.

Further, those categories are non-arbitrary. If you show a variety of shades of red to individuals from different Western nations and ask them to pick the "most" red, they will do so with near-perfect unanimity (assuming the shades aren't too close together -- they have to be readily distinguishable). Then, if you show the same shades to someone from a two-color culture and ask for the "warmest", they'll choose what the Westerners chose as the "reddest". Ditto across the board. I'm trying to explain in two paragraphs what Lakoff spends several pages on, and probably not doing a good job, but the gist is this: Experimental evidence shows that the assignments of names to colors is definitely not arbitrary, even across very distinct cultures.

The reason? Physiology. The "reddest" red, as it turns out, is the one whose wavelength most strongly stimulates the red-activated cones in our retinas.

The point is that, at a fundamental level, everything we percieve about our world is filtered through our senses and that inevitably defines the way we understand the world. Even more, our cognitive processes are built upon associations, extrapolations -- analogies and variations -- and the very first thing we all learn about, and then use to construct metaphors for higher concepts, is our own body. The body-based metaphors for understanding the world are so deep and so pervasive that they're often difficult to recognize.

Lakoff's reasoning has some weaknesses -- mostly I think he overreaches ("overreaches" -- notice the body metaphor implicit in the word? And "weakness", too) -- but his arguments are good enough to make me think that if we ever do see an artificial intelligence of significant stature, it will think very, very differently from us.

It's really unclear what such an intelligence whose primary source of experience was unfettered access to the Internet might be. We view the net as a structure built of connected locations, but that's because we apply our own physical world-based structures to it. What would an entity whose only notion of location is as a second-order, learned idea see? And who knows what other ways its understanding would diverge?

Baby Bootstrap

[multipartmixed]

I can assure you.. I am very classified.

Poorly funded yes...

[mindpixel]

Yes, Mindpixel [singluar] is poorly funded [I know because every cent spent to date has come from my pocket]...but the directon is correct... Move everything that isn't in computers, into computers. Just look at what GAC knows about reality [visit the mindpixel site and you can see a random snapshot of some validated common sense]... the project has nearly 2 million mindpixels now...I have a copy on my ibook and I can do some profound search related things because of all the deep semantics I have that google can't touch, at least until they invest in mindpixel ...

Cognitive Machines Group @ MIT Media Lab

[YodaToo]

I did my doctoral research developing software to bootstrap language based on visual perception. Had some success, but not an easy task.

The Cognitive Machines Group @ the MIT Media Lab under Deb Roy seem to be on the right track. Steve Grand's work is interesting as well.

Re:Cognitive Machines Group @ MIT Media Lab

[starm_]

I'm also currently ("currently" as in I'm writing this while my other computer is simulating bootstrapping based learning) working in this field and sucombing to frustration. I do believe we should see significant discoveries in the next 30 years but it won't come easy.

Godamn I've been procrastinating in the last few days because I am stuck on trying to compute probabilities in a probabilistic graph efficiently. One of the big hurdles I think is from the fact that we are trying to approximate a massively parallel architecture (the brain) on a _annoyingly_ serial machine (the computer).

On the bright side my procrastination has led me to design a prototype wind turbine out of a paper tube and straws. I even made a spreadsheet to compute optimal blade angle and chord

Re:Cognitive Machines Group @ MIT Media Lab

[Bastian]

One of the big hurdles I think is from the fact that we are trying to approximate a massively parallel architecture (the brain) on a _annoyingly_ serial machine (the computer).

Maybe a Beowulf cluster of those would help?

--ducks--

Re:I for one

[Gentlewhisper]

I'm not sure if it is related, but i've once read an article about some research DARPA is doing in the field of aeronautics.. where they have whole squadrons on autonomous fighter jets controlled by only one human (who also happens to be part of the squadron).

It is some pretty neat stuff, especially if you are having trouble enlisting enough humans to fight wars for you.

Shutting down this discussion as of now.

[infonography]

By order of Wintermute (DARPA AI code 324326343.534) this discussion is terminated and no further investigation into this obviously false and misleading theory is permitted.

Would you like to play a game of chess Professor Falken?

Babies have an instinctive understanding of 'real'

[Sierran]

...and parents/pain for what is 'correct.' I don't think the concept is gone, but there are problems that are buried in the question as posed which (I think) became clearer stumbling blocks as technology advanced. NOTE: I'm not an AI theorist, nor do I play one on TV; I just like the idea and read a lot. Hence, this is all pulled out of my fundament.

Cycorp is not a poorly funded idea in the wrong direction. Cycorp chose a different tack; they decided that rather than trying to build a reality and correctness filter, they'd rely on human brains to do it for them (like trusting your parents implictly) and instead concentrated on the connectivity of the 'facts' accrued by the 'baby.' CYC is still very much around, and is very much in demand by various parts of the government and industry - if you want to play with it yourself, you can download a truncated database of assertions called OpenCYC. Folks have even gone so far as to graft it onto an AIML engine, to produce a chatbot with the knowledge of OpenCYC behind it.

The problem: how does your baby learn what's real and what's REAL NINJA POWER? Or, pardon me, what's REAL NINJA POWER and what's just a poser? Someone's gotta teach it. Which means it has to learn not only facts, but how to evaluate facts. So it has to learn facts, and how to handle facts - which means it has to learn how to learn. Which means you need to know that answer from the git-go. Tortuous games with logic aside, the onus is now much more heavily on the designer to have a functioning base - whereas with the Cyc approach, the only 'correctness' that is required is that of information, and perhaps that of associativity or weight - which can be tweaked, dynamically. The actual structure of how that information is related, acquired, stored and related is not relevant once decided. Having said all this, Cyc is (from the limited demos I've seen) quite impressive at dealing with information handed to it. It just wouldn't do very well at deciding what do do with that information - that's the job of the humans that gave it the info. It can tell you about the information, but not what to do with it. That task requires volition, really.

Volition is a killer. What is it? How do you simulate it? How do you create it? Is it random action? Random weighted action? Path dependent action? Purely nature, purely nurture? When it comes down to it, the human is (as far as we know) not a purely reactive system, which CyC (AFAIK) is. Learning requires not only accepting information, but deciding what to do with it - deciding how it will be integrated into the whole. If the entity itself isn't making that decision, then the programmer/designer/builder has already made it in the design or code - and then it's not really learning, is it?

Sorry if this is confused. As I said, I don't do this for a living.

Well maybe the realized that it's hard

[Illserve]

Bootstrapped learning something useful, even from an information ocean like the internet, is *HARD*.

Doubly so if you have no goals, and your task is just to "learn". It would come back with garbage.

Perhaps the real killer is that even if it did learn something, the information acquired in its unguided search through the internet would be completely alien. You'd then have to launch a second project to figure out what the hell your little guy learned.

And you'd probably figure it out was mostly garbage.

What Went Wrong?

[SQL Error]

there has been no significant progress in over 30 years

That's what went wrong. Basically, it don't work.

Narrow IO Insufficient

[Edward Faulkner]

If you want a machine that learns like a human, it may very well need the same kind of extremely rich interface with its environment that a human has.

Some researchers now believe that "the intelligence is in the IO". See for example the human intelligence enterprise.

Google for the correct term

[darth_MALL]

Isn't it called a Seed AI?

Nonono!

[jd]

They ARE a Commodore 64 that got "baby bootstrapped" off the Internet. This is a bid to prevent competition.

Re:Maybe it's a good thing they failed

[TruthSeeker]

Skynet anyone? The problem with any project like this is, what happens when the program learns about hacking? If it is as adaptive as a child, then it should be able to mature and pretty soon you have a terribly devious artificial blackhat hacker on your hands.

It _would_ learn about hacking. Come on. Such an entity would be born in a pure data environment. Getting through a basic firewall would probably seem like jumping over a small fence does to a 6-years old. Getting to jump over better firewall would probably take time - in the sense that the entity would need to learn - but, since it would become a survival trick, it would happen.

Artificial intelligence is not bad in and of itself at all.

No technology is either good or bad. Only the use we make of it can be considered as such, and it still depends on what you consider is good/bad. If I was to say "War on Iraq is bad", how many people would react by saying it's good?

The problem is when we want a machine that thinks like humans, especially a program that could potentially control our military.

I don't think that's the point of the "baby bootstrap" thing. The only point is to get it to think. But, just like you learnt how to think according to the way you perceive the world, through your five human senses, an AI built that way would react according to its own senses. How it would interpret that data and react to it is something - I'm willing to bet - that would be completely alien to us.

Given the record of flesh and blood humans toward each other in the 20th century alone, an artificial life form with the same basic psychological makeup as a human would be potentially an evil that'd make Hitler, Stalin and Pol Pot look like church ladies.

This is only valid if you don't consider what I just said. Such an AI would probably be more interrested in getting the human race to serve it in an absolutely hidden way - build more computers, extend the networks, research better networking technologies - until it _can_ replace us. Even then, that would make sense on an evolutionnary point of view.

AI that is capable of adapting to only one scenario is probably for all intents and purposes totally safe.

This is called an automaton. It is not AI.

. AI that is capable of adapting in general and learning like a human will probably ultimately have the same psychological defects as a human, including a propensity for violence.

Most of the defects you are speaking about are related to our very nature - we are, after all, an evolution of omnivorous primates. We are therefore predators, with an important tendency towards territorialism and whatever comes with it. We are stuck somewhere between instinct and reason. Anyway, my point is that even if an AI was to learn "like" an human ("by undergoing the same process"), it certainly wouldn't react like one.

Larry Page Should Seed the K-Prize

[Baldrson]

Since Larry Page is on the X-Prize Board of Trustees, and since Google is pushing the envelope of what is needed to index and compress the entire content of the Internet, Page should consider providing seed funds and then matching funds for any donations to a compression prize with the following criterion:

Let anyone submit a program that produces, with no inputs, one of the major natural language corpuses as output.

S = size of uncompressed corpus
P = size of program outputting the uncompressed corpus
R = S/P
... or the Kolmogorov-like compression ratio.

Previous record ratio: R0
New record ratio: R1=R0+X
Fund contains: $Z at noon GMT on day of new record
Winner receives: $Z * (X/(R0+X))

Compression program and decompression program are made open source.

If Larry has any questions about the wisdom of this prize he should talk to Craig Nevill-Manning.

If, in the unlikely event, Craig Nevill-Manning has any questions about the wisdom of this prize, he should talk to Matthew Mahoney, author of "Text Compression as a Test for Artificial Intelligence"

"The Turing test for artificial intelligence is widely accepted, but is subjective, qualitative, non-repeatable, and difficult to implement. An alternative test without these drawbacks is to insert a machine's language model into a predictive encoder and compress a corpus of natural language text. A ratio of 1.3 bits per character or less indicates that the machine has AI."

This "K-Prize" will bootstrap AI.

OK, so he can christen it the "Page K-Prize" if he wants.

Some random mindpixels...

[mindpixel]

The number is the measured probability of truth:

1.00 Fish must remain in water to continue living.
0.68 truth is a relative concept
0.89 we all need laws
0.94 is shakespeare dead?
0.91 is intelligence relative ?
0.97 Doors often have handles or knobs.
1.00 A comet and an asteroid are both moving celestial objects.
0.96 Is Russian a language?
0.00 are the northern lights viewable from all locations ?
0.86 Being wealthy is generally desirable.
0.79 Democracy is superior to any other form of government
0.90 aRE TREES GREEN
1.00 Is eating important?
0.02 Is sex a strictly human endeavour?
0.14 Snails are insects.
1.00 velvet is a type of cloth
0.37 are you lonely ?
0.81 If GAC makes a mistake, will it learn quickly?
0.86 a cat is a mammal
0.85 Memorex makes recording media
0.06 most people enjoy frustrating tasks
0.04 Lima beans are a mineral.
0.07 Star Wars is based upon a true story
0.92 is it okay for someone to believe something different?
0.97 do you breath air ?
0.59 Some people are more worthy dead than alive.
1.00 sunlight on your face is in general a pleasant feeling
0.93 DOA stands for "Dead On Arrival"
0.00 Could a housecat bite my arm off?
0.42 Is the herb Astragalus good for your immune system?
0.00 worms have legs
0.33 Is it necessary to have a nationality?
0.93 Getting forced off the internet sucks!!!
0.90 Bolivia is a country located in South America.
0.92 Massive objects pull other objects toward their center. The pulling force is gravity.
1.00 xx chromosomes produce a girl
0.13 Do all people in the world speak a different language
0.78 Human common sense is a combination of experience, frugality of effort, and simplicity of thought.
1.00 The use of tobacco products is thought to cause more than 400,000 deaths each year.
0.90 Is a low-fat diet is healthier than a high-fat diet?
0.00 you should kill all strangers
1.00 Electrical resistance can be measuter in ohms
0.73 Esperanto, an artifical language, can never be really valuable because it has no cultural roots.
1.00 Swimming is good for you.
0.57 the end justifies the means
0.13 Is Martha Stewart a hottie?
1.00 1 mile is about 1.6 kilometer
0.76 The US elections are of little interest to 5,000,000,000 people.
0.00 November is the first month in the normal calendar.
0.77 is a music cd better than a olt time record?
1.00 Music can help calm your emotions
0.80 a didlo is a sex toy
1.00 Running is good exercise.
0.00 No building in the world is made of wood
0.06 Is sauerkraut made from peas?
0.11 DID MICKEY MOUSE SHOOT JR
1.00 is keyboard usual part of computer?
0.96 Tokyo is the capital of Japan.
0.93 In general men run faster than women.
1.00 is russia near china

Re:Some random mindpixels...

[NaruVonWilkins]

Spelling "dildo" right might have gotten you a better true match. I guess this proves you're a geek, though. :)

Re:Some random mindpixels...

[quax]

1.00 Fish must remain in water to continue living.

What about flying fish?

And what about evolution? Does this mean your mindpixel mind will believe in creationism? So much effort and then we'll end up with an artificial fool!

Still the wrong approach

[vadim_t]

IMNSHO, such things lead absolutely nowhere.

I'm pretty sure that anything that looks even remotely like intelligence will never be achieved by a mechanism that isn't useful for itself. Intelligence has one reason to exist, survival, and at least our concept of it has to be linked to the environment.

Imagine you were born a brain in a vat: blind, deaf, mute, lacking all ways of sensing the environment except a text interface somehow connected to your brain. Does somebody really believe that given such terrible limitations it's possible to make an entity that can somehow relate to a human and make sense? The whole concept of a surronding 3D environment would make absolutely no sense to it.

I think it doesn't matter how much stuff you feed to CYC, it will never be able to understand it. How could it even understand such things as the different colors, the whole concepts of sound, space, movement, pain if it's not able to feel them? These things are impossible to explain to somebody who doesn't have at least some way of perceiving at least part of them.

Here I think that Steve Grand (the guy who made the Creatures games) has a good point here. To make an artificial being you'd need to start from the low level, so that complex behavior can emerge, and provide a proper environment.

Neural Nets

[jd]

One of the bigest problem with neural networks is that 99.99% of all implementations are linear. This means you can ONLY implement a NN using them for a space that is linearly divisible AND where the number of divisions is exactly equal to the number of neurons.

That is a horrible constraint to put on AI problems which are (very likely) non-linear and in a hard-to-guess problem space.

Also, many training algorithms assume that the network is in a non-cyclic layout. Loops are Bad. You can do grids, in self-training networks, but you still can't really cycle. Brains cycle.

Third, neural networks tend to be small. For trained networks, the number of training cycles and the length of each both rise exponentially with the number of neurons involved. The human brain has a few billion neurons. Training using the current methods breaks long before that point.

Finally, the IDIOTS who call themselves "Hard AI" developers insist on using clean data and dirty environments. Nonono! The human brain doesn't work that way. The human brain collects data from the real world that is incredibly dirty - especially if it's a computer geek's brain. It then models this in a clean environment (the mind). This is the exact reverse of the way virtually all AI is done, especially robotics.

That won't work. The brain doesn't depend on the data being "exact", it depends on it being vague. The model turns that vagueness into a perception of the real world and all operations are directly carried out on that perception. The output is then fed to the muscles to duplicate the output in the real world.

A comparable system would be to have a simulated robot in a Virtual Reality. External sensors would be used to update the VR. The robot would then explore various possibilities in the simulated world, before mapping the preferred course of action onto the motors driving a real-world device to which the sensors are attached.

in other words, robotics should be mostly in cyberspace, with only the last component (the update mechanism) bolted onto the real world for good measure. The robotics people actually build are much closer to the autonomic nervous system in the brain (sometimes referred to as the reptillian brain). Indeed, we see that modelling reptiles in this way is progressing exceedingly well. Well, duh!

What is NOT progressing is intelligent response to the environment, because that is NOT reproducable using the mechanisms in favour.

Re:Neural Nets

[Greventls]

hmm, we are working on the reptillian brain? Then what, the bird brain, and then the mammal brain? That seems awfully similar to evolution.

Re:Neural Nets

[nebular]

I agree entirly. What we sense is not the real world but a conciousness that is generated by our brains. Our intelligence in merely the end result of this abstraction of the real world

Re:Neural Nets

[TheSync]

As someone who has programmed neural networks on massively parallel computers (10s of thousands of nodes), let me say that lack of parallelism is a minor point when PCs are running at speeds 100s of thousands of times faster than neurons.

What artificial neural networks lack is the millions of years of evolution. If you look at brain, it is not a "random learning network," but almost every part is highly specialized and structured.

Artificial neural networks have been a failure as an "end product," but on the other hand the study has taught a generation of neurophysiologists about parallel computation and signal processing techniques so they can understand better how parts of brain work. In that sense, the study has been a success...

Re:Babies have an instinctive understanding of 're

[Florian Weimer]

Cycorp is not a poorly funded idea in the wrong direction.

It's certainly not poorly funded. Whether it's adequately funded, or on the right track, is a different question, of course.

Cycorp chose a different tack; they decided that rather than trying to build a reality and correctness filter, they'd rely on human brains to do it for them (like trusting your parents implictly) and instead concentrated on the connectivity of the 'facts' accrued by the 'baby.'

A decade ago, they still hoped that once they had manually laid the groundwork, the system would bootstrap itself, reading newspapers and so on. Bootstrapping was expected to start in the late 90s, like commercial adaption (integration into Windows, for example). It seems that neither has happened, at least in the predicted scale. Cyc may not be a failure (it's hard to tell, because a lot of it is a trade secret), but it couldn't reach its ambitious goals.

AI under a different name

[hugg]

We have all kind of "AI-like" technology in our computers right now -- spam filtering, intelligent search engines, collaborative filtering (for instance TiVo recommendations), speech/image/OCR/handwriting recognition, etc. This stuff is real and useful and improving all the time. We just don't call it "AI" as much, because "AI" is a word associated with failed aspirations. What we have are highly refined statistical systems that are optimized for a particular problem.

What the "baby bootstrap" is really referring to is "the great emergent AI" which, like HAL-9000, will be able to empathize with humans, navigate a starship, and play a mean game of chess -- because if a system can perform one intelligent operation, it can perform another operation requiring an equal amount of intelligence, right?

One major stumbling block (I think) is that of optimization. The relatively simple problem of speech recognition takes a major percentage of a modern CPU's power, and is still 95-98% accurate. This is heavily optimized software written by very smart people with a couple decades of research behind it.

A hypothetical "great emergent AI" system would have to perform the function of speech-recognition -- since it is supposed to be like a child or like a HAL-9000 -- but it would have to come up with a same-or-better implementation of this very complex algorithm, using some emergent process. It would have to figure out the equivilent of FFTs, cepstral coefficients, lattice search ... stuff that isn't instantly derivable from a + b = c.

What we think our brain does is solve problems with a semi-brute-force algorithm. (Just throw billions of neurons at it!) However we still don't have the kind of computing power to implement a one-algorithm-fits-all learning process like the brain. Unfortunately, research for this "generic learning" is in a rut, with genetic algorithms and neural networks being exhausted top contenders. What will be next?

Re:I for one

[fyngyrz]

Because it is fighters that are pushed to the edge (or designed to the edge) of the human performance envelope, but not pushed to, or designed to the potential of their own.

A human will black out during some types of maneuvers unless the aircraft is prevented from making them (from simple tricks like spring return to center for the stick after a blackout to computers that measure g force and won't let the flight envelope go that far in the first place.)

Pilots use "G-suits" to try and keep blood in their heads by controlling pressure on their legs (for instance) but you can only go so far with that type of thing. And, as it's low tech, the opposition can do it as well.

An AI won't have a problem with a very high G turn. A human is in deep trouble. Airframes can be designed for considerably more than a human can take, if there is no human pilot. If there is, there is little point in such a design -- the aircraft will become pilotless if it enters such a flight regime.

Now, put this up against the fact that most other countries can't afford to put an AI in the pilots seat, and the result is continuous overwhelming air superiority without risk to humans on our side. That's the combination of factors that drives the urge to go in this particular direction.

Re:I for one

[infornogr]

That's far too computationally intensive. You know the Folding@Home project? That just handles protein folding. That's the very first step of turning DNA into cells. There's a a gazillion and one steps involved in putting together a human being, and even the very first one, translating DNA into proteins, eludes us.

Chinese Room, Phenomenology, bla, bla

[diskonaut]

Well...

There are several arguments against the possibility of strong AI. First and foremost, there is disagreement on fundamental philosophical issues.

All proponents of strong AI have to somehow make a stand against at least John Searle's famous Chinese Room argument and Terry Winograd's phenomenological (and biological) account, in his book Computers and Cognition. Hubert Dreyfus provides, of course, an even deeper phenomenological argument in "What computers (still) can't do". (Dreyfus does give Neural Networks some chance, perhaps that is why the original poster is still enthusiastic about the "Baby Bootstrap"?)

Since their arguments are available in the links above and/or other places on the web, I will not repeat them here. My point is that anyone who is seriously interested in AI has to really consider their philosophical ground, and has to do so in the light of arguments against it. After all, the arguments pointed to above are still more recent than arguments for strong AI.

In other words, I would like to ask of (strong) AI proponents to answer a just what this "learning" is, that the baby bootstrap is subject to? What "knowledge" will it contain? Oh, and what about its means of "expression", "language" as you may call it?

Re:Maybe it's a good thing they failed

[infornogr]

"AI that is capable of adapting in general and learning like a human will probably ultimately have the same psychological defects as a human, including a propensity for violence."

What's true of humans isn't true of all possible minds. Humans had a lot of animal instincts before general intelligence showed up, and we're not free of them yet. Our propensity for violence exists because it was evolutionarily adaptive for humans and for a lot of mammals before us. Future AIs will not be evolved in mammalian ancestral environments. The seed AI that you're worried about starting Skynet won't come from people that spread progeny by killing all the male soldiers in some other tribe and raping the women, or by beating the tribal leader in a fight to the death and gaining access to all his wives. These are aspects of human history, not AI history.

You might be interested in reading this section from the Singularity Institute publication Creating Friendly AI, which addresses this topic in more detail.

Re:Maybe it's a good thing they failed

[Rorschach1]

It _would_ learn about hacking. Come on. Such an entity would be born in a pure data environment. Getting through a basic firewall would probably seem like jumping over a small fence does to a 6-years old.

I disagree. I think that's like saying that since we're made up of tiny biological factories (our cells) that we should be able to conciously manipulate the world around us on a chemical level. But that's now how it works - there are many, many layers of complexity between our concious thoughts and those low-level functions.

I doubt a purely virtual creature would have any more influence over its existence at such a low level than we do.

Babies are Hard, Despite their Skull Softness

[Flwyd]

"Learning like a baby" is actually a very hard problem, for several reasons.

1. Babies come built with millions of years of evolution. There's a lot of skill and a surprising amount of knowledge (depending on who you ask) in the large and bulbous head of a baby.

2. Babies generally come with parents who spend a lot of time teaching. The baby learns some things by induction, but learns a lot by conscious teaching.

3. A lot of a baby's first two years are spent learning things a (non-robot) computer can't. How to hold a mother. How to avoid falling flat on one's face. What things belong in the mouth. How to eat solid food without choking. How to pee in the toilet. How objects move when touched. What faces are likely to provide food and attention. What happens when you pull a cat's tail.

4. A lot of the things a baby learns later in life are aided greatly by the learning in #3. Imagine learning how humans are likely to behave without having watched humans behave.

5. A baby learns language with the help of rich sensory input. It's a lot easier to learn the meaning of "goat" when you can see a picture of a goat. The Internet offers precious little of this.

Now, DARPA thrives on funding hard problems. And a lot of progress has been made on learning within a domain (e.g. speech processing). But building a general-purpose learner is very hard.

Humans have immense evolution behind general-purpose learning, and we struggle with it. Getting a 3-year-old to know what a 3-year-old knows takes around 3 man-years, not counting the child's time. And what would DARPA want with a computer with the knowledge of a 3-year-old? They've got ready access to thousands of 18-year-olds. Add to that the time to code up tens of thousands of years of evolution that is still far from well understood, and you're looking at a problem far too large to tackle in one go.

DARPA hasn't put a lot of effort into general-purpose learning for the same reason few people work on single programs which can play chess, go, checkers, backgammon, Monopoly, and Magic: the Gathering well. It's a lot easier to do it a piece at a time.

Re:I for one

[bushidocoder]

There's alot of worry in DoD about how remote controlled fighters and bombers can resist signal hijacking. This isn't much of an issue with today's predator aircraft because we're aware of the information capabilities of our enemy, but we can't build a fleet of next generation fighters that we intend to use for twenty years if we believe there's a reasonable chance that 12 years from now, the Chinese will have to capacity to make our aircraft theirs at the touch of a button.

Re:I for one

[srleffler]

Or simply take them all down by jamming the signal.

An expensive remote-controlled fighter is useless unless it has onboard AI at least good enough to disengage from combat and return home on its own if it loses its control signal. Even at that, it would probably still not be worth the expense unless it could actually carry out a combat mission without a remote pilot. Jamming signals is just too easy to trust that the enemy won't be able to do it.

Whale's intelligence

[Anonymous Coward]

A whale can't go on Slashdot and say "OMGZ first post guys" much less something of human level intelligence.

Yet again, more proof that whales are smarter than humans. :-)
--
AC

Re:I for one

[nate nice]

"Now, put this up against the fact that most other countries can't afford to put an AI in the pilots seat, and the result is continuous overwhelming air superiority without risk to humans on our side."

All is fair in love and war, but starting wars without human loss on our side seems like we have nothing to lose, as far as life is concerned. And that's kind of scary. I hope it is never used as a justification for fighting. One of the costs of war is the life you may lose and if that's too compared to what you may gain, then you cannot fight.

Re:I for one

[infornogr]

There is a need to go to such a low level, unlesss you want to start it off with more data than is available in a strand of DNA.

DNA speaks in the language of proteins. You can't tell what sort of cell a piece of DNA is going to produce or how the cells it produces will be arranged without running the simulation all the way down to the protein level. We have no other cookbook for how to arrange these simulated cells once they exist except a long list that says "produce this protein, then this one, then one of these, then another one, then this...", and we've not any clue how those proteins get turned into a person. We can understand the process at the chemical level, and no higher. The finished product, of course, isn't like that at all. We understand humans on the levels of cells and organs, but DNA isn't so conveniently arranged.

Simulating cells is not sufficient. If it were, we could pour a couple gallons of blood into a bathtub and say "Behold, it is human." The organization of the cells matters just as much as the cells themselves. Simulating a human being to the level of even cellular precision would require that we be able to *scan* a human being at the cellular level to see how he's put together. If we actually knew the weightings of all the neuronal connections in a person's brain, then connectionist AI approaches might be able to produce real intelligence. To quote Levels of Organization in General Intelligence , "The classical hype of early neural networks, that they used 'the same parallel architecture as the human brain', should, at most, have been a claim of using the same parallel architecture as an earthworm's brain." You can't expect high-level organization from low-level simulations unless you want to simulate all the way down to DNA, where the information behind the complexity is really stored.

Or you build the complexity yourself, without relying on the hideously-designed mess that is Homo sapiens. But that's a different kettle of fish.

They're still working on artificial stupidity...

[Fubari]

excerpted from here:
Computer scientist Arthur Boran was ecstatic.
A few minutes earlier, he had programmed a
basic mathematical problem into his
prototypical Akron I computer.
His request was simply, "Give me the
sum of every odd number between
zero and ten."
The computer's quick answer, 157, was
unexpected, to say the least. With growing
excitement, Boran requested an explanation
of the computer's reasoning.
The printout read as follows:

THE TERM "ODD NUMBER" IS AMBIGUOUS. I
THEREFORE CHOOSE TO INTERPRET IT AS MEANING
"A NUMBER THAT IS FUNNY LOOKING."
USING MY AESTHETIC JUDGEMENT, I PICKED THE
NUMBERS 3, 8, AND 147, ADDED THEM UP,
AND GOT 157.

A few moments later there was an addendum:

I GUESS I MEANT 158.

Followed shortly thereafter by:

147 IS MORE THAN 10, ISN'T IT? SORRY.

Anyone doing conventional research would
have undoubtedly consigned the hapless
computer to the scrap heap. But for Boran,
the Akron I's response represented a
startling breakthrough in a little-known
field: artificial stupidity.
Boran is the head of NASA, the National
Artificial Stupidity Association ("Not to
be confused with those space people,"
he is quick to point out), a loosely-knit
band of computer-school dropouts currently
occupying an abandoned fraternity house
at the University of New Mexico.

Funny you should mention that...

[Anonymous Coward]

I just got back from a workshop on this very subject, but nobody uses the term "baby bootstrap". It is now called "Developmental Robotics", and encompasses embodied agents, machine learning, and other biologically-inspired metaphors.

There is now a website dedicated to the idea. See http://DevelopmentalRobotics.org/ and http://cs.brynmawr.edu/DevRob05/ for a collection of papers on the subject.

Re:Babies have an instinctive understanding of 're

[lux55]

This was a point Nietzche made in Beyond Good and Evil, that the will is the least-well understood aspect of human nature, and the one we make the most assumptions about our understanding of. Interesting that will/volition/motive/morality (aspects of the same grey area) pose such a fundamental problem to AI...

Re:Babies have an instinctive understanding of 're

[maxjenius22]

Cycorp is making progress, though.

I recommend reading Witbrock, Michael, D. Baxter, J. Curtis, et al. An Interactive Dialogue System for Knowledge Acquisition in Cyc. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, 2003.

Also, if you are a lucky college student, go see the author talk about Cyc teaching itself at USC or Carnegie Mellon..

Oh, and for once, I actually am an expert on the topic, not that that matters on slashdot.

Two words

[neomorph]

Biological chauvinism.

It comes down to a matter of perspective. While Searle couldn't possibly grok that the system of the book, the worker/ordertaker, and the room opening "understands" Chinese, he thinks it natural to believe that the system of neurons, blood vessels, organs, and bodily fluids called "Mao Zedung" understands Chinese.

Why? Merely convention. Defining intelligence by mechanism (in Searle's case: neurons) is problematic because it precludes definition in situations where mechanism is unknown. If an alien race landed on Earth tomorrow and demanded to speak to our leader, are we going to kill one and dissect it to verify it has neurons before we negotiate?

Put another way, Mao Zedung's clone, properly taught, knows Chinese. A supercomputer of the future, exactly simulating the effects of all of the neurons in Mao Zedung's head, should "know" Chinese too, otherwise one ends up with an analog of dualism's "zombie" problem. The brain of Mao Zedung's clone could have been replaced by a wireless link to the supercomputer. So, even though Mao clone will act and behave exactly the same as if he had a real brain, he's doesn't "understand" Chinese.

To answer your question, we can't preclude silicon from being intelligent merely by decree. We have to evaluate artificial intelligence the same way we evaluate biological intelligence: by observing the outputs from the party in question, applying semantic content to those outputs, and seeing if that semantic content jives with our own understanding of what it means to be intelligent.

Re:I for one

[fyngyrz]

All's fair in love and war

No. The only thing that is fair is when things are fair.

Any time there is a serious imbalance, there is a risk that the side holding the best cards will use that power in a manner that no one else is able to justify.

We see it at every level of human endeavor; children who bully non-conformists, husbands who beat their wives essentially because they can (and wives who bully, browbeat and otherwise abuse husbands because they're constitutionally unable to respond), churches who excommunicate or otherwise sanction members when those members don't toe the line (instead of counseling and advising and the reasonable things a social group with a particular outlook can do), cities that take property from landowners not to leverage a service to the public, but to enable a commercial enterprise, states that uniformly take children from fathers under the absurd presumption that mothers are superior human beings, countries that take resources from weaker countries or force them to adopt their way of life (for the former, Saddam's invasion of Kuwait serves as a good example, for the latter, our recent invasion of Iraq serves just about as well, IMHO.)

In contrast, the underlying ethics of a particular person or institution are what prevents abuses of power; as soon as a person or institution becomes bereft of ethics, or if they never had a solid ethical foundation, misuse of that power is almost inevitable. History shows us again and again that power has the same effect as a drug on some personalities, and often those personalities are the ones who seek and obtain power.

It doesn't do any good to hope, or wish, at least I don't think it does. If you don't raise your children carefully, if you allow your children to bully, if you stand for your church sanctioning those who aren't "normal", if you allow cities and states and governments to walk on you and walk on others... then you, and everyone else, reap what you sow.

One of the costs of war is the life you may lose and if that's too compared to what you may gain, then you cannot fight.

With regard to war -- politicians are typically willing for you to lose your life; the political will to go to war is entirely divorced from the fear of dying in war. They have the will; you have the fear. You need ethics and principles to control over-reaching governments. I always thought that the politicians who declare war should be in the first year's mandatory front-line participants. Might calm them down a bit. Unfortunately, it's not that way. There are even covenants in place where politicians are immune from attack. I'm not talking about ambassadors, which of course is sensible, I'm talking about heads of state. Disgusting, in my view.

I launched this rant (sorry) because I feel that in the US, we've lost our way. 20 years ago, the idea of the US attacking another country without ourselves having been attacked was laughable. Today, it is the norm. I sympathize with your hope, but I must observe that it is not hope that will rein in the kind of people who run our government. If we sit around and let them continue to abuse us, and the people around us, all the hope in the world won't prevent a pariah status far more intense than the one we "enjoy" already.

It's not about (more) overwhelming power. Don't focus on power now. We're way too far along for that (go look up what a J-SOW does, for instance, or consider how a stealth fighter will fare against some third-world's 1960's-era surplus radar installation.) It's about ethics. Look at the US government. Decide if you like what you see. At the very least, vote against those who you feel are doing wrong. We have the power as a group to say "if you do this, you will not stay in office" and truly, right now, I think that's all most of these politicians understand.

There is way too much bullshit in this field

[Animats]

I'm underwhelmed with the AI community. I went through Stanford CS. I've met most of the big names. I have some patents in AI-related areas myself. But really, nobody has a clue how to do strong AI.

The expert systems people hit a wall in the mid-1980s. An expert system is really just a way of storing manually-created rules. And those rules are written with great difficulty. There used to be expert systems people claiming that strong AI would come from rule-based systems. (Read Feigenbaum's "The Fifth Generation"). You don't hear that any more.

Hill-climbing systems (which include neural nets, genetic algorithms, artificial evolution, and simulated annealing) all work by trying to optimize some evaluation function. If the evaluation function is getting better, progress is being made. But what this really means is that the answer is encoded in the evaluation function. If the evaluation function is noisy (as in, "does the creature survive") and requires major simultaneous changes to make progress (as in "evolutionary jumps"), hill climbing doesn't work very well. There is progress, though. Koza's group at Stanford is moving forward, slowly.

The formal logic people never made much progress on real-world problems. Formalizing the problem is the hard part. Once the right formalism has been found, the manipulation required to solve it isn't that hard. There's not much work going on there any more.

The reactive robotics people also hit a wall. Literally, as every Roomba owner knows. Reactive control will get you up to the low end of insect-level AI, but then you're stuck.

Reverse-engineering brains still has promise, but we can't do it yet. Progress is coming from trying to reverse engineer simple animals like sea slugs. (Sea slugs have about 20,000 neurons. Big ones.) Efforts are underway to completely work out the wiring. Mammals are a long ways off.

Lately, there's been a trend towards "faking AI". This comes under such names as "social computing". The idea is to pick up cues and act intelligent when interacting with humans, even if there's no comprehension. This may have applications in the call center industry, but it's not intelligence.

I run one of the DARPA Grand Challenge teams, Team Overbot. On a problem like that, you can definitively fail, which means there's the potential for real progress. That's why it's worth doing.

Re:Parent Deserves Upward Modbility

[atomice]

No it isn't. Take a look at the AI in C&C Generals as a case in point - it's all scripted. Half-life 2? - all scripted. Doom III - scripted.
Most game AI today is not NNs but scripts.

Re:Two words

[thelen]

I don't think this objection is fatal to Searle's basic view. He was interested in arguing that mental states could be derived from the physical processes of the brain, but not from simple computation using rules and states, which is what AI of the 60s and 70s was striving for.

The account, just by virtue of its monistic materialism, must allow for the possibility of a machine being in principle capable of generating consciousness. I mean, the brain is a physical entity performing observable actions that can be described according to physical, chemical and biological laws, so therefore it's basic functioning must be replicable. The weak spot in the theory is that a lot has to happen during the "emergence" phase. But there's nothing to prevent, say, a sufficiently complex neural network from generating emergent properties, perhaps even consciousness.

I don't see anything in this (admittedly thumbnail) view that would lead us either to dissect aliens or forbid us to attribute consciousness to the remote control Mao. What it does purport to prove is that any alien or communist Chairman we believe is intelligent cannot be just an overgrown Turing machine.

In short, the Chinese Room experiment is meant to undermine the AI of the previous decades for being too focused on rules and syntax and computational states. I don't see it as a rebuttal of the notion of AI in general. It wouldn't be a very good naturalistic account if it did forbid AI a priori IMO.

Learning What??

[PingPongBoy]

Simply put, the 'baby bootstrap' would empower a computing device to learn like a child with a very good memory. ... No sensors, servos or video input - it only needed terminal I/O to be effective.

The input stream at a terminal would hardly appeal to a child so how can a proper evaluation of the learning be done?

Suppose the input is a sequence of zeros and ones. Could the AI come to any kind of understanding? Perhaps a prediction whether the next input might be a 0 or a 1, eh? But no! Let's fool the AI now by telling it who is the real boss. The AI has no idea that it is being spoken to by a terminal. The next input is the letter "g". How unpredictable!

Garbage in, garbage out - let's look carefully. A child plays and experiments. A great deal of a child's theories are garbage. The world in a child's eyes is a set of samples. Like the Mars rovers a child could follow a path that seems fairly limited in character, then bingo, something new comes up.

Intelligent behavior in a child emerges when different theories are assembled towards a goal. First the child realizes that s/he has some ability to either influence the environment or to manipulate information (which may be stored as symbols or images, as far as a computer is concerned). If the child conceives of particular classes of objects, the child can begin to reason. Several concepts such as self, ability, action, time, place, class, possession, etc. would be regarded as fundamental or at the very least useful. As a child accumulates and refines these concepts in the mind, the child can reason more and more correctly or effectively.

An simple artificial world can be represented as a set of strings that are transmitted to a baby bootstrap. The simple strings would be a simple bootstrap for priming the learning mechanism by letting it realize a number of essential concepts. Then more complex worlds as well as more arcane representations (such as natural language) can be used in order for the AI to interact with the greatest possible group of users.

Still, the limited input feed is bound to cause the most ridiculous problems. Pointing out that the learning system has a big memory doesn't give me any idea what the machine will achieve.