Slashdot Mirror
Artificial Intelligence Overview
spiderfarmer writes: "Well, it feels slightly odd to suggest one of my own articles, but here goes. I've recently completed a brief overview of the current state of AI. The article concept was focused on Cyc, but scope creep being what it is, I ended up doing an overview of the entire field. Some of the Slashdot gang were fairly helpful in pointing me towards experts who would talk to me and towards white papers and books I might not have otherwise found. So, I thought they might be interested in how I put all the information together."
Dr. Lenat and others in the field of AI research should know better than to make claims about consciousnes and morality in a public forum. Cognitive scientists don't even begin to agree on what consciousness is, let alone what it would take for a machine to replicate it. Some very respected individuals do not even think that human consciousness can be replicated within the forseeable future (e.g. Roger Penrose's The Emperor's New Mind). Like any other scientific discipline, these sorts of claims should be left to peer review. Claiming to have invented a conscious machine would be akin to a physicist claiming to have unified quantum with relativity, but without having submitted their findings to any publication.
try A.L.I.C.E. (that's http://www.alicebot.org/ for the goatsecx paranoid). Its one of the better bots that has won awards and stuff. Sure it isn't perfect, but its a neat toy to play with.
Good quote, too many chars. Seriously, the slashdot 120 char limit sucks!
The scientist's explanation took one paragraph, and even sounded like it had a goal - allow a machine to use an encyclopedia to gain new information in a useful manner. This is an important step to an A.I. that can interact with people - you can then train it on reference materials, and have it "understand" them at a certain level.
This scientist is NOT mistaken - he would have to know that "common sense" does not equal "the human brain's inate ability to make sense of the culture it grows in". If I had to draw a distinction, "common sense", as you are describing, is static, tuned to one culture, while the "common ability" is semi-dynamic, able to learn, but (maybe) unable to unlearn.
You could try to fake common sense, by programming your own cultural assumptions into the program, subjecting it to cultural stimulus, and fine-tuning the program. Or you could attempt to program "common ability", train it on cultural materials for a few years, and try to tune the program to build it's own "common sense" in a way that is more like a human. I think these scientists are trying to do the latter.
I'm not sure what your tangent about post-modernism and 1984 have to do with A.I. - are you just making a rant about scientists who didn't get the memo that we are in post-modern times?
An interesting question is if human intellegence can be removed from the human - does it take eyes to understand the phrase "I've got the blues"? Does it take a parent to understand why many grade school teachers are women and most world leaders men? Does it take walking upright, starting at a tiny height and getting bigger, to understand skyscrapers? Or does that just take a penis?
Now, I'm using the "white-space" sense of understand - to be sympathetic to the person who has the blues, to feel an unexplained shame when the president is caught sleeping with a women not his wife, to feel an exhiliration driving into a new city. Can these be simulated in a computer without a body and a human's lifetime? Can these things be removed, still leaving a "human" intellegence? If we interacted with this intellegence, would we say it passed the Turing test? Would we want to interact with it?
Perhaps that's one level of A.I. above where this guy is aiming. It would be extremely useful just to have an intellence with a little of the human ability. You could train it on, for instance, medical journals. A doctor could then descibe symptoms, research, or an interest, and get summaries or references to the library. Once you trained it in the basics, you could burn it to a CD, send it to a doctor, who could then train it for his specific interests. Think of it as a very limited secretary, who requires some training and aclimation, but is still smarter than a PC.
This is probably the best A.I. can do for a few years - get to the point where you can train an A.I. for a particular subject, then meaningfully interact with those interested in the subject - like a very bad librarian. It's only when the clones come out in force that you can hook a computer up to a fetus, and do some real human A.I. training.
I'm only 30 years old, and as a child I remember the "scary" face of AI being presented to the public as living machines which would out-think humans and render us redundant, engineered by unethical scientists.
Nowadays AI is never mentioned in popular media. It has been replaced by the new emphasis in public-facing science: cloning and gene therapy.
This is the new AI in the mind of the ordinary citizen. It will lead to the destruction of the human race, and poses many ethical and moral questions. In the UK it is being demonised by the popular press without real debate, much like AI probably was 20 years ago.
Incidently, this was an excellent "heads-up" article for a novice like me, and I gained significantly from it.
"If you look at an encyclopedia, you'll see a great deal of knowledge of the world represented in the form of articles. Common sense is exactly not this knowledge. Common sense is the complement of this knowledge. It is the white space behind the words. It is all of the knowledge that the article writer assumed all of his/her readers would already have prior to reading the article -- knowledge that could be put to use in order to understand the article. Cyc is about representing and automating the white space." (I love that answer.)
Common sense is about representing and automating the white space?
I think these AI researchers need to talk to a few more sociologists. Human common sense is extremely culturally divergent and goes far beyond the simple, textbook logic cases that certain engineers in this field would probably cite. "Reading between the lines" involves not some native common sense that is wedded to intelligence, but a collectively evolved cultural contextualization. When we read an article in an encyclopedia, a lot of other stuff other than intelligence comes into play: x years of public school education, idiomatic constructs, varying by geographic location, that may or may not enhance or obscure meaning, and, of course, the double meanings and entendres inserted by bored or biased encyclopedia writers.
The entire postmodern project of literary criticism has been aimed at proving this point- at proving that there is no such thing as a standardized set of meanings, and that every meaning is contextualized. The Modernists wanted to rationalize and bureacratize speech, to restrict the number of meanings, and to leave what is unsaid in a narrow, predictable whitespace of a unified "common sense."
Of course, there is a language like this, developed in the first half of this century. It takes away as many English words as possible to restrict the meanings that we are able to THINK, let alone say. Of course, this language is called Newspeak.
Goat sex free since 2001
My name isn't really Jenny....
I used to work in AI Alley in Cambridge during the 80s. The industry got hyped to death by claims and demands on performance that wasn't possible. Lots of interesting things were done and lost during that timeframe. The thing is, we hadn't even reached artificial stupidity at that point. I'm not sure we have yet but we're closer in some domains. Until people unlink visions of Cherry 2000 and C3PO from the AI moniker, it's going to be tough getting people to set realistic goals. Just like the Lisp environment back then, I see the research and technology backdooring into products every day. More and more of the stuff that we did back then is making it out as "new technology" under Microsoft when we had it 20 year ago, but hyped it into obscurity. The industry seems to be coming out of it's own dark ages lately and I hope the media doesn't get back on the bandwagon to beat it back down. I thought the article was very good and you did a LOT of good research and didn't just edit together all the buzzwords you found. The response has been good in the community I still keep in touch with and you better watch out or you'll be changing people's minds about talking to journalists. Many of us wish you had been given more access to Cyc and their underlying knowledge engineering techniques to do that topic/aspect justice as well.
Good Job!
Me : Can u imagine a Beowulf cluster of yourself?
Cyc : -1 Troll.
With an academic background in moral philosophy and today working as a developer of data mining systems, I can empathise with the author's frustration at trying to understand how (if at all) morality and AI intersect.
The main problem really is that the term 'AI' is applied to any algorithm for classification, prediction, or optimisation which operates using anything beyond a simple set of heuristics. Such algorithms seem magical to the lay-person, resulting in the over-enthusiastic application of the 'intelligence' moniker.
Summary
'AI' is a term used inappropriately for a range of algorithms that attempt to learn without having to specify an exact set of rules for every case. Although these algorithms are currently incapable of displaying real intelligence, it is possible that one day they may. This point is however debatable, and the interested reader should read for themselves the differing points of view of experts in the field, including Daniel Dennett, Roger Penrose, Steven Pinker, Richard Dawkins, and Douglas Hofstadter. If they do ever get to the point that they can act intelligently and flexibly, it will be important that they are trained with appropriate moral premises to ensure that there actions are appropriate in our society.
To understand these so-called 'AI' tools it is useful to develop a little structure...
Output
AI tools are used for classification, prediction, or optimisation. Classification works by showing a computer a set of cases which have a number of properties (sex, age, smoker status, presence of cancer...), and 'training' the algorithm to understand the patterns of how properties tend to occur together. Prediction can then be used to show the algorithm new cases in which one or more of the properties are blank--the algorithm can use its classification training to guess the most likely values of the missing properties. For instance, given sex, age, and smoker status, guess the probability of presence of cancer. is a generalisation of classification--rather than training to minimise classification error, train to maximise or minimise the value of any modelled outcome. For instance, whereas an insurer could use classification algorithms to find the likelihood of someone dying by age x, an optimisation approach could be trained to find the price at which modelled profitability of an applicant is maximised.
Functional form
AI tools create a mathematical function from their training. For instance for a classification algorithm this function returns the probability of a particular category for a particular case. The form of this function is an important factor in classifying AI tools. The most popular forms are 'neural networks' and 'decision trees'. Neural networks are interesting because certain types (networks with 2 hidden layers) can approximate any given multi-dimensional surface. Decision trees are interesting because given a large enough tree any surface can be approximated, and in addition a tree can be easily understood by a human, which is very useful in many applications. Other functional forms include linear (as used in linear regression which many will remember from school) and rule-based (as used in expert systems, and similar to a decision tree). One interesting functional form is the network of networks which combines multiple neural networks, feeding the output of one into the input of others. This forms allows the training of network modules that learn to recognise specific features, which is closer to how our brains work than the single network approach.
The most flexible functional form is that used by practitioners of genetic programming (which also defines a specific training function). Genetic programming creates a function which is any arbitrary piece of computer code. The code is often Lisp, although lower level outputs such as assembly language and even FPGA configurations have been used successfully.
Training function
The training algorithm looks at the past cases and tries to find the parameters of the functional form that meet the classification or optimisation objective. This is where the real smarts come in. One naive approach is to try lots of randomly chosen parameters and pick the best. Genetic algorithms are a variant of this approach that pick a bunch of random sets of parameters, find the best sets and combine features from them, introduce a bit of additional randomness, and repeat until a good answer is found. Local/global search works by picking one set of parameters and varying each property a tiny bit to see whether the result is improved or gets worse. By doing this it locates a 'good direction' which it uses to find a new candidate set of parameters, and repeats the process from there. Hybrid algorithms are currently popular since they combine the flexibility of genetic algorithms with the speed of local search. Most neural networks today are trained with local search, although more recent research has examined more robust approaches such as genetic algorithms, Bayesian learning, and various hybrids.
Learning type
Supervised learning approaches take a set of cases for training and are told "here is the property we will trying to predict/optimise, and here is it's value in previous observed cases". The algorithm then uses this context to find a set of parameters for the functional form using this context that the analyst provides. Unsupervised learning on the other hand does not specify prediction of any particular property as being the training goal. Instead the algorithm looks for 'interesting' patterns, where 'interesting' is defined by the research. For instance, cluster analysis is an unsupervised learning approach that groups cases that are similar across all properties, normally using simple measurements of Euclidian distance (that's just a fancy word for how far away something is when you've got more than one dimension).
Contextual learning is a far more interactive approach where the analyst interacts with an algorithm during training constantly providing information about what patterns are interesting, and where the algorithm should investigate next. Systems like Cyc use contextual learning to try to capture the rich understanding of context that humans can feed in.
AI and moral philosophy
We are still a long way from seeing an algorithm that can interact in a flexible enough way that we could mistake it for human in a completely general setting (the Turing Test for intelligence). However, given the ability of flexible training functions such as genetic algorithms, we may find that one day an algorithm is given enough inputs, processing power, and flexibility of functional form that it passes this test. The 'morals' that it shows will depend entirely on the inputs provided during training. This is not like humans, who have some generally consistent set of moral rules encoded through evolutionary outcomes (for instance, tendency to care for the young and related). Our moral premises are the underlying 'givens' that form the foundation of what we consider 'right' and 'wrong'. Ensuring that an AI algorithm does not act in ways we consider inappropriate relies on our ability to include these moral premises in the input that we train it with. This is why Lenat talks about teaching Cyc that killing is worse than lying--this is potentially a moral premise. Finding the underlying shared moral premises of a society is a complex task, since for any given premise you can say 'why?' But repeatedly asking 'why?' you eventually get to a point where the answer is 'just because'--this is the point at which you have found a basic premise.
Summary
'AI' is a term used inappropriately for a range of algorithms that attempt to learn without having to specify an exact set of rules for every case. Although these algorithms are currently incapable of displaying real intelligence, it is possible that one day they may. This point is however debatable, and the interested reader should read for themselves the differing points of view of experts in the field, including Daniel Dennett, Roger Penrose, Steven Pinker, Richard Dawkins, and Douglas Hofstadter. If they do ever get to the point that they can act intelligently and flexibly, it will be important that they are trained with appropriate moral premises to ensure that there actions are appropriate in our society.
I hope that some of you find this useful. Feel free to email if you're interested in knowing more. I currently work in applying these types of techniques to helping insurers set prices and financial institutions make credit and marketing decisions.
Jeremy Howard
The Optimal Decisions Group
Don't make the mistake of attirbuting human motives to a computer just because it has a certain characteristic. I know that this can be tempting...
... well, she's quite familiar with the way to the Japanese resturant, and has solid habit patterns that tend to lead her in that direction from some corners when she must move, and doesn't have a direction firmly in mind. ... Was I talking about the AI or my wife (or my wife's car)? Well, the car doesn't have that kind of brain. But suppose you got in the car and said, "Take me to dinner.", that might be a reasonable description. But my wife would do it because she wanted to. The AI would do it because it was a minimum effort solution to a common problem that had just been posed.
... it would need to learn to avoid telling jokes. And it might find that quite ... disturbing? But to attribute any particular human emotion to the process would be .. misunderstanding. There would certainly be emotion analogs, but they wouldn't be close analogs to any particular human emotion. Or, if they were, it would be quite surprising. (They don't have the same "genetic" history determining their substrate ... it's much MUCH more reasonable to attribute similar emotions to cats.)
My wife talks of the car as "knowing the way to..." or "wanting to go to...". She doesn't actually believe this (I don't think she does), but she thinks I'm being silly to object to putting things this way. But when thinking about AI computers, this can be a good (and dangerous) model. "The car knows the way to the Japanese resturant."
The difference is that the AI doesn't have the motives for initiating action. Now some designs have "super-goals" that probably will never get fulfilled, but a) they didn't choose those goals, and b) someone else gave the goals to them. Of course a car might well have built in desires to keep the tires safely inflated, to avoid running out of gas, to keep the battery charged, etc. But these are quite different from anomie.
Perhaps people would choose to build AI's capable of feeling lonely. But this would be a design decision, not inherent. Or perhaps they would feel something that would be translated into English as lonely, but for which super-goal frustration in the absence of actionable choices would be a better name. It might well not have the rise and fall pattern of human loneliness. Or it might. A hierarchy of needs might cause an AI to experience a similar rise and fall in level of frustration at incapacity for making progress in less important goals. As if a car with the lower importance injunctions to "keep your owner healthy" and "laughter is the best medicine" were owned by an asthma sufferer
I think we've pushed this "anyone can grow up to be president" thing too far.
I would recommend re-thinking your division of AI into subfields. You are indescriminately mixing technologies and application areas.
For example, neural networks are a technology and NLP is an application area. I know people working in NLP that use Lisp, and I know others that use neural networks. In AI, technologies and application areas are (mostly) orthogonal.
Granted, there probably isn't a perfect breakdown of AI into subfields, but making the distinction above will help you and your readers get a grip on what AI is all about faster.
Sheesh, evil *and* a jerk. -- Jade
I was suprised that there was no information at all to be found regarding genetic programming. This method builds a large population of random computer programs and then refines them through genetic mutation to accomodate a specific task. Darwinian selection ensures that only the most fit programs survive, and less useful ones die off quickly.
I have been doing some work involving genetic programming lately, and have found it to be an amazing tool for finding creative solutions to complex problems. The problem domain I have been training my genetic program to solve is purely mathematical, but it seems to me that the technique could easily be adapted to find solutions to some of the tougher problems in AI, including but not limited to: data mining, natural language processing, and parellization.
I read somewhere (can't find the reference right now, sorry) that some work was being done whereby the genetic programs were being evolved that could themselves create neural networks. Each genetic program could be considered a template for creating a neural network. This seems to me like the most likely means of creating a software that could eventually pass a Turing test. I won't get into the self-conciousness debate here.
std::disclaimer<std::legalese> sig=new std::disclaimer; sig->dump(); delete sig;
The A.I. effort is often classified into two camps.
One is to approach human intelligence. This usually
implies conversational ability, since a hallmark of
human intelligence is language. This A.I. approach is
called "hard A.I.".
Soft A.I. looks at sub-problems, such as problem
solving, image understanding and so on.
Many of software inovations originated in A.I.
labs (e.g. interactive editors, bitmap graphics).
(During the early 80s these spinoffs were sometimes
confused with A.I.)
A problem with both kinds of A.I. is that its a
receding target. Once an important goal has been
reached, e.g. a chess computer that beats grand masters,
people write it off as a nice trick,
but not really A.I.
So I proprose what I call "interesting A.I.".
Two hallmarks of human intelligence are language
and curiosity. So if an A.I. could TELL us
something new and interesting on a regular basis,
then I would call it a success.
I suspect A.I.s will first arise in entertainment
computing: either as a robo-toy, a synthetic game
player, or synthetic actor in a film. This will be
a results of people's drive for challenging
creative play.