Slashdot Mirror
Alan Turing's Prediction for the Year 2000
Chernicky writes "In 1950, Alan Turing , the father of computer science and (arguably) artificial intelligence, made a prediction about the year 2000. Turing said that in about fifty years, the answers of a computer would be indistinguishable from those of human beings, when asked questions by a human interrogator. With the year 2000 upon us, Dartmouth College is offering a $100,000 prize to the first programmer that can pass the Turing Test. The deadline for submissions is October 30, 1999. "
Dartmouth College is offering a $100,000 prize
to the first programmer that can pass the Turing Test.
Ummm, I sure hope I can pass the Turing test. I know some of you out there might have problems passing it, but I'm pretty confident I can pass.
Of course if he meant 'first program' it might make more sense.
The Turing Test has long been discounted as a bad goal of AI research, although people have been doing Turing Test "auditions" for years.
;) )
:-)
:-)
The problem with the Turing Test is that it tries to make a computer human and that's not really what AI is all about - it's more about trying to solve problems using various techniques in order to make programs useful. (Maybe making a computer human is not all that useful
The problem is that the program only needs to pass 5 minutes worth of conversation. That's a pretty narrow goal. Technically, it's not really artificial intelligence at this point - it's just a ruse (however, it's still extremely diffucult to program natural language capabilities and have "common sense" -- two goals that are themselves not bad ones to do research in).
Douglas R. Hofstadter wrote an interesting article about this - he had a conversation with a program named Nicolai (I think). It was quite amusing - the program spits out some very interesting answers.
Anyway, no one has yet succeeded at this and if you feel you can get a program to imitate a human for 5 minutes, go right ahead. You'll earn that $100K
Woz
what I posted a minute ago may show up soon, but I can't be sure, so here I post it again.
Is www.forum2000.org a fake? or is it an honest-to-deity AI capable of answering questions in a lifelike manner?
Dan
It was only after I myself had begun setting up a BBS that I came across this BBS door program. I don't remember what it was called, but it pretended to be a chat program. Basically, it responded to specified keywords with a random sentence from a huge flatfile database, and even pretended to have typooo^H^Hs from time to time.
I then realised that I'd been had!
Some sysop must have been laughing his ass off at this young kid who went by the handle "Orion", chatting away with a very crude AI and being suckered into it the whole way.
I look back on those days and wonder how I missed it. But it just goes to show you that, as much as you might be fooled by a computer, we've got a long way to go before we reach anything approximating independent thought. Personally I don't think it'll ever happen - but it might be neat to be proven wrong.
please do not mistake my intent- i would be quite
impressed with anyone who could pass the turing
test.
however, how much further does this really get us
than building a computer which can beat kasparov
in a (relatively) high speed chess match. chess
seemed like a big thing to teach a computer once,
but it has been relegated to the relatively
trivial now.
it seems to me that a program which passes the
turing test may well fall into the same category.
(i am assuming here that the program merely
appears to be having a conversation- that it is
not a language _understanding_ system.) it would
simply become something that people would set
loose in chatrooms, or attach to old unwanted
e-mail accounts, and watch the fun.
what i'd like to see is someone tackling a truely
significant problem. like programming a computer
to be able to vaccume your house.
(setq howareyoulst '((how are you) (hows it going) (hows it going eh)
(how\'s it going) (how\'s it going eh) (how goes it)
(whats up) (whats new) (what\'s up) (what\'s new)
(howre you) (how\'re you) (how\'s everything)
(how is everything) (how do you do)
(how\'s it hanging) (que pasa)
(how are you doing) (what do you say)))
(setq qlist
'((what do you think \?)
(i\'ll ask the questions\, if you don\'t mind!)
(i could ask the same thing myself \.)
(($ please) allow me to do the questioning \.)
(i have asked myself that question many times \.)
(($ please) try to answer that question yourself \.)))
(setq foullst
'((($ please) watch your tongue!)
(($ please) avoid such unwholesome thoughts \.)
(($ please) get your mind out of the gutter \.)
(such lewdness is not appreciated \.)))
(setq deathlst
'((this is not a healthy way of thinking \.)
(($ bother) you\, too\, may die someday \?)
(i am worried by your obsession with this topic!)
(did you watch a lot of crime and violence on television as a child \?))
)
(setq sexlst
'((($ areyou) ($ afraidof) sex \?)
(($ describe)($ something) about your sexual history \.)
(($ please)($ describe) your sex life \.\.\.)
(($ describe) your ($ feelings-about) your sexual partner \.)
(($ describe) your most ($ random-adjective) sexual experience \.)
(($ areyou) satisfied with (// lover) \.\.\. \?)))
(setq stallmanlst '(
(($ describe) your ($ feelings-about) him \.)
(($ areyou) a friend of Stallman \?)
(($ bother) Stallman is ($ random-adjective) \?)
(($ ibelieve) you are ($ afraidof) him \.)))
-- Virtual Windows Project
It just occurred to me that in the movie Blade Runner (and the PKD book as well?), the test for whether or not a suspect is a replicant or human is basically a fancy Turing test, isn't it?
Place the subject in front of an interrogator and try to provoke an emotional response, indicating humanity. Sufficiently advanced replicants are good at fooling the test ("Rachel took nearly 50 questions") but to date all replicants are distinguishable from humans.
Seems pretty allegorical to me. What was the test called in the film? Who was that doctor / scientist? Would he have been eligible for the reward?
Man I want to see that movie again now...
Other thoughts, since I'm on a tangent: how about a program that can seem more real than Zippy the Pinhead? (Shouldn't be too difficult.) Or one that is less boneheaded than the average Slashdot AC poster? (Shouldn't be too difficult.) Sounds like it's time to get coding...
DO NOT LEAVE IT IS NOT REAL
I seem to remember that Zummy was fooling a lot of reporters (they thought that the 'bot was actually a Linux technician). Perhaps the maintainer should try and submit that?
Hold a Turing Test that has only computers or only humans answering and don't tell the judges. See if you get some judges saying "That was a computer for sure.", or vice versa.
This raises yet another issue with the test -- a human can very easily give responses like a computer, thus fooling the judges. Is that fair? Maybe some humans are like computers with their answers.
In fact, one time a participant was talking about Shakespeare, and was a complete expert on the subject. The human jugde was conviced he was a computer because his answers were so exact!
Yet another problem....
Woz
Erwin!
I was on a MUD. Somebody struck up a conversation with me, and then suddenly stopped. He turned to a companion and said, "OC: I feel really dumb -- I actually thought that 'bot was another player."
I must say that I was rather embarassed at being thought a 'bot, and immediately denied it -- at which point the other player said, "OC: Well, it is really believeable -- see how it even denied it was a 'bot? Whoever wrote it was good."
From what I read, most people working in AI don't treat them as something worth while. It's fairly obvious that programs won't be able to pass the turing test for some time (decades, maybe centuries), and the results of such tests only make it less likely that people working on valid AI projects will be taken seriously.
The Loebner Prize has it's own homepage. Chech out the transcripts of the conversations. The most 'clever' programs simply look for keywords, some insist on asking all the questions, some are 'whimsical' and use metaphores while constatnly switching topics, none show any comprehension whatsoever.
People give Eliza way too much credit. I mean, it's a fun little program and was mostly cool for a while, but it's not even close to be able to pass the Turing Test. Since Eliza replies in questions to avoid contradicting herself, the only way she could pass a Turing Test is if you were talking to a New Yorker.
-
User: Are you smart?"
Pesci: What'dya mean, "Are you smart?" You think I ain't smart? I'm just an idiot? Huh? Is that what I am? An idiot?
User: No, that's not what I meant...
Pesci: Oh, so now you're calling me a liar? I'm stupid and a liar?
User: Uhh, are you okay?
Pesci: Yeah, I'm fine. Why? You think there's something wrong with me? You think I'm crazy? Huh? I'm crazy? I'll show you fucking crazy!
User: I think this program is bugged.
Pesci: Oh, so now I'm broken? Huh? You think I wasn't programmed right? You think I'm supposed to sit around and take your shit? Huh? I'm supposed to be nice to you? Huh? I'm just a program? Is that it? Just a program to amuse you? Is that what I am? Is it?
User: No, I didn't mean it like that...no, please, don't...nooooooooo!
*BLAM*BLAM*BLAM*
Programmer: Oh, shit, Joey, you didn't need to do that...Oh, shit. I knew there was something wrong when the beta testers disappeared...oh, shit, oh, oh, oh, shit...
We all know you're a bot, so you may as well stop trying to fool everyone.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
It's actually quite simple. You have to pick your subject carefully (intelligence is something we want to avoid) and then spring it on them when they don't expect it.
I did this with AOL Instant Messenger. I saved a bunch of my gaim conversations and then read over them and customized Eliza to make it sound as much like me as I could. Then some perl magic to make it work with Toc and I left for a party and then a movie.
I got back at around 2:30 in the morning and saw a friend talking to it. He had been chatting since 11:00 pm!!! He didn't even dimly suspect that it might be a computer, but he was getting pretty pissed off - it was saying pretty stupid stuff that usually didn't make sense, and it repeated itself every 5 or 10 minutes.
I laughed over that one for a looooooong time. It might not work anymore, tho... anyone know if Aol pulled the plug on Toc?
--
grappler
Vidi, Vici, Veni
One particular episode that comes to mind is The Saga of Roter Hutmann , available at http://www.nothingisreal.com/saga/. This is the story of a computer science major who spent hours every day talking with Julia, a Turing test program, even going so far as to ask it out on a date, before he finally voiced to me his suspicions that she was "not human". Ironically, he then proceeded to call her a poorly-written program... Julia used to be accessible via telnet (fuzine.mt.cs.cm.edu, user "julia") but, alas, is there no more...
Anyway, check out the Saga if you've got a few minutes to spare as people keep telling me it's the funniest thing they've read for a long time...
Regards,
...is that it avoids any argument about whether a program is really intelligent or actually 'understands' by defining intelligence as behaving sufficiently like a human (ignoring the physical aspects of humanity) that other humans accept it as one.
:-)
This isn't easy at all -- imagine asking a computer program to not only suggest a move in a chess game, but to write a poem about a subject of your choosing, compare and contrast two public figures, and so on.
I don't think any of this can be done without a *deep* understanding of language and human culture.
Of course there are *many* very useful things for AI to achieve which fall short of passing the Turing test -- in fact I think by the time we can pass the Turing test we'll probably have achieved everything else -- except super-human intelligence, but perhaps that's just a matter of cranking up the clock speed
I have discovered a wonderful
Although the Turing test is widely regarded as a tool to test intelligence, this statement is very questionable. The Turing test only test how well a program can simulate a human.
For example: if a friend of yours can multiply two large numbers, you'll say he's smart; however no-one ever called a calculator 'smart'.
If a person memorizes all countries with their capitals, you'll also consider this person intelligent; computers are far better in memorizing things.
The point is that this program (trying to pass the Turing test) will not only have to fake intelligence, but also stupidity. If the interrogator asks it to factor 4553536663, it will have to lie and say it doesn't know or it will loose credibility. The question here is: is it favorable for a computer (or any other device) to deny its capabilities, just because our definition of intelligence might be a little off ?
42 !
They have this contest every year. Some years, the contestants do well, others, not so well. When taken as an abstract, i.e. "A computer that can always fool any human for any length of time into thinking that he is talking to another human", the Turing Test is valid -- but untestable. Once you put constraints on it ("these 10 people for 15 minutes...") it's no longer valid because each constraint is a weakness (maybe the people were stupid. Maybe if they just had time to ask another question they would have been able to tell the difference...)
I think something important that's forgotten frequently in dealing with natural language technology is that right now, in almost all cases, you don't want to have a conversation with your computer! You want to tell it to turn on the lights, and to ask it how much money you have in the bank, and to find cool new warez and MP3s, dood. The sentence structure for queries and commands is far different (and far similar) than trying to parse out conversations in which context almost always becomes the downfall of comprehension.
Someday, yes, people will want to have a conversation with the machines that control their houses. I envision a machine that can tell by my sentence structure what mood I'm in, and put on some appropriate music, set the lights, and so on. But those things will all happen *after* we get the basics down, like differentiating "Lights on" from "Could you turn on the lights please, computer?" and having them both do the same thing. Nobody would call the former true natural language. It's when we can do the latter, and have "noise suppression" be so seamless that you can say what you mean in almost any conceivable way, that people will take it seriously as an interface.
www.HearMySoulSpeak.com
About the Turing test. There are many myths and legends about it. People claim certain knowledge of the test, but don't think them out and see what's wrong. Common sense applies more here, than any university degree.
;) in some more obscure areas. So we don't have a clue what it's all about!! Really. We just like to simplify things to the bone. And make ourselves look better than the crowd.
;). In computer programs it will ne unaffected unless the technitian trips over a wire or something.
:(
;] ). All these particles (or more correctly multi-dimensional waves), are processing internal data and interacting with their surroundings. Therefore, everything physical can be considered to exhibit a certain degree of intelligence!!
;)
One claim is that the Turing test is the only way we got to determine if a computer program is intelligent or not. This is derived out of the notion that we think we can recognize intelligence when we see it. But the test says nothing of common error probability (many humans have actually failed the test for being an AI), and the capabilities of the judges. If you read some of the transcripts from the past official Turing tests you'll be horrified how quick some judges are to judge, and what simple questions they ask. Many of them appear to be bored with it all. This also applies to the human candidates. Some of these faults in the past can be blamed on poorly written programs, that couldn't compete in any way. The past Turing tests actually had limited discussion topics, so that the programs could be programmed for a specific discussion topic. But think of a super-program (that is not super by today's standards) among those. It could actually pass in the tired and disappointed athmosphere four years ago. To quote from "Tomas Covenant The Unbeliever": Any test is just as good as the tester himself.
About Humans. In our arrogance we say that we are intelligent, and everything else is not. We are amazed and dazzled by pets who performs instant rescue operations in fires and drowning accidents. For how can animals be intelligent? We don't measure intelligence, we blatantly state that things around us that ain't human is not intelligent. By unconciously applying our own version of the Turing test to everything around us. Of course, many of us do regard animals as intelligent, to a lesser degree, but most humans think of intelligence as a binary state.
About Intelligence. But it can be measured. It's not an ON/OFF switch for us to decide it's state. Heck, we don't really have a clear-cut definition of intelligence even today! Other than that faulty "It's not human-like" negativity test, and IQ tests which is only a test to separate "dumb" people from the rest.
And there isn't just One Kind of Intelligence (to Rule them all). You have social-, technical-, langual-, mathematical-, logical-, motoric-, coordinatic- and many, many more intelligences. There exists no test that tests it all, and no tests are very accurate. Many people who are considered "dumb" really excell (how the hell is this bloody word spelled?
My definition of an intelligent system, is an open-minded and positive test. Wether I can measure it or not a system is intelligent to a certain degree if it contains information and processes this information within itself. It MAY receive input data, and it MAY emit output data, but that is only essential to my perspective of knowledge (not beliefs). The type of data-storage medium is not essential. Neither is the medium processing the data. The essential is that information is being altered inside the system, and fed back in a feed-back loop. Thus, the system has a way of "viewing itself" (definition of a reflective system).
The internal processes can involve operations like copy, addition, inverse, etc. These would be atomic functions. While multiplications, subtraction, divisions and exchanges would only be optimizations, since they always could be expressed by a set of atomic operations. But the data doesn't have to be numbers, and the atomic functions would be different for neural networks, images, symbols or even colours for instance.
To complicate things even more, processes could run in parallell internally in the system. In real life, nevral networks in our brains all process in parallell to a certain degree. (Ie. I'm sure there are semi-synchronisation methods between parts of the brain, even though they might be complex or chaotic)
In information theory, you can express any information in binary numbers (00101011). This simplifies things, but you'll need a non-ambigous specification to convert data both ways. Some types of data could perhaps be more effectively processed than strings of binary data (ie. linked-lists, images, chinese symbols), simplified in complex structures of binary strings.
Input and output data in a feedback-loop would permit the system to develop with its surroundings. To what extent is unknown. Ie, how much intelligence and knowledge would the two systems contribute to each other? Limitations would be imposed by information storage sizes, lack of atomic functions, dead-end loops, etc. Especially lack of creativity (a random function) would be a dramatic limitation to the extent of intelligence and knowledge possible to be learned and taught. Read-only areas in the system's data or process-storage would be another severe limitation.
Systems lacking a trait that exists in another system could interface with that other system in a symbiosis, to use the resources found there. This is in the extreme case the basic principle of an artificial neural network. Where everything is shared holographically in the structure of the neurons' connections (and each connections weights).
On the difference between intelligence, knowledge and their respective levels. The usual pit-trap is to not distinguish intelligence and knowledge. I prefer to define level of knowledge as the amount of non-redundant information a system can internally access within a given time/number of cycles. While level of intelligence to be complexity of a given task to be solved within a given time/number of cycles.
These levels are next to impossible to measure very accurately in real life, but of course you have imperfect methods. Just not count on them for anything else than what they are. One type of method is to measure intelligence from the output of the system, in light of the input data or not. You can also test intelligence by scanning the actual code and data the system consists of, if you are able to "X-ray" it. You will have to be able to determine how intelligent the algorithm is. Of course, in real life, the observation will always affect the state of a running system (Real life is ALWAYS On, darn
These definitions leaves one thing hanging if you're calculating in real-time: processing cycles per time unit (e.g. 450 MHz). I don't consider a system processing large amounts of data (a supercomputer) to be more intelligent, by the definition above and "common" reason. But you would have to multiply this speed with the intelligence level to get the total intelligence-effect (ie some of what Turing and IQ tests are really testing).
I know this is all hard and difficult to understand and think over. The definition is very impractical too. But it's a much better place to start, than just saying "I don't see the intelligence in this" when you haven't even decided for yourself what intelligence really is! That simply shows alot of ignorance. Besides it's the modern way to go. Most AI programmers building neural network live by it. (Sadly I'm not
The definition doesn't exclude anything physical the right to be intelligent. We human beings consists of thrillions of living cells. They in turn consist of billions of atoms and molecules. Which again turns out to consist of even smaller "particles" of less physical nature (see the religion of modern science [not a book, it's for real!
I think this ALSO applies in cases where we are not able to detect the output data or the non-human intelligence in it. Science is too eager to test for negativity and simplify things, thus many creative theories are crushed by the latest dogmas. (Scientific people think they know better than everybody else just because they use fancy language to make themselves misunderstood.)
Now if you've grasped the ideas I've expressed here, you'll know that the Turing test is a bogus test. Both in the computer lab as well as in real life.
- Steeltoe (really tired of hearing those people say Turing test is all we got)
PS: Gee, this edit-window is tiny!
http://www.debunkingskeptics.com/
I think this has been discussed a little in another thread, but the Voight-Kampf test did not measure intelligence or the ability to "pass" as human, it measured empathy as a measure of true humanity.
This is an important distinction. The replicants were already Turing compliant, but they were not human. Dick believed that empathy was the defining aspect of being human. Dick's replicants would have been able to pass any Turing test with ease. In fact, they passed the most difficult Turing test of all: they were able to live in human society, hold jobs, sing Opera, make love; but they weren't human.
You sound a lot like Eliza yourself with that comment :-)
:-)
It says more about the friend. It took a LOT of customization to get Eliza to do that. Nothing spiffy, just a lot of words for it to watch for and a variety of responses. When I was done with it, it didn't sound anything like the original psychiatrist version.
This particular friend was actually an annoying guy from my CS class who got my AIM name from somebody and kept bothering me. Instead of putting him on my blocklist, I gave him the Turing Treatment (TM)
Now he's been bothering me more - he's fascinated by my customized Eliza and he thinks I am really on to something big in the field of AI. Sheesh...
--
grappler
Vidi, Vici, Veni