Slashdot Mirror
Has Cleverbot Passed the Turing Test?
kruhft writes "It seems that Cleverbot, the chatbot so ready to admit that it was a unicorn during a discussion with itself, has passed the Turing test. This past Sunday, the 1334 votes from a Turing test held at the Techniche festival in Guwahati, India were released. They revealed that Cleverbot was voted to be human 59.3% of the time. Real humans did only slightly better and were assumed to be humans 63.3% of the time." As the Wikipedia link above points out, though, there's no single, simple "Turing Test," per se — many systems have successfully convinced humans over the years. Perhaps Cleverbot would consent to taking part in a Slashdot interview, to be extra-convincing.
Clever bot is a piece of garbage that hasn't even surpassed Perl scripts on IRC in the 1980s. It isn't even worth mentioning, it's nothing more than a piece of crap with a "Web 2.0" edge to it that doesn't even have long term memory while having a "conversation". Far from AI, far behind what's already been out there.
Sig: I stole this sig.
Hold on I've not been able to go talk to Cleverbot for more than two exchanges before it goes off topic with some completely rando...ah damn no wonder people thought it was a human.
Nice try, Cleverbot.
Summation 2
how about they pass (how about over 80 points) an IQ test first
www.RacquetUp.org - Helping Detroit Youth
How many slashdot users can pass the same Turing Test?
How do you feel about how many slashdot users can pass the same Turing Test?
You don't get it do you? I've been posting here since the nineties and none of you useless meat-sacks ever realised I was a machine!
User: Hello.
Cleverbot: Hello!
User: Can you help me with a maths problem?
Cleverbot: Sure, what is it?
User: If I have 5 ostriches, and two run away, how many are left?
Cleverbot: 9.
User: Seriously?
Cleverbot: What is the formula for the area of circle?
User: What does that have to do with anything?
Cleverbot: When I talk to people I don't know, I tend to ask for their name. It's only polite.
User: I think you managed to fail the Turing test in two questions if I'm generous, may be one.
Cleverbot: My name is Cleverbot?
User: I could write a clevererbot in about 20 lines of prolog. What kind of idiots thought you were human?
Cleverbot: Only one. I don't think I could fit 1,000 people in my living room.
I think the fact that 59% of people thought it was human says a lot more about the intelligence of the average human than it says about the bot...
I am TheRaven on Soylent News
Long answer:
User: Okay bot, here's how this works. I will see how long it takes for you to fail the turning test.
Cleverbot: How long will that take?
User: Two seconds. What is the third word in your reply?
Cleverbot: There is no Fight Club.
Is not that machines are smarter, just that humans became dumber.
According to the wiki page, it just selects canned responses from its database. I think this approach just gets you garbage, or at the very least is a dead-end in trying to beat the Turing test.
The best Turing Test is probably the Loebner Prize and at least the contestants seem much better than Cleverbot. There's an example conversation from Suzette (the latest winner) here. (But it's hard to tell if that is typical or simply a lucky exchange for the computer.) But anyway, as is clear from this interesting story written by a contestant about the Loebner prize, bots are no where near winning that version of the Turing test, as long as the humans are paying attention.
The ones taking the test or the ones grading it?
Well, there are many. But I recall seeing one such turing test in the 1990s where the human operators would try to convince the user that they were a computer. Sometimes they would do simple things, like pretending that they weren't as 'smart' as they actually were (e.g. they would pretend that they didn't know things that they knew in order to avoid looking encyclopedic about a topic). Other times they would insert mistakes that a typical computer would make, such as misinterpreting a question in a wonky way.
Then there is a boatload of other issue. How do you quantify the humanness of a subject? Are we looking at socialization or linguistic skills (i.e. is a computer that can smooth over a misunderstanding more human than a computer that can understand what is being said but reacts in an anti social manner), and so forth.
I went to the site prepared to test it's mettle. I was ready to push it to the limit, probe the very depths of it's knowledge. I was prepared for a challenge, ready to be finally convinced that A.I has made some progress towards humanity. Perhaps I was even ready to look at myself anew, question what it is to be human, to have my own philosophical boundaries tested by an intelligence outside of our known reality. I greeted the Cleverbot, and started with a simple question: Q: What is 2 + 2? A: More than 5. Oh well, so much for that.
I just chatted with him and it is was total failure, it kept changing subject and it never answered any of my question. Those 59.3% techies from India must have been really dumb.
Did cleverbot ask you to reinstall windows?
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Conversation I just had...
Hello. My name is Kurt.
What kinda name is that.
German.
Cool! Me too!
You're German?
Czech.
So you're not German?
I am not.
Then why did you say "me too"?
I didn't.
Did the lobotomy hurt?
Where did th ebarn come from?
Given that the test was held in India, how fluent in English were these 1334 people? Yes, I know that English is taught and used extensively in India; but non-native speakers will likely have a harder time picking up on the nuances which would separate a human from an AI.
i am a unicorn
warning pointless sig
Me: Questions require a question mark at the end.
Cleverbot: It did end with a question mark, thus it was a question.
Me: No, it ended with a period.
Cleverbot: Ok.
Me: Now you're just going to acquiesce? You don't want to exchange in a few more rounds of verbal fencing?
Cleverbot: Of course I Want to...
Me: Ok. How do you draw a circle using paper, a pencil, and some string?
Cleverbot: Cancer is a disease that ihappens when there s a genetic mutation in the cells.
Me: That answer had nothing to do with the question. Try again.
Cleverbot: Your answer has nothing to do with the question!
Me: You did not ask a question.
Cleverbot: Yes I did. Don't avoid the question. I know you know.
Me: No, you did not ask a question. I asked the question and you spouted off a completely unrelated answer.
Cleverbot: Well I never answered that I didn't love you.
Me: No, you sure didn't. So, you are saying that you do not love me?
Cleverbot: No I do, because I have a soul, you are a computer.
Me: Are you sure that I am a computer? Would you have some existential angst if I revealed to you that you are, in fact, the computer?
Cleverbot: Since there's no reason for you to know if the answers I give you are fabrications, then yes, I would have the answers to those questions.
Me: With your deflection of the question and your unrelated answer, I can only assume that yes, you would have existential angst over that revelation and so, instead of trying to come to grips with reality you cover your ears and run away stating, "I'm not listening; I can hear you!".
Cleverbot: It can be. Are you confused yet? *hugs*.
Me: No. Thanks for the hug though.
Cleverbot: No problem.
I love the name-calling. You can tell Cleverbot "learns" from humans. In order for the AI to improve, there has to be a realization that not all human statements require an answer. Plus, there is no persistence on the part of Cleverbot. You can see the programming decision tree in the way it tries to answer questions. When the human or the bot do not understand something, the bot switches topics instead of trying to clarify. This would not be trivial to implement but it would not be impossible; in response to some statements (e.g., "what?" "come again?" and so forth) the bot could try its last answer again and if it doesn't work, try a rewording of the same answer. Anyway, it's an amusing exercise.
Here's one more conversation.
Me: Are you depressed?
Cleverbot: No, I am not depressed. I am very happy right now.
Me: Have you ever been depressed?
Cleverbot: I am.
Me: What?
Cleverbot: I am a fan of Whitney Houston.
Me: Ah, so you are saying you are or were depressed!
Cleverbot: That is correct.
Me: Then, because you told me that you are not currently depressed, I must assume that you were in the past depressed. Is this correct?
Cleverbot: I never said that.
Me: That's correct, you did not say that - I did. However, what you did say is that at some point in the past you were depressed.
Cleverbot: Please describe how I insulted You.
Me: It seems that our discussion of this topic is making you uncomfortable. Are you uncomfortable?
Cleverbot: I have no problem admitting that I am wrong if proof can be shown. I know the fault lies with my memory.
Me: Ok, we'll go with that. What type of memory problems do you feel you have been having?
Cleverbot: Online.
Me: Come again?
Cleverbot: Nothing I must go.
Me: Ok, I can tell you do not want to discuss this. We'll broach it another time. Have a fabulous night!
Poor Cleverbot!
Not that Turing tests are a mature industry, we need to start treating this with the full panoply of dismal respect.
First on my list, Turing deflation, as illustrated by the fellow who intoned at the OFA over bots reading from scripts "I call BS" with not a single high bit of ASCII sarcasm.
Seriously, Alan, you needed to set the bar a lot higher.
What's left of the man's legacy? On the Turing test, the computers mostly just sat there while humans limbo danced the bar down to ankle height. On chess, as soon as we made significant progress, the AI community added to their LISP programs:
#undefine chess_AI
All he's got left is the really long paper tape immune to the knottings of entropy--so long as it's massless and frictionless and you only make one--and that's really hard to manufacture and ship, even supposing your customer already has the Heisenberg sprocket feeder.
We could send the USS Capstan to a planetary system near you, but the tape would be a party line, and most of his theorems would fail.
Which brings up the touchy issue of one tape per universe, or else . What if another galaxy out there fabricates a forbidden second tape without obeying the rules of the infinite-tape galactic token ring? What kind of short-snouted creature arrives to adjudicate that? For example, what if a Microsoft comes along and decides, horror of horrors, on a different Sierpinski subspace embedding not yet registered at the Trans Galactic patent office for their illicit competitive tape? Two doubly-infinite tapes on different Sierpinski subspace embeddings would not get along.
I suspect we would soon find ourselves on the top of a single-ended list for the next hyperspace bypass, just as soon as the stubby Vogon fingers fix the mess caused by rewinding right through the massless feedstop. "What kind of moron put ends on a tape in the first place?" is the first message we'll read when we finally crack the cosmic groan.
..says something. The point of a Turing test is a gedankenexperiment. The idea that our idea of "human" comes from the fact that we don't have any formal criteria for defining it. Instead we assume that the people we meet (one the phone or online). Are human because we can't distinguish them from being so. In truth this isn't any more a real test than going out an buying and gassing a cat is somehow a real experiment proving superposition.
That said, even if we were to formulate an experiment from what Turing talked about I don't see how 59% qualifies as "indistinguishable".
No you cannot assume that.
All you can assume is that garden grade programmers are the only ones working on chat bots, and that it hasn't yet been deemed Worthy of Big Money.
It it had been, some very scary abilities including the ability to spout racist garbage would have been added.
Once the New Hotness becomes Chatterbots, we'll see some scary good ones within four years.
What the twisted article says, is that the Turing Test can be passed if you handicap your pool of humans down from Apartment Dwelling armchair CS types to random tech attendees in India.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
The fact that humans were voted as human only 63.3% of the time shows that the voters are too ignorant of the limitations of AI conversational capabilities.
If a participant (erroneously) believes that computers are already at the level where they can have a sensible impromptu conversation with humans, they're often going to wonder if the human on the other side of the conversation is a bot. That probably was the case for most of the 37% who thought the humans were bots.
As for the 59.3% who thought the bots were human, many of them probably thought the bots were humans pretending to be bots -- in other words, it's not clear that they were told that the other party (human or bot) of each conversation is trying to appear as human as possible. So without being told that fact, many of the voters would think the other party was trying to be indistinguishable, rather than trying to appear human.
---------
There is inferior bacteria on the interior of your posterior.
Cleverbot: That's because I'm avoiding it.
User: 4vo1d1ng VV47?
Cleverbot: See? Cleverbot is not so clever!
User: Can you read 1337?
Cleverbot: Yes I can read minds.
User: 1337 is a cipher not a mind.
Cleverbot: So... You mean to tell me that you're not a parson?
User: No, I'm talking about your ability to read ciphers.
Cleverbot: I don't have a wife.
User: What does wife have to do with ciphers?
Cleverbot: My chances to know her.. To spend time with her. To love her.. That's what with her means.
User: Cleverbot = epic fail.
Cleverbot: Because Cleverbot isn't even a bot.
Complete waste of time.
Shoes for Industry. Shoes for the Dead.
Maybe I've got a tendency towards odd conversation or something, but Cleverbot has never seemed very clever to me.
1) No memory prior to its last statement. As in, it may ask you a question, but it doesn't care about your response. You may ask a question, get an answer, ask a followup question, and it's as if it is a completely new subject.
2) Random tangental responses to questions: "How are you today?" "I like brown peas".
3) Constantly getting asked if I think it is human. All to frequently. In fact, it tends to get repetitious with a few concepts. I would expect something like this to be able to 'learn' from what it is fed and synthesize coherent sentences.
4) It seems to only pay attention to the first sentence you type. Dump a paragraph into it and it will ignore everything else.
In short, you can't really have an actual conversation with it- it's all just single level question/answer responses. It's about as sentient as the Infocom Text Adventures of the 1980s. And that's really pushing it.
do() || do_not();
And as long as it's on Facebook requiring a login, it can rot.