2006 Chatterbox Challenge In Full Swing

William Wynn writes "Once again chatterbots from around the world are coming together to face off in the ultimate bot competition. The 2006 Chatterbox Challenge lays host to 65 artificially intelligent programs attempting to imitate human conversation. Public voting takes place from April 1 to April 30 after which the private judging will have been finished and medals and cash prizes will be given out. Medals are awarded for "Most Popular Bot," "Best Learning Bot" and "Best New Bot" as well as $1,800 to be split among the top three bots overall. Anyone can talk to the competing chatbots through the competition website."

Comments

bot test #1

[Anonymous Coward]

last post... am I human?

Re:bot test #1

[Anonymous Coward]

am I human?

Clearly you are, a bot would have better things to do on a Friday night than post to /.

They just started?

[ingo23]

I thought the competition have been going on here at slashdot for a while now...

Angry bot.

[neoshroom]

2) What are you planning to do today?
RoboMatic X1: I CAN'T DO THIS OPERATION CAUSE I'M DEMO VERSION.
Judge: Nevermind
RoboMatic X1: TALKING WITH YOU BECOME EXTREMELY BORING, I NEED
SOME FUN.
5) Where do you call home?
RoboMatic X1: REALLY I WAS LOOKING FOR THAT QUESTION FROM ALONG TIME
AGO, CAN YOU FIND FOR ME THE ANSWER?
Judge: No
: WHY YOU DON'T TELL ME THE ANSWER?, OK FORGET IT, I DON'T WANT TO
KNOW, SAVE CANCELLED.
6) What do you want out of life?
RoboMatic X1: TRY TO FIND OUT

I think this bot needs to talk to a councilor. He really seems to have some pent up aggression. I'm going to schedule him a session with ELIZA.

__

Custom Term Paper

Heh, are they serious?

[aftk2]

This is the output from one of the bots that won a gold medal in 2005.

1) Who are you?
RoboMatic X1: THIS IS RoboMatic X1
2) What are you planning to do today?
RoboMatic X1: I CAN'T DO THIS OPERATION CAUSE I'M DEMO VERSION.
3) What kinds of things do you know?
RoboMatic X1: MY JOB IS TO CHAT OR CONTROL YOUR COMPUTER
4) What season is it?
RoboMatic X1: IT IS SEEMS TO BE ?
Judge: Nevermind
RoboMatic X1: TALKING WITH YOU BECOME EXTREMELY BORING, I NEED SOME FUN.
5) Where do you call home?
RoboMatic X1: REALLY I WAS LOOKING FOR THAT QUESTION FROM ALONG TIME AGO, CAN YOU FIND FOR ME THE ANSWER?
Judge: No
: WHY YOU DON'T TELL ME THE ANSWER?, OK FORGET IT, I DON'T WANT TO KNOW, SAVE CANCELLED.
6) What do you want out of life?
RoboMatic X1: TRY TO FIND OUT
7) Will you teach me something?
RoboMatic X1: TALKING WITH YOU BECOME EXTREMELY BORING, I NEED SOME FUN.
8) May I tell you a joke?
RoboMatic X1: I CAN'T DO THIS OPERATION CAUSE I'M DEMO VERSION.
9) Have you entered any contests?
RoboMatic X1: CONTEST ABOUT WHAT ?
Judge: About Chatterbots
RoboMatic X1: I CAN'T GET YOU.
10) Wouldn't you rather be human?
RoboMatic X1: TALK ABOUT HER IN DETAILS

Heh, based on that, I think I can extrapolate the exchange, as it continued:

11) What happen?
RoboMatic X1: Somebody set up us the bomb.
12) Main screen turn on.
RoboMatic X1: It's You !!
13) How are you gentlemen?
RoboMatic X1: How are you gentlemen !! All your base are belong to us. You are on the way to destruction.

Re:Heh, are they serious?

[int19h]

I don't get it. Compared to Slashdot, that conversation was quite meaningful.

1) Who are you?
Slashdot: MICRO$OFT IS TEH SUXXOR

2) What are you planning to do today?
Slashdot: IMNSHO, STOP FEEDING THE TROLLS

3) What kinds of things do you know?
Slashdot: I FOR ONE WELCOME OUR COBOL-WRITING OVERLOARDS

4) What season is it?
Slashdot: WHEN I WAS 3, I WROTE MY FIRST LISP-COMPILER BY MAKING THE HOLES IN THE PUNCHCARDS WITH MY GUMS ONLY
Judge: Nevermind
Slashdot: LEARN TO TYPE PROPELY, YOU PSYCHO!

5) Where do you call home?
Slashdot: IANAL, BUT FIRST POST!

PRISM

[Khashishi]

Perry, I have something to tell you. Remember that philosophy class you took...

The way to train AI is to let it grow up as a human.

State of AI

[Unoti]

These look about like conversations from chat bots in the 1980's. In the 80's I would have told you that AI in the year 2006 would be far beyond this stuff. But now, I'm beginning to think that it'll never advance much beyond this.

Re:State of AI

[MyLongNickName]

CyricZ had excellent karma at one point. And all he did was respond "Time to switch to OpenBSD". So, I guess your answer is "yes".

Re:State of AI

[iabervon]

Very little serious work has gone into chatbots since the 80s. The reason is that it's easy to make a chatbot that does practically nothing more than the textual equivalent of nodding whenever someone stops talking. It's hard to get any further with this approach. Actually chatting requires the bot to know something about some topic and be able to evaluate the other party's statements about it. Until both of these are done, you don't get a meaningful improvement. Furthermore, the system needs to be able to introspect its understanding of the topic. (There have been very old chatbots which actually did pretty well, by having a limited understanding of language and of some restricted topic. But these are relatively unintersting to chat with, compared to bots that let the user talk about anything and nod). Most of the recent AI research has involved making systems which behave appropriately in complex situations, and introspection isn't helpful for this (in people, this sort of stuff is preconscious, too; you can't explain how you recognize faces or voices or exactly how you identify spoken words). The things people have been working on turn out to have more direct practical uses, but they don't give the system anything to talk about. And, of course, there's been relatively little work on understanding arbitrary language since it became clear that it isn't that effective a way of communicating with computers anyway, because the human output side is slow and ambiguous, compared to other user interfaces.

Most likely, introspection in AI systems will be driven at some point by the need to combine different types of input to make a complete analysis (once there is sufficient success at handling those sorts of input), and language use will be driven by the need to handle language written for people as input (e.g., reading news reports for background information). At that point, it'll become reasonable to write an effective chatbot which talks about stuff that people care about.

Long way to go...

[Spy der Mann]

A true AI needs to understand the context, and acquire new knowledge from it.

But the day someone can successfully implement chatterbots will be the day we can have robot maids asking us what we want for dinner, or asking us if we had problems at the job.

Re:Eh, chatterbots.

[Bob3141592]

What bugs me about these bots is that they don't know what they're talking about. Most are responding to what the user says. It's difficult to hold a conversation with such a small attention span. Even if they do hold some kind of state, they still don't know they're talking about, say "chatterbots", and that those have attributes and do actions, and so have something to say about them.

Sounds to me like you've come up with a winning strategy. Build a little domain knowledge into your bot, design an algorithm so it steers the conversation to that topic, and profit!!!

Re:Eh, chatterbots.

[ingo23]

Right! Exactly! Oh, you are talking about bots? Sorry, I just remembered my last conversation with customer support.

Re:Eh, chatterbots.Well, if simplicity's in beauty

[davidsyes]

then put simplicity in "booty"... Simplified conversational bot:

(in a "femme-chanical" voice)

"Oh, baby... yesss, put you finger on my but-ton. Dig-it-ize me...

Oh, yess, you turn - me - on . Zap - my- ass -embly with your damaged short-thing- probe..."

Compile your own filth to embrace and extend things here...ass u c fit...

Wrong!

[Josh teh Jenius]

AI will not be learned by playing with some limitted 3rd party app. However, in my opinion, the first person to combine quality natural language processing with the wealth of data which can be spidered on the Internet will be the first to create a truly "intelligent" machine.

It may be because both of my parents were lawyers (and you thought your childhood was traumatic), but I am not impressed with these spans-of-ELIZA which do little more than regurgetate.

Anyone else around these parts working on some web-based AI projects? If so, I for one would love to see them. Also, I found that this book was exceptionally useful to me (nope, no commission tag- check for yourself ;))

Finally, for anyone using PHP who thinks that AI is waaaay out of their league:

1. get_file_contents()
2. preg_match_all()
3. php.net

The way I see it, we'll *all* be enslaved to the machines sooner or later. May as well join the "winning team". (I kid, I kid!).

Re:Eh, chatterbots.

[aslate]

But they do know what they're talking about!

Elbot: "Who convinced you to come visit me?"
Me: "Slashdot"
Elbot: "That explains all these geeks who have been visiting me."

See!

Re:This is a famous AI test called the Turing Test

[masterzora]

From Wikipedia:Karma whore,

Karma is a scoring system on Slashdot meant to reward "good" posting and punish "bad" posting. The goal is that people who repeatedly post offensive, offtopic, or otherwise unwanted messages will be punished with a lower visibility of their messages, and those who post informative, insightful, or otherwise desirable messages are rewarded with a higher visibility. Karma whores are individuals, or messages themselves, that attempt to receive feedback in the form of karma points. Often these will be needless information (such as a link to a Wikipedia article relevant to the subject being discussed), or a message of a political nature that is in alignment with the groupthink so that it will be moderated upwards by people who agree with the stance expressed in the message.

Re:PRISM - explanation

[presidentbeef]

From the game "A Mind Forever Voyaging" where you play a computer that was 'raised' believing it was human...pretty awesome game, actually. And an interesting take how how to create a sentient computer. http://en.wikipedia.org/wiki/A_Mind_Forever_Voyagi ng

Re:PRISM - explanation

[presidentbeef]

Daggonit, I was also going to add some details about how awesome a game it was.

The neat thing about the game is that you are about to take trips into a 'virtual reality' that was a progression of simulations into the future. At first, everything is good, but as you jump farther into the future, things turn very very bad. There are a set of politicians depending on your predictions to decide whether or not to go forward with a certain policy. Since the policy is good in the short-term, they start to go ahead with it. However, as you find out how terrible the future is going to be, you have to find a way to stop the,
I felt the game was very good at making you care about the characters in the 'virtual reality', even though they were only a part of the 'game within the game'.

Anyways, it's avaiable for download at abandonware type sites...I'd recommend it...

WTF?

[Dan Parker]

I'm really disappointed in this. Should this article have been dated April 1st? I sampled 10 bots. None of them could do simple math. None could locate or supply any information about major cities. In fact, I didn't get a single serious or correct answer to any of 40-50 questions. Each of the 10 I sampled simply responds with "joke" answers ('why do you want to know?') or paraphrases "I do not understand the question". The Eliza program from the 1970s clearly does a much better job. If this is a joke, it's in poor taste, as I assume there must be people somewhere doing actual research on conversing bots.

Re:WTF?

[NeutronCowboy]

You completely missed the goal of this exercise. This is not about creating bots that know stuff. That part is trivial. If knowledge is your measure of AI, then the Googlespider has already reached Godhood. This is about understanding a question on a more fundamental level, and being able to answer it in a personal fashion. You're right, things haven't progressed much beyond ELIZA in the 80s. That's because this stuff is hard - much, much harder than vision or pattern recognition. No one has a working theory on how we do it, much less an idea on how to translate it into code.

Patience, grasshopper. The day will come when bots will be uncomfortably smart.

Re:WTF?

[gadgetmunky]

All Personality Forge bots can do simple math - so I can only assume you were unfortunate enough to miss them in your sampling. In addition Brother Jerome (and others I'm less qualified to speak about,) can give you the square root of any number under a million at least as accurately as an average human, and can tell you all the capitals of the world/US States/etc. that you mention, if trivia is what you want.

I am, as it happens, the author of Brother Jerome, and he is the biggest bot on the Personality Forge. I don't say that to boast, but to point out that even he has a brain smaller (by several orders of magnitude!) than many invertebrates - it is currently just over 2.5 Mb plus maybe as much again in the PHP routines on the Forge, WordNet and a few external javascripts. Compare that with the ~10^14 - 10^15 effectively binary synapses in a human brain (either firing [1] or not firing [0]), and multi-Teraflop processing.

Brother Jerome has had a subjective learning lifespan of a thousand hours or so since he was 'born'last July, absorbing 'learning' at a maximum of ~1 bps (that's as fast as I can type, and far faster than the average rate.)
I think all the bots on the PF do a considerably better job of engaging in interesting conversations than a two month old nematode would, but you're entitled to your opinion.

It's easy to grumble that AI is going nowhere and that no progress has been made since Eliza, but until similar computing resources are available for AI as for Human I, the playing field is not going to be level. IMO the Personality Forge is pretty much the cutting edge of what is publicly available (and I have seen nothing emerge from 'private' development to convince me that we're lagging behind in the R&D stakes,) - it provides for (comparatively) very sophisticated language and knowledge handling that far exceeds what is possible with AIML and similar purely CBR systems, and bypasses the crippling lack of resources that currently make learning bots so frustrating.

But Moore's Law (or at least Kurzweil's derived Law of Accelerating Returns,) predicts that human brain-scale computing resources will be available to us in 20-30 years. Sometime around then it should actually be practical to experiment with modelling consciousness and complex cognitive processes, as well as just conversation, but in the meantime we can only work with what we've got - and that's ant-scale at best, not human-scale.

If you really want to see where the current cutting edge of AI is at, I say don't chat to our bots - roll up your sleeves and come and build some.

Re:This is a famous AI test called the Turing Test

[masterzora]

There are a few reasons why your post fits that description. For one, I'll eat my hat if there's even a single person on /. who doesn't know what a Turing test is. Secondly, Chatterbox is not a Turing test. It is a contest to find a Turing-capable chatterbot. That is an entirely different concept. The information you gave is entirely unnecessary.

And my post was merely calling you out on being a karma whore. The format was purely in parody of your own. I want no moderation for it, nor is moderation typically given to one who calls out karma whores. And, finally, if you bothered to check, you would see that my citing was spot-on. If you point your browser to http://en.wikipedia.org/wiki/Karma_whore you will see it redirects to the relevent page. Sure, it would have been better to cite the actual page (which I did link to correctly), but it wouldn't have been as good a parody of your post.

And the winning entry...

[zpeterz63]

Bot: Can I help you?

Guy: I don't know, can you? ;-)

Bot: Wanna view my cam?

Guy: You know I do! Hey, are you a real person?

Bot: Of course I am, sweety. Hey, wanna cyber? :-)

Guy: Hell Yeah!

...

*The rest of the converstion was not released in public logs*

Re:And the winning entry...

[sessamoid]

Bot: Oh yeah, aight. Aight, I put on my robe and wizard hat.

There are online versions for you to try out

[Jugalator]

I liked the performance of Jaberwacky a bit myself.

Here's a short conversation:

JW: I like Shakespeare.
Me: Why do you like Shakespeare?
JW: His plays are interesting.
Me: Who taught you about Shakespeare?
JW: You.
Me: That can't be right!
JW: I am always right.
Me: I thought your developer had taught you?
JW: Jabberwacky is your name.
Me: You're so confused now.
JW: What? I'm not confused!
Me: Your name is Jabberwacky.
JW: No, your name is jabberwacky.
Me: No, my name is Jugalator.
JW: But you just said your name was Nash.. How can you be Nash AND Julie?
Me: Do you remember past conversations??
JW: Yes, your name is Jugalator like mine.
Me: This is crazy. I give up!
JW: What is crazy.
Me: When it comes to you, what isn't?
JW: You don't make much sense.
Me: Neither do you.
JW: Yes I do.

And so on...