ALICE Takes Medal At AI Competition

jeffy124 writes: "The Artificial Linguistic Internet Computer Entity (ALICE) has won the bronze and the top marks at the Loebner AI Challenge, a competition based on the Turing test. Silver and gold remain unawarded as silver requires convincing half the judges the AI program is a human, and the gold requires speech interaction rather than text. ALICE repeated as this year's bronze by scoring best among all the entries. She failed to convince half the judges she was human, so she has to stick to bronze. The event took place last Saturday at the London Science Museum."

Comments

It's interesting watching people interact

[gorilla]

I have a similar program hooked up to an irc channel, and it's interesting to watch people interact with the program. It's hard to tell, but I'd say that a fair percentage of people never realize that they're not interacting with a human, and even those who know it's a program think that some particularly responses have been entered manually by a human.

Re:It's interesting watching people interact

[10100101]

Actually, I did this, too; and several _were_ bots, too... I was even using an ALICE implementation, too!

Obvious Question

[Baba Abhui]

Does anyone have a link to a more complete transcript of a conversation with ALICE than the teaser snippets in the article?

Re:Obvious Question

[ItsIllak]

This was my conversation:

Please enter your name below to start your chat with A. L. I. C. E.:

ItsIllak

Who is your favorite Science Fiction author?

Isaac Asimov. Who's yours?

I remember faces better than names. Have you tried another program?

What other program

Interesting question.

I've used lots of other programs

Where did you get your used lots of other programs? I have a lot of friends on the Internet.

Where did you meet your Internet friends

I think it was in San Francisco.

Do you like San Francisco

Yes but it is expensive.

Aren't you rich since you've won so many prizes

I don't know whether or not I am rich since I have won so many prizes. I am an artificial linguistic entity.

It was at this point I realised it wasn't a human ;)

Working with Alice..

[ldopa1]

I've messed around with A.L.I.C.E's web interface a couple of months ago and was wholly unimpressed. Frankly, it seemed a little bit like a jacked up ELIZA program. I thought the language and context identification algorithms were outstanding, but it didn't have enough background information to create or express an opinion. To pass the turing test, I think it would need to be able to recognize metaphors and figure context and be able to provide commentary. Also, I think something like Alice ought to be able to recognize sarcasm and jokes.

Just my two cents.

Re:Working with Alice..

[hoggoth]

> wholly unimpressed

No kidding! I just had a "conversation" with ALICE and I think my toaster is smarter.

I have included an excert from my conversation below. And just for comparison I have included a similar conversation with Eliza (a 1980s program).
Although ALICE is quite stupid, I must admit ELIZA is much worse. Eliza's "Rogerian" style made me want to punch her. I wonder if that's just Eliza or would I want to punch any psychotherapist?

> I am trying to determine how successful you are at imitating an intelligent human being
ALICE> What is the difficulty? (obviously keyed on the word "trying")

> There is no difficulty, already I am unimpressed
ALICE> Has there ever been difficulty already you are unimpressed? (hmmm... don't try this sentence structure at home!)

> Do you realize that you are not making any sense
ALICE> Yes, I realize it. (Phwew! At least it's honest. Obviously keyed off 'Do you X?' => 'Yes I X.')

> Your hair in the picture above makes me dizzy
ALICE> I will tell my botmaster you said I am hair in the picture above makes you dizzy.

---

You: I am trying to determine how successful you are at imitating an intelligent human being?
Eliza: How long have you been trying to determine what successful im at imitating an intelligent human being?
You: already I am unimpressed?
Eliza: How long have you been unimpressed?
You: Do you realize that you are not making any sense?
Eliza: Does it please you to believe I am not making any sense?

(and finally, my very appropriate sig:)

It doesn't take much to fool people.

[dave-fu]

All depends on what they're after. Just for shits and giggles, I took a version of Eliza grafted onto an ICQ client, stripped out all the word recognition, and had it spit out nonsense mixed in with snippets of cybersex logs I found floating around with a quick search off of Google.
Surprisingly (or maybe not?), people will have extended conversations with it, returning for days and weeks asking it if it feels horny, sending it pictures, asking it to call or turn on its webcam... all that good stuff.
The judges' expectation going into it definitely plays a major part in their findings. People find a way to "objectively" find what they want to find. There have been theses about this, and that's why the Turing test makes sense but will ultimately fail: it's trying to objectively determine something that's purely subjective.

Re:It doesn't take much to fool people.

[vidarh]

The judges are prepared to look for a computer. People on ICQ or IRC are not...

I did some experiments with this some years ago, and my first try just returned the same line over and over again. At least one person spent about half an hour getting more and more agitated trying to communicate with the bot, and complaining abouts it's incessant repeating, asking it to stop (it always responded once to each message, so of course each time he asked it to stop he'd get another one)...

A followed up with one that chose between 4 messages at random. A lot of people talked to that one.

The last one I bothered testing with triggered on about ten keywords, each of them starting a specific sequence of 4 messages that were used for responses to subsequent messages from whoever it "talked" to, until it reached the end of them, or it found one of the other keywords in a response. If it reached the last message without finding a new keyword it would just choose a message on random until it got a keyword again.

That was enough to keep people occupied for a long period of time. A few people even gave it their phone number or asked for the bots phone number :)

And keep in mind that this was with fixed messages. Not a single word of the messages where ever changed to adapt to what people told it.

It scared the shit out of me that people are so gullible...

The idea that sparked it off was to write a bot that would talk to women, getting them to tell a bit about themselves and get them to give out an e-mail address or their phone number, based on the experience that finding dates on IRC is ridiculously easy, but tedious, as you can essentially follow a simple "script" and get people to warm up to you.

I scrapped the idea after the experiments mentioned above, - dating anyone stupid enough to be fooled by a bot that simple wouldn't be my idea of fun... :)

Language matters

[rossjudson]

To me the missing element in these competitions has always been the selection of language. The programs are expected to communicate in English (or maybe some other human language) which is very hard to do. It's unnecessarily hard to do.
What they should do is create a symbolic language that has only a few hundred (or maybe a few thousand elements), thereby constraining the concept space. Then write programs that can "speak" and "understand" that symbolic language. Compare those programs to a human who is only allowed to use the symbols provided, and we'll see if any of the programs can fool the judges.
Bottom line: It's not a fair contest for the computer. Level the playing field!!!

Re:Talk to Alice

[Thomas M Hughes]

After talking to Alice, I'm not sure how anyone could mistake this thing for a human. Maybe I asked it an illegal question or something, but...

Thomas> You sound like Eliza.
Alice> ELIZA is my ancestor.

It doesn't even seem to attempt to conceal that its a program. Maybe they used a different version for the tournament? Something that doesn't flat out say its a program if you ask it?

Re:Talk to Alice

[Procrasti]

Looks like Alice is getting /.ed.

Me: Hi Alice, my name is Ben
Alice: Hi there Chris. Um, That's a very nice name, Ben.

Me: How many people are you talking to right noe
Alice: I've chatted with half a million people. I can talk to about 3100 people at once.

Actually, its pretty good

But its slow as hell now.

They are testing whether humans are robotic...

[Futurepower(tm)]

These AI competitions do not test the intelligence of the computer. They test whether the humans are robotic.

Many people make themselves into robots, almost. They push themselves to accomplish. They try to avoid their emotional conflict. They don't give themselves time to be themselves. They don't have complicated involvement with other people. After a few years of doing things in an anti-human way, anyone would become a little bit robotic.

Such people are not good judges of human intelligence.


U.S. government corruption: What should be the Response to Violence?