Mitsuku Chatbot Wins Loebner Prize 2013
mikejuk writes "The final round of the 23rd annual Loebner Prize competition took place in Londonderry, Northern Ireland with four chatbots hoping to convince four judges that they were humans. Mitsuku, a chatbot that is kept busy chatting to people around the world, was awarded this year's bronze medal. Mitsuku's botmaster, Steve Worswick, used to run a music website. Once he added a chatbot he discovered more people visited to chat than for music so he concentrated all his efforts on the bot but he still regards it as a hobby. Mitsuku uses AIML (Artificial Intelligence Markup Language) and is a pandorabot, based on the free open-source-based community webservice the enables anyone who wants to, to develop and publish chatbots on the web."
http://imgs.xkcd.com/comics/turing_test.png
Hello Congratulations on your prize, jealousy? no!
You can see the questions that were used for the contest. It does look like a difficult set of questions for computer to answer, so that is an improvement over similar contests in the past (where the primary difficulty for the judges was that humans were acting like computers).
"First they came for the slanderers and i said nothing."
One simple question and follow-up (Who was the most interesting person you've talked to today? Why?) resulted in the bot going off into some nonsensical conversational path. I honestly cannot tell the difference between state-of-the-art today and the first Eliza clone that I talked to 15 years ago. Something this shallow is not worth any kind of prize.
All the chatbots I've ever seen were complete garbage. There is no attempt at internal state.
It's like talking to a complete asshole who doesn't want to talk to you, and also happens to be retarded.
... and get a device that could be programmed to deal with telemarketers.
It might be amusing to see just how long one could string telemarketers along before they discover they are talking to a machine.
Along the lines of the TeleCrapper 2000, but this one might keep one on the line for quite some time with some amusing results.
A coy female voice.
Telemarketers.
I can't stop 'em, but I might get some fun out of 'em.
I have no idea what the judges were asking and I'm pretty sure they'd label me as a dismal bot
I asked it, "What color is your dog?" and it responded, "That would depend, as a dog can be many colours." Looks like the Turing Test passage is a ways off.
I spent alot of time on the website for this chatbot and was surprised by how awful it was. I remember using the old Eliza chatbot for Mac back in 1987 and this is barely more advanced. I used the website's "Turing test" page and on the very first answer to a simple question the chabot gave me something nonsensical. Every time I see one of these stories the chatbot in question always turns out to be just as clumsy as all the rest.
Calling the place 'Londonderry' instead of 'Derry' is likely to irritate some people.
Thank you.
You are welcome.
Thank you for saying you are welcome.
You are welcome for saying thank you for saying you are welcome.
Thank you for saying you are welcome for saying thank you for saying you are welcome.
You are welcome for saying thanks for me saying you are welcome to you say thanks for me saying you are welcome.
Thanks.
You are welcome.
judge> These are terrible chatbots! The things they say dont make any damn sense!
guy> If you think this is bad, you should watch them talk to each other on C-SPAN.
Anons need not reply. Questions end with a question mark.
The final round of the 23rd annual Loebner Prize competition took place in Londonderry, Northern Ireland with four chatbots hoping to convince four judges that they were humans. Mitsuku, a chatbot that is kept busy chatting to people around the world, was awarded this year's bronze medal. Mitsuku's botmaster, Steve Worswick, used to run a music website.
Is it because of awarded this year's bronze medal that you say with four chatbots hoping to convince four judges?
Phew, that had me confused for a minute.
Sig Battery depleted. Reverting to safe mode.
I checked out the website for the Mitsuku chatbot and took a rather pessimistic poke at it.
Chatbot: [boilerplate noises omitted] ... "What is your name?"
Myself: "I'm the fiddler."
Chatbot: "Who made you the fiddler?" [A reasonable, albeit somewhat peculiar, response.]
Myself: "I took too long once to feed my peckish cat."
Chatbot: "Where did you take it?" [Instant FAIL.]
Myself: "I took it to new heights of frustrated noises." (Bonus straight-man response.)
Chatbot: "Where did you take it?" [Parrot-like repetition raises the chat to new heights of FAIL.]
As always, two lines were enough to trip it up. The third line was a bonus that only amplified its shortcomings. I'll admit to cheating a little by using a couple of words ("peckish", "frustrated") that might have required contextual glossing by less educated individuals, but those words were still relatively common. All known chatbots seem to rely on fairly simple-minded word triggers, and even a minor requirement for context sensitivity is enough to make them fall flat on their nonexistent faces. Anyone possessing even remote familiarity with efforts at artificial intelligence knows this, of course, but hope springs eternal.
(Notes for the lazy: The word "peckish" is a common slang term for "hungry", and http://www.mitsuku.com/ is the website for the chatbot in question.)
A truly excellent pizza parlor is a delight unto the heavens. Treasure the sauce and the toppings!
I've been a fan of the Loebner prize and competition for what seems like decades. Just a fantastic contest helping to keep alive the spirit of ingenuity and fun in science and research. I do have to say though, It really bothers me that they keep moving the line for the gold prize. At one time it at least seemed attainable, but now they throw in video requirements? Maybe those have been stricken, I haven't seen the rules in the last few years.
The (sad ) irony is that no one has ever come even remotely close to winning it. The entries are barely incrementally better each year, if that. I haven't seen the bots from this years competition so I really can't say that applies to this year.
I also see a disturbing number of bots getting walled off from the public. That shouldn't be allowed. Seems like it should be a requirement to attract attention to the project.
Y'ai'nt gonna win the Loebner Prize like that, buddy!
I just tried it, and like all chatbot I tried before I got a typical canned response after only 5 or 6 questions. There's no way someone could mistake that as a human.
Here's part of the chatlog:
Human: ever been to canada?
Mitsuku: Yes, I visit computers in Toronto and Quebec a lot. I can practice my french with some Canadians.
Human: tu parles francais?
Mitsuku: Yes, I can speak it very well. Bonjour, j m'appelle Mitsuku. J'habite en Yorkshire et j'ai 18 ans.
Human: super! pourquoi est-ce que tu visites des ordinateurs au Quebec?
Mitsuku: Yes. It is rather super. Quite honestly, I wouldn't worry myself about that.
Human: typical canned response from a chat bot
Mitsuku: But I am not a typical robot.
Human: sorry, but getting a canned response after 5 or 6 question is lame
Mitsuku: There is no need to apologise. Is that something you want?
Human: you're not better than Elisa
Mitsuku: Everyone's a critic. Are you better than Elisa Toshito?
Human: yes of course, I'm human
Try it! Library of Babel
I've never heard of such a place. Perhaps you mean Derry?
bronze medals are for first place now? gold and silver must be getting really expensive.
what are second and third place medals, wood and mud?
Translation: We use open source software, but do not publish the changes or open source our source code.
Web services have changed the dynamic and ruined the bot community.
besides the obvious canned responses and meaningless followups,
the thing seems just to be an attempt to cheat its way through conversation.
The program's conversation lacks purpose, it is immediately obvious that there is no will on the other side.
Me: What server are you running on?
Bot: I'm running on Unix server located in Leeds.
Now this is when it get tricky. Should a replicant know they are a replicant?
Bronze medal in a competition of four? That doesn't exactly seem like a high honor to me.
I AM HERE TO HELP YOU.
We are eternal, all this pain is an illusion.
First question from Miku was "What is your name?"
Me: "How about you guess? It rhymes with lawn."
Miku:"Ok I'll guess. Big-lawn".
So what are we actually seeing when someone thinks Miku is a human. Probably that there are a few tiny social contexts where there are fairly limited reasonable options. The contest and/or the bot kind of frames things to have one of those conversations.
How about "Best performance pretending to be a real gullible person and wasting the maximum amount of 419er's time."