Slashdot Mirror
Inside the 2012 Loebner Prize
An anonymous reader writes "Not a single judge was fooled by the chatbots in the 2012 Loebner Prize, which was won by the bot Chip Vivant. According to a journalist who was a human decoy in this year's Turing Test, interactions with the humans was a tad robotic while the bots went off on crazy tangents talking about being a cat and offering condolences for the death of a pet dragon."
Of course, all the *real* chatbots are too busy with their day job - posting spam to twitter and pumping out mass emails.
At first glance, I read it as "Inside the 2012 Lobster Pie".
Did they ask the bots what was the best smartphone? We all know it's a bot if they didn't answer the N900
I was promised a flying car. Where is my flying car?
I'm kinda conflicted - is this off-topic or on-topic?
A friend of mine, who long ago worked for Thinking Machines, explained the weakness, "It is all about maintaining state." A stateless AI is far easier than a stateful one. Once the machine has to retain state, the algorithms become logarithmically more complex. Therefore, the way to test a bot is to say something like, "Remember this phrase, 'pink elephant'. I'm going to ask you after we have talked a while.." Then have several exchanges and ask, "What was that animal I told you to remember?" Most humans (except Alzheimer patients) will have no trouble with it, but the machine will fail. It they add a piece of logic to catch obvious clues like this, then a slight mod such as "have you ever seen a pink elephant? . . . what animal was I talking about?" will usually defeat it.
Humans are actually very poor at remembering. Try to recall the color of the last Volkswagen you passed on the street. However, we have developed a natural ability to prioritize our memories based on context and our personal & social needs. We tend to remember most of what turns out to be relevant. Until AI develops a means to judge context, it will suffer the weakness of being out of touch with our reality.
I'm happy for all the bots that got to compete this year, but I was a little unhappy on the preliminary round of this years competition compared to other years I entered. Only 4 entries can make it to the final round of the competition. There were 12 entries this year but 7 were disqualified due to contest management (Hugh Loebner) not having enough technical knowledge to get the entries working. Some well known bots based on ALICE AIML were disqualified, Cleverbot was disqualified, and my own Ultra Hal was disqualified ( http://www.zabaware.com/webhal ) Internet communication is prohibited so we all have to send the bots as self installing programs that can utilize the contests LPP protocol. My own bot is Linux based, which is a big hurdle for the preliminary round, but I sent it as a virtual box image to simplify it for contest management, but he didn't know how to deal with it.
But luckily there will be another competition this year as part of Alan Turing's 100 year centennial at Bletchley Park on June 23rd and recognized by the Olympics http://www.reading.ac.uk/news-and-events/releases/PR445524.aspx Some of the disqualified bots including my own will be competing there.