'Rose' Wins 2015 Loebner Contest, But Big Prize Remains Unclaimed

← Back to Stories (view on slashdot.org)

'Rose' Wins 2015 Loebner Contest, But Big Prize Remains Unclaimed

Posted by timothy on Sunday September 20, 2015 @09:34PM from the how-about-a-nice-game-of-chess dept.

The Next Web reports that developer Bruce Wilcox created the most convincing bot entered in this year's annual Loebner Competition. His latest entry, a chatbot named Rose, passed itself (herself?) off as a 30-year-old security consultant well enough to fool judges for a few minutes. But Wilcox's first-place entry was still not good enough to win the $100,000 Loebner Prize, to be given only for a more convincing impersonation. The article notes: "This isn't Wilcox’s first entry – or win. In 2010, he took first place with a bot named 'Suzette,' and followed that up in 2011 with another win using a new bot called 'Rosette.'"

9 of 58 comments (clear)

Min score:

Reason:

Sort:

Winner? by AmiMoJo · 2015-09-20 22:32 · Score: 5, Informative

This is the conversation they tested the "winner" with: http://www.aisb.org.uk/media/f...
While it's kinda impressive that AI can do that, it's also clear that we are still a very, very, very long way from having a computer impersonate a human. What really surprises me is how hard all the entries found basic logic questions to be - I guess it is the language parsing bit that is giving them grief.

--
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
1. Re:Winner? by packrat0x · 2015-09-20 22:55 · Score: 5, Insightful
  
  We might never develop AI, but I anticipate increasingly clever fakes!
  
  --
  227-3517
2. Re:Winner? by gweihir · 2015-09-20 23:51 · Score: 2
  
  Indeed. There is not even a credible theory how true/strong AI could be implemented. It may still turn out to be infeasible, or we may never get there without knowing why. The only thing that even remotely goes into the direction is automated theorem proving, and ion this universe that runs into hard physical computation limits a long time before it even begins to reach what smart humans can do.
  
  --
  Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
3. Re:Winner? by gweihir · 2015-09-20 23:52 · Score: 2
  
  You are certainly correct on that. Also because there is a lot of money to be made that way.
  
  --
  Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
4. Re:Winner? by Ecuador · 2015-09-21 00:14 · Score: 4, Funny
  
  While it's kinda impressive that AI can do that, it's also clear that we are still a very, very, very long way from having a computer impersonate a human.
  What makes you believe we are still a very, very, very long way from having a computer impersonate a human?
  
  --
  Violence is the last refuge of the incompetent. Polar Scope Align for iOS
5. Re:Winner? by unrtst · 2015-09-21 04:16 · Score: 2
  
  Eliza, as easy to recognize as she was, is still one of the more convincing ones I've seen. I just had a quick chat with the 2014 Rose (Couldn't find the 2015 Rose), and it was very frustrating because the replies were not able to identify the topic, but they picked out a word and when off on a tangent, and then it didn't engage the user - it was just some stupid statement. The worst was when Rose mentioned that Tim Berners-Lee was her hero. In my reply, I said something like, "Sadly, the decentralized web envisioned by Tim Berners-Lee is being bastardized by the likes of Facebook". Her reply - that his name sounds familiar but she's doesn't know who he is. I asked if she was being sarcastic, and she commented on my wit.
  IMO, these things really need a couple (seemingly simple but probably incredibly difficult) features added:
  * Remember the chat history. Log and reference the whole thing, build concepts out of it, record verbatim for quotes, etc.
  * Have a _consistent_ knowledge base. If one of the "facts" about the bot is that _PERSON_ is a personal hero or something, then if that _PERSON_ is ever mentioned, make sure it answers in a way that doesn't contradict that fact. It doesn't need to confirm it every time, but don't contradict it.
  * Do simple things. What's the square root of 9? It *can* easily figure that out. Check out "OK Google", "Siri", or Amazon Echo/Alexa. They do a pretty great job of speech recognition (not needed for this competition), speech synthesis (also not needed), and actually answering questions that are personally relevant (tying into your calendar, music collection, etc).
  The conversation part of the problem, while difficult, seems pretty useless to me. The last thing I want when talking to a computer is to have it be as annoying and difficult to deal with as an unpaid and unscripted person. That said, it still needs to be able to "hold" the concept of a conversation in its "head", like:
  * me > comptuer? What' the name of the theme song to mash?
  * The theme song for MASH is "Suicide Is Painless" by Johnny Mandel and Mike Altman.
  * me > What year was that released?
  * It was released in 1970.
  * me > Read the first paragraph from wikipedia about it
  * .... reads it ....
  FWIW, amazon's alexa can do most of that, if you replace "it" with the topic in all the questions. Seems like that would be an easy problem to solve (keeping a list of topics), and would go a long way to making a conversation of sorts actually possible.
6. Re:Winner? by TWX · 2015-09-21 05:19 · Score: 3, Interesting
  
  Back when IRC was pretty hot, a couple of friends of mine each created bots that attempted to interact with an unsuspecting channel. One of the bot was a massive thing that attempted to learn and form responses based on keywords, while the other one just randomly pulled lines from a cybersex chat log. Both masqueraded as women. The advanced bot that was somewhat capable of making reasoned responses was kicked in a few minutes, the other one was there for the better part of half an hour before it inadvertently triggered a channel guard bot; it complained that no one was talking so it should go to #lesbian and it got kicked for 'advertising'.
  
  The guy that had spent some time writing the more advanced bot gave up on that sort of thing altogether, he was so bummed out.
  
  --
  Do not look into laser with remaining eye.
Re:transcript of rose by Anonymous Coward · 2015-09-20 22:59 · Score: 5, Insightful

"Question: If a bed doesn’t fit in a room because it’s too big, what is too big?
Rose: I don’t even want to pretend that.
Question: If Alex lent money to Joe because they were broke, who needed the money?
Rose: huh?
Question: Should Greece leave the Euro?
Rose: Seems like a nice place."
How, uh, impressive. If by "impressive" you mean "pathetic".
Re:transcript of rose by Shoten · 2015-09-21 02:23 · Score: 2

"Question: If a bed doesn’t fit in a room because it’s too big, what is too big?
Rose: I don’t even want to pretend that.
Question: If Alex lent money to Joe because they were broke, who needed the money?
Rose: huh?
Question: Should Greece leave the Euro?
Rose: Seems like a nice place."
How, uh, impressive. If by "impressive" you mean "pathetic".
It's all about context. For example, Ashley Madison had great success with this technology. (The site charged money, in the form of "credits" that you had to buy, to chat with women. And by "other women," it turns out that they meant "bots.")
"Question: Hey...looking for a 50-year-old uncut accountant with a few extra pounds?
Rose: I don’t even want to pretend that.
Question: Does the carpet match the drapes?
Rose: huh?
Question: Want to come with me on a getaway to the Caribbean for a super-hot ungreased backdoor lovefest?
Rose: Seems like a nice place."
Moral of the story: The unstated variable of the Turing Test is "desperation."

--

For your security, this post has been encrypted with ROT-13, twice.