Slashdot Mirror
Inside the 2012 Loebner Prize
An anonymous reader writes "Not a single judge was fooled by the chatbots in the 2012 Loebner Prize, which was won by the bot Chip Vivant. According to a journalist who was a human decoy in this year's Turing Test, interactions with the humans was a tad robotic while the bots went off on crazy tangents talking about being a cat and offering condolences for the death of a pet dragon."
Of course, all the *real* chatbots are too busy with their day job - posting spam to twitter and pumping out mass emails.
This is funny since Slashdot has been the main testing ground for chatbots. We have all had to read posts from them here, do you really think all Anonymous trolls are really people? BTW. most of my enemies list are chatbots Real people would not be nearly as stupid as these clowns.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
At first glance, I read it as "Inside the 2012 Lobster Pie".
Did they ask the bots what was the best smartphone? We all know it's a bot if they didn't answer the N900
I was promised a flying car. Where is my flying car?
Hello sir, have you heard of CleanMyPC...?
From a programming perspective, algo trading is simple math, not AI. The challenges lie in A) Processing the ungodly amount of data necessary in a short enough time period to be relevant and B) Coming up with a formula that accurately identifies good trades. There is literally no connection to AI, you don't want the program to "pick" anything, only to quickly identify predefined situations.
From a programming perspective, algo trading is simple math, not AI.
I don't think you appreciate just how complex trading software has become. The big players have moved WAY beyond "If stock X dips below Y price, then sell."
What political party do you join when you don't like Bible-thumpers *or* hippies?
They make financial instruments complex on purpose as well. Can't understand it? Can't tax it!
Then your system has to model trading these things... you either end up with fractal complexity, or dangerous assumptions...
I'm kinda conflicted - is this off-topic or on-topic?
Seriously, if a remote chat started talking to me like that, I'd say "Oh, hi Kim, I didn't know you were online".
From a programming perspective, algo trading is simple math, not AI. The challenges lie in A) Processing the ungodly amount of data necessary in a short enough time period to be relevant and B) Coming up with a formula that accurately identifies good trades. There is literally no connection to AI, you don't want the program to "pick" anything, only to quickly identify predefined situations.
What do you exactly think is the difference between current AI and "an algorithm that quickly identifies predefined situations"? Aside from the obvious example of natural language interaction that was hilariously pointed out in the OP. AI is nothing more than an (occasionally very complex) attempt at hacking English into a variety of numbers, and then performing math on them in interesting ways (what words are statistically likely to show up near what other words, and in what order, for example.) Algorithmic trading might have more specific activity at the low levels (like monitor 1000 different asset prices for a specific pattern) but that is hardly unlike an AI, its output is just handled differently.
A friend of mine, who long ago worked for Thinking Machines, explained the weakness, "It is all about maintaining state." A stateless AI is far easier than a stateful one. Once the machine has to retain state, the algorithms become logarithmically more complex. Therefore, the way to test a bot is to say something like, "Remember this phrase, 'pink elephant'. I'm going to ask you after we have talked a while.." Then have several exchanges and ask, "What was that animal I told you to remember?" Most humans (except Alzheimer patients) will have no trouble with it, but the machine will fail. It they add a piece of logic to catch obvious clues like this, then a slight mod such as "have you ever seen a pink elephant? . . . what animal was I talking about?" will usually defeat it.
Humans are actually very poor at remembering. Try to recall the color of the last Volkswagen you passed on the street. However, we have developed a natural ability to prioritize our memories based on context and our personal & social needs. We tend to remember most of what turns out to be relevant. Until AI develops a means to judge context, it will suffer the weakness of being out of touch with our reality.
I currently work at a financial company, and I agree. One of our algorithms for "buy or sell" currently relies on a few hundred data points for a few hundred securities. The algorithm boils down to "did this particular statistic recently rise or fall more than a certain amount based on its history?". Sure, the code is a little hairy in places, but not too bad with adequate commenting. There is nothing I'd consider AI, as the program doesn't attempt to learn. When the investment folks see something that should change, they tell me.
You do not have a moral or legal right to do absolutely anything you want.
Can someone please explain to me how to read the chat logs? I am confused as to the actual exchange that is going on. Which transcript is the Bot, which is the human and how am I to sync the two parts of the conversation up?
Fantasy remains a human right; we make in our measure and in our derivative mode... -- JRR Tolkien
I'm kinda conflicted - is this off-topic or on-topic?
I think you would be an excellent judge for next years competition.
You and the GP keep using the word "AI". I do not think it means what either of you think it means.
Algorithmic trading does indeed involve a great deal of classical machine learning, which is one area of focus within the very broad field of artificial intelligence. What you're ascribing to "AI" is the specific goal of artificial human intelligence, regularly granted the misnomer of general artificial intelligence. The example you gave would be of more interest to translators of unknown languages than to people trying to create an artificial brain.
This was one of the many misconceptions about intelligence that muddied AI efforts in the twentieth century, incidentally. Parsing and producing language isn't the end-all of everything like logicians once believed; other factors, like knowledge representation, planning, and creativity also must be considered. These have (for the most part) redefined the AI landscape as it's been introduced; KR and fuzzy logic led to expert systems in the 80s (which were pretty successful in some contexts but an absolute pain to build) and classical planning led to a wide variety of different manifestations like the intelligent agents of the 90s (which weren't so intelligent), automation of flight planning, and enemy AI in real-time strategy games. Computational creativity has yet to really have its time in the sun, being mostly displaced by a vastly-improved second round of more involved linguistic analysis called natural-language processing (culminating in things like IBM's Watson, although that involved a measure of creativity as well.)
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
/. has had users like this for years!
Mod me down, I shall become more off-topic than you could possibly imagine.
I'm happy for all the bots that got to compete this year, but I was a little unhappy on the preliminary round of this years competition compared to other years I entered. Only 4 entries can make it to the final round of the competition. There were 12 entries this year but 7 were disqualified due to contest management (Hugh Loebner) not having enough technical knowledge to get the entries working. Some well known bots based on ALICE AIML were disqualified, Cleverbot was disqualified, and my own Ultra Hal was disqualified ( http://www.zabaware.com/webhal ) Internet communication is prohibited so we all have to send the bots as self installing programs that can utilize the contests LPP protocol. My own bot is Linux based, which is a big hurdle for the preliminary round, but I sent it as a virtual box image to simplify it for contest management, but he didn't know how to deal with it.
But luckily there will be another competition this year as part of Alan Turing's 100 year centennial at Bletchley Park on June 23rd and recognized by the Olympics http://www.reading.ac.uk/news-and-events/releases/PR445524.aspx Some of the disqualified bots including my own will be competing there.
This is NOT about AI, this is a bunch of whankers whanking.
READ the chat logs, it is not a human trying to see if an AI can hold a realistic conversation but rather to see if it can trip a human/ai up. This is like trying to proof cats can't see color by poking its eyes out and then saying "AHA! See it can't see color". Or me proving you are a lousy at playing catch by shooting you in the head then mocking your mother for bearing such a clumsy child.
Hell, read the chat logs and try to tell who the humans are. It is sad to say that AI chat programs are still not that much more advanced then Lisa or whatever the name was but whats the point in trying to test if chat AI can respond to insane questioning? Congrats, you are now the winner of the bot best capable of holding a conversation with an insane person. WHOO!
Don't let these people near the flying car concept, their test track would include surface to air missles because you know, that is a good test of a flying car.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Meh, from what I see if you're one of the favoured companies when you screw up big time they'd roll back the trades for you. I bet even someone like me can make a lot of money if my trades got rolled back whenever I screwed up big. And if you screwed up really big you get a bailout and keep your bonus.
There was also that infamous case when humans outsmarted the algo but they got prosecuted, convicted and lost their profits: http://www.ft.com/cms/s/0/f9d1a74a-d6f3-11df-aaab-00144feabdc0.html
Why should they get prosecuted? Why should the algo users keep the money if it wins when they also get to keep the money when it loses? Sounds stupid and unfair to me.
While I am sure that your friend is mostly right, it ought to be easier for the bot to remember stuff than that.
These are programs, right? Just allocate some data. Without trying to pun Facebook, just keep a file on every person from the bot's perspective, so if it understood "remember this animal" at all, then it just sets "Judge1 Likes Pink Elephants".
I know "Devil is in the details" but I often feel I could design a chatbot that would never make certain kinds of mistakes. Getting totally lost, sure, that's the signature problem of AI, but "I am a cat", no.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
And I reply that a perfectly valid branch of AI is designing defensive routines against trick questions. I feel that is an area the contestants don't pay enough attention to.
You know, like the one in one of the logs "time flies like an arrow, fruit flies like a banana, which is the simile?" should kick right back to the judge with "what's a simile?" and follow it with "nah, I never liked that english class crap".
Or an even crazier example, something like "would Richard Stallman fit in a breadbox?" it should kick back with "Wait, what?"
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
IV surface modeller? Vol below, buy; vol in 90th percentile sell?
I've eaten hippo. Not a hippo, obviously.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Naw, why one or the other? Just do both. We could have the manpower is we "wanted to". You know, instead of artificially limiting the field to 3 man operations.
Say that 15 people in the 100 person team design the defensive routines, which are in some ways simpler because all trick questions have low legit semantic content. Plus speaking of humans it's what siblings and college students do to each other all the time.
The other 85 members of the team can go back to regular language processing.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine