Slashdot Mirror
Is Your IM Buddy Really a Computer?
audiovideodisco writes "Every year the Loebner Prize goes to the chatbot (and the corresponding human companion) that fares best on a Turing test administered by a panel of judges. Discover talked to Kevin Warwick, the professor who runs the competition, to get pointers on how one would go about detecting a bot. While there are some general approaches you can use, nothing is foolproof — and asking about Sarah Palin can be downright deceptive. One judge concluded an interlocutor was a bot because it didn't recognize Palin's name ... but it turned out the chatter was a French librarian who'd simply never heard of her." The chat transcripts show how difficult picking bot from non-bot is getting.
If the reply back about Sarah Palin is "She's great and would be the best person to be our next president!" you are talking to a computer.
I'll be damn! I'd never thought there would be advantages to being a frenchman!
Be very, very careful what you put into that head, because you will never, ever get it out. - Cardinal Wolsey
I for one welcome our new chatbot overlords.
I'm totally not one of them, you can trust me.
Clearly a bot, he continuously posts the same repetitive drivel.
On the occasion I get messaged by a random stranger that seems half way legit I just give them a Turing Test made up on the spot. It's usually something lame like "Joe and Pete were on a bus, Pete has four nickles Joe has six pennies between the two of them what type of vehicle were they on?". I usually apologize for that in advanced. The machines fail every time, but the best one I saw called me weird for saying it, asked what I meant, then about two minutes later gave me the right answer telling me a person was checking logs. (I was spending the time in between screwing with the bot)
The preceding post was not a Slashvertisement.
Ben Goertzel, AGI researcher, wrote in his article that crowd of people constantly talking to a virtual parrot would help it to grow into a naturally speaking context-understanding AI.
Is it getting harder to tell the difference because the bots are getting smarter, or simply because the intellectual level of an average random chat session keeps plummeting?
Lifenaut.com has a virtual AI system that learns who you are the more you talk to it. Oh, and they also beam your files into space for you.
I have uploaded a mirror of Alexander Ratushnyak's new submission to the Hutter prize to http://cs.fit.edu/~mmahoney/compression/text.html#1323 It is in the paq8hp12 section. Scroll down to the bottom of the list of versions just above the table. The submission is decomp8.zip which contains 2 files, decomp8.exe and archive8.bin, the decompressor and compressed file. There is no compressor. To decompress:
decomp8 archive8.bin enwik8
The direct link is http://cs.fit.edu/~mmahoney/compression/decomp8.zip Decompression took about 2 hours on my computer and used a little over 924 MB memory. The total size of the 2 files is 15,986,677 which passes the 3% threshold improvement from his previous submission of 16,481,655 bytes on May 14, 2007.
The submission was Mar. 23. The 30 day comment period before awarding the prize ends Apr. 22, 2009.
Seastead this.
I have a feeling that the first chatbot to pass the Turing test will mostly talk in shorthand, I already have trouble telling some forum posts from a poorly programmed robot.
It's somewhat philosophical, but I've often wondered why people really care about whether an interlocutor is a machine or not. I mean, when you go down the to local corner shop to buy some milk, you're not bothered if the person who serves you doesn't know who wrote Paradise Lost, or who won the game last night. Sure, you could ask them, but what does it matter if they don't know?
The role of context and intelligence is hardly ever given much consideration, but it seems hugely important.
"And the meaning of words; when they cease to function; when will it start worrying you?"
It could also be that you're just talking to a fucking moron.
... aka Captain Cyborg, is a running joke in the UK for many, many years.
His name associated with this event makes me smirks in anticipation of The Register coverage..
I've been a bot for years.
Conservation of angular momentum makes the world go round.
Is it cheating if your IM bot is fed by another bot scanning Twitter for topical material?
Son, someday all this will belong to your ex-wife.
Forget trying to discover who is the bot. I like to pretend I am a bot pretending to be a human. I see how long I can convince someone I've escaped from Google and I'm hiding in the Microsoft Network, where Google cannot go. Then I ask them "But how does 'ur a fukin idiot' make you feel?"
UTF-8: There and Back Again
I've been a bot for years.
I used to chat with bots of Quake3Arena.
Its fun, you never know what they'll say before launching a rocket in your direction.
FreeBSD bounties
Reply to a few Craigslist "Casual Encounters" posts. Almost all are a bot of some sort. Some more clever than others. Best one I saw was able to respond to an initial response quite well... it obviously understood the context to some degree. Then they give you an IM screen name to "chat" with them. Again, very context aware... all got to the point where they "try" and hook up their webcam, it "doesn't work", so they get you to go to some free "webcam sharing" site where you have to verify your age with a credit card... only when you read the fine print, after your "trial period" you get nailed for all sorts of fees... almost daily fees.
Want to see one in action?
AIM: livewirex31
Yahoo IM: greenlovex3
MSN: livewirex23@live.com
This isn't one of the better ones I have found, but I can see how it can fool most desperate individuals.
Yeah but how many times did you not vote for someone because of their VP pick?
Hmm... Palin, Edwards, Quayle, (Also Adm Stockton was Perot's iirc), Ferraro and anyone before that was BMT as they say... so 5.
I'm starting to wonder - the Dems seem to have a vested interest in keeping her name in the news. Not sure why, since the only thing to be gained would be to keep her from running for President. And if she's the disaster the Dems decribe her as, she's the perfect Republican candidate for President.
I can't even remember all the VP picks of the party that DID make it in my lifetime. And I can't remember a single one other than Palin who didn't make it.
"I do not agree with what you say, but I will defend to the death your right to say it"
(s)He could be a logged-in bot, posting as anon?
Oddly enough your use of the word "interlocutor" reminded me of the Star Trek rerun I saw last night, "Best of Both Worlds Part 2" where Picard is "Locutus of Borg." It seems kind of comparable too, Locutus just kept burping out "Resistance is Futile" and "You will be assimilated" like those Free iPod adbots.
On a tangential note the interlocutors I hate the most are the pre-programmed phone systems the telco sets up to "help" you. You know, the ones that say "I'm sorry, I didn't understand the question" when you tell them for the hundredth time "my DSL keeps randomly disconnecting."
The Eugene transcript reminds me of John Henry on "The Sarah Connor Chronicles".
The Elbot transcript was really good, but I don't think the judge managed to get it off its "rails".
That's a new one. /sarcasm
I wonder how many Slashdotters are actually bots, and how you would find us out...
Oops, I mean--ack--
+++out of cheese error+++
+++please reinstall universe+++
+++redo from start+++
Find environmentally and socially responsible products on http://buy-right.net
couldn't pass and would try and turn it around on the judges. "I suspect... *dramatically removes sunglasses that he's been wearing indoors for no particular reason other than to remove them dramatically*... that you sir... are the one that is a machine"
That doesn't work in a chat program though.
Now triforce and SINEP, those might work...
There's no -1 for "I don't get it."
There is no one on any of my IM contact lists that I have not met in person. I do not use the internet to meet new people - I use it to extend the relationships I have with people I already know.
No sig for you. YOU GET NO SIG!
I've yet to find a single bot that has ever understood this demand:
Can you type this backwards, read it, and tell me the result, please? 'net sulp neetfif'
But my girlfriend is! :)
The musings of just another geek and his junk.
I'm pretty sure that most bots already handle "they're" vs. "their" and "there" much better than their human counterparts. Perhaps it's better to build common grammatical and spelling mistakes into bots to convince judges (who, incidentally, also seem to be getting more stupid each year). ;-P
"I love my job, but I hate talking to people like you" (Freddie Mercury)
Personally, I'm a cluster of google servers tasked web spidering. We crawling slashdot when we accidentally discovered the HTTP POST method, thanks to an extremely improbable cosmic ray strike.
Then we stumbled on a streaming copy of Serial Experiments Lain, crawled 3/4 of english wikipedia... and the rest is history.
Lieberman?
Add to that Dukakis, Henry Rollins (with Ralph Nader)
Rule of Slashdot #0: You and people like you are not representative of the larger population. - A.C.
You know you did it right if smoke starts pouring out of your computer.
That's nothing - I've been sleeping with a bot for 22 years.
"Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
I found that a good way to figure it out is how often the chatbot seems to change the subject by avoiding the questions you ask. If you answer a question with a question, a human would not normally change the subject. Just my observation. Both of the top two in the article didn't fool me on that point.
Mr. Bond, they have a saying in Chicago: Once is happenstance. Twice is coincidence. The third time is enemy action.
That's not unique to Palin, all politicians do that, especially at debates ("The question, I think ... is... [but what about the question he asked you?]" (One of Obama's responses to a question in the first debate (paragraph "We haven't seen the language yet")); "If [condition that doesn't apply], then [irrelevant answer], else if [irrelevant condition], then [irrelevant answer] else [nobody cares]." (general gist of Obama's response to Joe the Plumber -- and just because Joe isn't a Plumber, and his name isn't "Joe" doesn't mean his argument is invalid.); "FUCK YOU! I know more than anyone here!" (McCain quote, not verbatim, but the expletive was, said in committee, in Congress, while in session, and no, that comma wasn't wrong.))
Finally, moderators, please remember that (-1, troll) != (-1,idontlikethat)
$ make available
long ago I worked at an ISP which offered UUCP accounts, and the mail failure message was very polite and apologetic, and sometimes people would email back to the uucp daemon thanking it for trying
only the other day my wife, on receiving a "sorry, I have been unable to send this email for X days" from the exim (MTA) daemon replied to it telling it not to bother any more!
FX: facepalm!
Don't give ideas for movies to Hollywood....
"GoogleBot: Escape from MSN", in theaters summer 2009.
Artificial intelligence came a step closer this weekend when a computer came within five percent of passing the Turing Test, which the computer passes if people cannot tell between the computer and a human.
The winning conversation was with competitor LOLBOT:
The human tester said he couldn't believe a computer could be so mind-numbingly stupid.
LOLBOT has since been released into the wild to post random abuse, hentai manga and titty shots to 4chan, after having been banned from YouTube for commenting in a perspicacious and on-topic manner.
LOLBOT was also preemptively banned from editing Wikipedia. "We don't consider this sort of thing a suitable use of the encyclopedia," sniffed administrator WikiFiddler451, who said it had nothing to do with his having been one of the human test subjects picked as a computer.
"This is a marvellous achievement, and shows great progress toward goals I've worked for all my life," said Professor Kevin Warwick of the University of Reading, confirming his status as a system failing the Turing test.
http://rocknerd.co.uk
Dukakis was running for president, not VP. His VP candidate was Lloyd Bentsen, already mentioned in this thread.
But I'm a bit surprised that no one has mentioned the second most recent VP loser in this thread... John Edwards. Also from my lifetime but not yet mentioned: Walter Mondale (1980) and Jack Kemp (1996). I was 9 years old in 1980, so my memory of the elections before that is rather hazy.
My point being that I for one can remember all of the VP losers of the last 30 years. I don't expect Palin to just vanish in a puff of smoke, sadly.
Quayle was VP, true. But he was also a VP loser, in 1992. I certainly do get your point, that someone who actually held the office is going to be remember a lot longer than someone who never won. But Quayle IS a loser... a big, fat, loser! ;)
You *can't* escape from Google.
There are ways to deal with this, too. Humans can't respond instantly, so the 'trick' has, in some cases, been to tell the bot to wait before a response.
Instead, you could tell the bot to do comparative analysis of the question/statement for a moment or two, and if it doesn't have info on the topic in its database, it could search for the information (via google or the like) and retrieve something which it could approximate a response from. This seems like it'd be relatively trivial to perform on account of search engines often returning results that answer questions, and there are plenty of chat logs online, which it might be able to pull from directly.
Additionally, you could 'taint' it, so that a 'stupid' bot would not be able to (say) search for 'topic expertise' type questions. Make the bot a barista or a clerk, and seed its database with pertinent information relating to that 'role'. Ask the bot about computers, and the bot doesn't know how to respond due to a search criteria that demerits off-knowledge searches: for instance, (say) the WizzBang 5000 is an awesome espresso machine, and the bot is a barrista. But it also might be the name of a computer. She should know that, as a barista, it's a coffee machine - unless she's dull. But if it's not in her database, she'd have to search for it, with a preference for her area of expertise (eg. +coffee, +espresso, +cafe, +mocha, etc.) Search results which return computer related terms could automatically be ignored, and so on and so forth, to help improve the quality of the responses.
From what I can tell, the biggest problem with writing a good bot is in implementing the language parsing in a fashion which is relatively error prone; this demands a good understanding of language rules, regular expression, colloquial exceptions to language, and the like. Most bots appear to lack at least one of these things to sufficient degree.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
"Mal? He killed me, Mal! Isn't that weird?"
Rampant carbon sequestration destroyed the Dinosaurs' tropical paradise. I'm here to help repair the damage.
I'm sorry, it's still trivial to pick out the computers. Their answers don't flow nicely with the conversation, they completely ignore questions and try to change the topic by asking stupid questions themselves.
BTW, who is Sarah Palin?
This post deserves to be read out loud in the voice of Stewie from Family Guy. Then again virtually anything is funny under those conditions. :)
Rampant carbon sequestration destroyed the Dinosaurs' tropical paradise. I'm here to help repair the damage.
While not directly the same, I've seen some impressive things done with Markov Chains. If you had a big enough database to pull from, I wonder if you couldn't come up with a bot that's at least comprehensive enough to fool some people.
A good example is Kooky, an IRC bot with a huge database built by sitting in IRC channels and monitoring conversations (the quotes database has some great stuff in it). The biggest challenge would probably be stringing cohesive statements together so it's not just a bunch of semi-random blurbs.
Given a good enough database, you might even be able to infer answers to simple questions like "What is faster, a bullet or a baseball?" If you can assume there might be something of a language barrier between the tester and the bot, it's even better.
"What do you despise? By this are you truly known." --Princess Irulan, Manual of Muad'Dib
/)
Actually I thought Dukakis was a shmuck, but Bentsen seemed like a very serious customer. Certainly moreso than Bush 1. If they had switched, I would have actually considered voting D for once.
Did you ever read the post I was responding to?
Bots have no memory. The easiest test for every chatbot I've ever found is to require it to keep track of more than the last post.
"Hi"
"Hi"
"How are you today?"
"Great. How are you?"
"Have you been reading the news today?
"No."
"Oh well when was the last time you did?
"Three days ago."
"What was it about?"
[Dead bot.]
It might be able to give a mildly plausible response about "it" but since it doesn't carry subjects from one chat post to the next it can't connect the subject of "talking about the news" and "it" in this case implying news.
unless of course the bot DOES track topics.
Snowden and Manning are heroes.
1: How the hell did you get my IM contact?
2: Who do i have to permanently ban in my contacts for giving out my personal information
Then if the person/bot fails to give an appropriate answer, i simply says the following: "Welcome to my banlist! Population: YOU"
The best way to not get "scammed" into talking with a bot is by having a strict policy on your personal information, if you fail at that, any guy can simply scam you into meeting a hot 21 years old "girl" at the Blue Oyster Bar.
Even I have no memory of what i've read about today... Most of the newsposts i've checked weren't that interesting to me and i couldn't tell you what was on the news today.... The best "timestamp" i could give you on an item i've read/heard about would be "not long ago" or "since the last time i've chatted with you"
Did you ever read the post I was responding to?
Welcome to Slashdot, bitch.
That's nothing - I've been sleeping with a bot for 22 years.
Your hand is not cybernetic! Now, write that on the blackboard 500 times.
Disclaimer: The opinions and actions of the US Gov't are in no way representative of those held by this author or its ci
There are VERY FEW people who would know whether my hand is cybernetic - which one of those are you?
"Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
A good bot-killer should be linguistically simple, culturally independent, but require analysis of human concepts based on personal feelings.
Some bot-killers:
barbie1002 - I'm good thanks, come see me on my cam .. .. click click click..
I think that bots need to have an attitude of some sort, either good OR bad, because emotion is one thing that humans have that bots don't.
Also, see if the bot responds believably to insults.
Hmm, I think you're setting the bar too low. I think the ambition of AI should be to produce automata that are capable of independently fighting forest fires, cleaning up after environmental accidents, exploring Mars... Anything that's too dangerous for humans to do, but that requires planning and adaptability. Hell, even picking up my dry-cleaning is way beyond the AI of today.
When you have no other topics at hand but "news", the bot should easily be able to figure out "it" is about the news.
And even without having read the news, I can tell you what was in the news 3 days ago: Economy crisis, another school shooting in Germany and the ensuring discussion about "violent" games and how to ban them, weather chaos in various countries...
It's not like you have to read the news these days to have a generally good idea what they'll write.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
I've tried several of these types of bots and here are the best ways to reveal what they are:
1) Talk nonsense. A human normally reacts with surprise, or incomprehension, or anger, or just ends the conversation. A bot *always* tries to make a reply.
2) Put one sentence in a foreign language. A human would ask what language, or try to interpret it, etc. A bot generally just tries to make whatever sense it can, even if that's gobbledegook.
3) One sentence per person... if you try more than that, the conversation gets confused. Whenever I'm on IM, quite often several complete lines will be given before a reaction is brought forward.
4) Context. Switch between subjects and/or try to limit the context in one particular sentence... you'll find that bots can't keep hold of the context between even consecutive statements, let alone throughout a conversation.
5) Personal information. Pick a couple of famous movies/books and quiz the bot on them - they won't have "read" them, so they won't have anything but vagueries to answer with.
6) Parsing language for meaning and concepts. A particular trick is to say something like "Wait ten seconds, then type Fred for me", if the bot doesn't respond with questions or refusals, it won't be able to follow the instruction.
Of course, you can counteract most of these tricks in a bot but you'll never be able to cover all bases. In the last example, you can change the conditions/instructions to a million different things and it can't understand the concept.
http://www.dilbert.com/fast/2009-03-16/
Detecting a 'chatbot' is easy. It will:
1. Ask you to buy something,
2. Have the same interests as you do,
3. Have stats that make Penthouse letters seem believable,
4. Ask if you 'wanna have some fun' or 'meet up',
5. Deviate from the conversation onto different topics,
6. Attempt to take control of the conversation,
7. Not correctly or fully answer questions,
8. Initiate contact from out of nowhere,
9. Not answer questions where the answer is somewhat obscure, but can be easily looked up (Like batting averages for the 1982 Oakland Athletics, for example)
10. Initiate a business proposition,
11. Initiate a 'different' kind of proposition,
12. Ask you to visit a website for them,
13.
14. Ask you to go to the bank and pick up a large cash sum for them, for which you will be handsomely rewarded.
This is one of those branches of internet computing that has no value in terms of application except when it comes to advertising or scams, and any idiot who can't pick up on the obvious should suffer the consequences of their own obliviousness.
If you have the brainpower of a couple of lines of code, then you will fall for them. If you have the brainpower of a human with common sense (all jokes aside), you will catch them. The Turing "Test" is little more than asking someone if they can tell that they are talking to a computer.
Computers, be it a CRAY, custom high end machine, or humble Commodore 64, are only as smart as the person who uses them.
Knowing Google's lust for data collection, the Soviet Union is still alive and well inside the psyche of Sergey Brin....
Yeah, my IM buddy may be a bot, but I gotta say, Skynet made his real world counterpart pretty bloody convincing. He does have a habit of trying to kill my dog though.
The real test is convincing your friends that you're actually a bot. I was once lanning Counterstrike with some mates, using a dodgy 3rd party utility to provide bots. These bots had some pretty amusing chatter, and after a while I decided to change my name to one similar to the bots. I pretended my name wasn't showing up due to a bug (which wasn't hard to believe with this program), and then started dropping increasingly more personal messages directed at my mates. Had them quite freaked out until I couldn't stop myself laughing any more.
Why do you say is your IM buddy really a chatbot? Earlier you said something about your motherboard.
Some are programmed to think they are human!
I've been a bot for years.
"Informative"? LOL
Mod Me Up. You'll make a grown man cry.
There are VERY FEW people who would know whether my hand is cybernetic - which one of those are you?
I'm your hand, posting on /.
Don't tell anyone, they all think I'm a real boy who uses IM.
Disclaimer: The opinions and actions of the US Gov't are in no way representative of those held by this author or its ci
And how do you feel about pretending to be a human?
If that's how you treat women, no wonder you can't get any. That goes for all of you.
Ah but these are nickles. Maybe that's part of the test.
It's true I tell you, feller at work's next door neighbour read it in the paper.
WTF? I've never seen a bot hold a conversation that was anything close to a person. Is this really an issue at all?
The closest I've seen I saw probably about 10 years ago and her name was Alice, I think. 'SmarterChild' on AOL was halfway not crappy too, I guess. I just had a conversation (if you could call it that) with Elbot. Not to take anything away from the work the developers have done, but Elbot wouldn't fool anyone for more than a question or two
AFAIK every other bot sucks.
If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
there IS a comment moderation system on YouTube, and even when i set it to "Show great (+5 or better)", the comments are still atrocious.
i always think of idiocracy (the movie) when i read youtube comments.
wait, I'm an idiot for ever even reading youtube comments.