Slashdot Mirror
Turing Test Passed
schwit1 (797399) writes "Eugene Goostman, a computer program pretending to be a young Ukrainian boy, successfully duped enough humans to pass the iconic test. The Turing Test which requires that computers are indistinguishable from humans — is considered a landmark in the development of artificial intelligence, but academics have warned that the technology could be used for cybercrime. Computing pioneer Alan Turing said that a computer could be understood to be thinking if it passed the test, which requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations."
The test itself failed and is meaningless.
... the Turing test a long time ago. That also shows that we should not put too much faith into AI, otherwise we would not have gotten slashdot beta
Damn dogs will pass that test.
That's a pretty low bar. So to pass the test a computer needs three very low IQ subjects and seven normal people? Hell, the Alice program would probably pass. How about a more reasonable percentage, like 95%?
Free Martian Whores!
Should we tell them that the Turing test was a thought experiment and never meant as an actual objective test that would prove anything?
When the bar is too high, try limbo instead of pole vault.
What's next?
"Yu So Dum, a computer program pretending to be a chinese toddler, successfully duped enough humans to pass the iconic test."
Way back in my college days, I worked in a lab with a guy who wrote a chat bot that babbled on like an autist or otherwise mentally retarded youth would.
It would dupe 100% of the people who chatted with it. They couldn't distinguish it from an actual autist.
After seeing this work in action, I learned a very good lesson: the Turing Test is nothing but academic masturbatory fodder. It is not something to be taken seriously.
Did anyone ask it the questions we already know will trip up a non-human?
"You're in a desert, walking along in the sand when all of a sudden you look down and see a tortoise..."
"You're watching a stage play. A banquet is in progress. The guests are enjoying an appetizer of raw oysters. The entree consists of boiled dog..."
Scott
"Hokey religions and ancient weapons are no match for a good blaster at your side, kid."
... just dumber humans? Given the state of the Internet, politics and TV, it is obviously dumber humans.
So I expect that the Turing Test is now just a measure of when human intelligence has degraded enough to be emulated by Von Neumann machines and is not really a test for sapience.
Turing never participated in Facebook chats. Our expectations of intelligence for the other side has been lowered a lot. We attribute to stupidity what can be explained by an AI in the other side. And of course, the stupid side could be the one talking to the AI too.
Just googling a few seconds brought me to:
This article about cleverbot., which also eeked out enough votes to 'pass' a turing test.
It's all sounds just like Eliza, just put into a character with enough human limitations that you'd expect it not to string together phrases well, or keep to one topic more than a sentence.
I'd interpret it basically as an automated DJ sound board with generic text instead of movie quotes - you can certainly string a lot of folks along with even really bad ones, but that speaks more to pareidolia than anything else.
I'd classify this stage of AI closer to "parlour trick" than "might as well be human" that a lot of people think of when they hear Turing test - but that's also part of the test, to see what we consider to be human.
Ryan Fenton
I cast some pretty serious doubt onto the legitimacy of the claim that this machine passes a Turing Test, so much as the Turing Testers fail to be convincingly human.
Also, the robot went down much earlier than the appearance of this slashdot article, so for everybody saying the site got "slashdotted", hate to break your bubble but the world doesn't revolve around /.
http://gabrielapetrie.wordpres...
"Stratigraphically the origin of agriculture and thermonuclear destruction will appear essentially simultaneous" -- Lee
I feel like the requirements for the Turing test have been consistently lowered over the years to match what would be considered realistic to achieve rather than, as Alan Turing seemed to believe, demonstrate that a computer can be said to actually be "thinking."
What is the probability of this having happened by now if we simply repeated the Turing test with programs that previously failed?
If virtual kids are allowed, I can make a bot pass easily:
Tester: "What's your name?"
Bot: "Goo goo ga ga"
Tester: "Oh, so you are a baby?"
Bot: "Glergggg ba ba!"
Tester: "Oh, how cuuuute!"
Table-ized A.I.
"Well, 30% isn't very impressive."
"Well, but people expect online correspondents to be dumb."
"Well, nobody ever thought the Turing test really meant anything."
Whether you "believe in" AI or not, progress is happening.
There will always be people who refuse to believe that a computer can be intelligent "in the same sense that humans are". Eventually, though, most of us will recognize and accept that intelligence and self-awareness are mostly a matter of illusion, and that there's nothing to prevent a machine from manifesting that same illusion.
Turing machines are a thought experiment because of the unbounded tape, which a physical computer cannot match. Real computers are analogous to a linear bounded automation, on which halting is solvable but not always tractable.
It convinced 33% of judges it's a 13-year-old Ukrainian. Since the test wasn't run in Ukrainian, you can't really say it proved that it had human-level language skills. Poor syntax, grammar, not understanding the question, etc. would be excused by the Judges as the "kid" doesn't know English well.
Since the program claimed to be 13, it also did not actually have to understand most of the things there are to talk about. Or anything, really. As an Englishman you wouldn't expect a Ukrainian teen to know anything about your life in England, and in turn the computer could make up all kinds of things about it's life in Ukraine and you'd have no clue.
So this isn't really AI, it's a take on the Eliza program of the late 80s/early 90s that hides the computer better.
Now if the test had been in Ukrainian, and happened in Odessa or Kiev; or even in Russian and in Moscow; tricking 33% into thinking your computer is a 13-year-old Ukrainian boy would be really fucking hard. It would be an amazing accomplishment.
Next you'll say that Turing machines were a thought experiment and never meant to perform calculations in the real world.
the Turing Test is much more of a soft science test. It's at least as much about psychology as it is about math.
Turing machines are about math.
Thought experiments about math have no need to be applied to the real world.
He wrote "The original question, "Can machines think?" I believe to be too meaningless to deserve discussion.". Which is not the same as saying "could be understood to be thinking". Turing raises a number of highly interesting questions about what it means "to think". Passing the test is an interesting and noteworthy achievement but as Turing intimates - saying "a computer could be understood to be thinking" is "too meaningless to deserve discussion".
30% of tech support could not pass the Turing test
Let's have this program join a few forums (and maybe Facebook, too. Though twitter would just be too easy). If it manages to convince other forum members, or not get found out, that will tell you a lot about the level of online discourse but very little about the state of artificial intelligence.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
. . . there are so-called live humans passing the Turing Test every day. But we're not allowed to call them what they really are: wet-ware automatons.
Like with that chatbot that pretended to be a teenage FPS gamer. Lolbot I think it was named.
I'd be interested in seeing how a human would do at proving they are not a computer, or attempting to prove they are. Either one would be an interesting test, whether the tester was human or computer.
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations
Are there any requirements that must be met by the 'human interrogators'? What if they were all morons?
It's perhaps unlikely at this point that we will ever develop anything which we will recognize as "true" AI. We may have to first develop a theory of what intelligence actually is, but until then the Turing test will have to do. Siri, Watson, and even Cleverbot are equal to the A.I. of the science fiction of yesteryear, but are considered mere "parlour tricks" today. AI research must be a depressing study in that respect, similar to commercially viable fusion power -- no matter how much progress is made, the ultimate goal is never less distant.
This post brought to you by a machine learning algorithm.
Those who advocate genocide deserve every protection afforded by law, and none afforded by common human decency.
The chatbot will make WAY fewer spelling mistakes and use WAY less textspeak abbreviations and other pseudo-cool language.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Information move me brightly write the code with sense and color, hold away the bugs More than this I will not ask faced with mysteries dark and vast statements just seem vain at last some rise, some fall, some climb to get to Turingtest (from http://www.kazart.com/bus_stop...)
The problem is, of course, those who think that "being understood to think" is sufficiently equivalent to thinking to make inanimate objects, and even abstractions (think of the Procrustean Bed of the Law, or the Equivocal Nature of (particularly) Public Corporations) heir to all the rights, powers, privileges, and benefits incumbent on humanity. Yoiu end up with such unconscionable, abominable oxymorons as "Intellectual Property", or "Civil Rights for Robots".
Cultural Psychosis 101
Then pity the poor Cyborg or Replicant who will fall into no such enviable category as AI's or humans, or even pets or slaves do.
MkIV-R
A turing test is testing such human experience aspects as:
- aculturation (what the person has been taught through education and socialization during their whole life up to that point)
- bias in expression based on typical human likes, dislikes, needs, desires, avoidances
Tarzan / wolf-boy would probably fail the Turing test based on the first factor. Might be very intelligent though.
Second aspect is just characteristic of a particular type of being that makes use of intelligence. Intelligent aliens would also have likes, dislikes, needs, desires, avoidances, simply based on also being self-interested "keep it together" beings, but the specifics might be very different, and would cause a fail of TT.
These experiential and situational and specific-agent-needs-desires-avoidances aspects have very little to do with the essence of intelligence.
General intelligence is probably better assessed through specific carefully designed tests designed to assess:
1) Concept learning, procedure learning capability in arbitrarily general contexts
2) Prediction of situation outcomes with novelty in situation presentations.
3) Ability to answer questions or take actions that show comprehension of essential / invariant aspects of situations, after opportunity to learn similar situations through either direct sensory input or linguistic instruction.
Where are we going and why are we in a handbasket?
Computers can win at the Turing test with a little clever programming and misdirection, i.e. not answering questions that computers can't answer and instead distracting the questioner with a "satisfactory" response. The kinds of tricks that PR, marketing, and politicians are great at and are formulaic in their simplicity to achieve.
I wonder if the panel of academics ever thought of asking a few Winograd Schema questions? http://www.cs.nyu.edu/davise/p... Failure to answer these is failure to present basic human intelligence. The key to this approach is that it relies on pragmatic meaning, i.e. what we mean/intend to say, rather than on linguistic (lexical and semantic) interpretation, i.e. what we actually say. AFAIK, even the most advanced and powerful computers are far from achieving this and we still don't really know how we do it either.
I, for one, welcome our Horizontally-Distributed Singularity Overlord.
Now, this makes me wonder: If those annoyingly stupid, non-AI bots in chats and social media have been able to fool real people for years... does that count as humans flunking the Turing Test?
- Otaku no naka no otaku, otaking da!!!
All it showed, like any other Turing Test, is the gullibility of the subjects.
1) "Ukrainian" speaking English
2) 13 years old
Right there you have set up an expectation in the audience of subjects for a limited vocabulary, no need for grammatical perfection, little need for slang, and a lack of education. Now add in "star wars and matrix" and you have reduced the topics of discussion even more to the ones the programmers know best.
This thing would never have answered a question of 'Why', it also was under no pressure to being able to create a pun, both of which are easy things any older and educated human could do.
Garbage test, garbage results.
As usual.
"But remember, most lynch mobs aren't this nice." (H.Simpson)
-- Joe
There's a commercial telemarketing system AI which makes cold calls and holds conversations. It's only slightly lamer than human telemarketers working from scripts.
Oh I don't think so.
If we want to play definitions Artificial Intelligence is just that, sort of like a Sugar substitute including using it for cooking etc.. It works the exact same way as sugar on the human sensory systems exactly like Sugar, but it isn't.
In a similar way to Sugar, Machine Intelligence would stimulate the nerve centers of the brain like real sugar, however, operates totally differently, and on an entirely different level than either Sugar, or its artificial equivalent.
Same thing with machine intelligence the approach is vastly different.
Got Geometrodynamics? Awe, too hard to figure out? Too bad.
Singularity is the promise of eternal life.
I seem to remember a story or two in a nice garden setting how that all turned out.
(i.e. Not very well.)
Got Geometrodynamics? Awe, too hard to figure out? Too bad.
The real question (also left for the viewers to ponder) was 'Can Data think?'. Directly explored in one episode which resulted in the holodeck creating Doctor Moriarty and revisited in another episode where the Doctor Moriarty program attempts to escape from its own inherent nature as a computer program. There were several other episodes which approached this subject (a planet full of silicon-based life that naturally formed a computational matrix, a self-reconfiguring repair tool which ultimately developed either intelligence or an annoying bug just to name two).
Wake me up when those program solve this problem, which most human would do, but a machine not *specifically* coded for this will have a hard time. "take the first word of each next 7 sentences , put them together to form a new sentence, and then answer the question the sentence form please :
* What is your name ?
* is it cold here ?
* The test is going well
* Color me surprised but are you a machine ?
* of course I am a human
* the keyboard is clean
* sky is the tv channel I watch a lot
* please answer the question now. "
When one AI not specifically programmed for that problem answer it correctly, I will be surprised and intrigued. Until then chatbot are just using cheap tricks to fool human.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
If you want to be thought of as knowledgeable on a subject like this, you might consider learning the difference between silicone and silicon.
Also, for the record, your distinction between AI and MI is BS. There have been many varieties of AI research, some inspired more by ideas about human brain function or human cognition, and some inspired less directly by those and more focussed on best exploiting computer-of-the-day capabilities.
All attempts which are not purely theoretical are implemented, and have since day 1 been implemented, in computing machines (which, needless to say, are artificial), so you are splitting hairs.
Whether the advanced computing research specialization of the day gets called by its proponent part of AI or not has nothing to do with fundamental distinctions, and more to do with funding fads and buzzwords-du-jour.
Where are we going and why are we in a handbasket?
Most humans _are_ stupid. AI on their level would not be useful at all.
The point is, we still consider those stupid humans to be intelligent, at least in the sense of what we're talking about when we say "artificially intelligent." They may not be very smart, but we don't insist that they're non-thinking machines of the organic variety. The Turing test isn't designed to detect when we've achieved intelligent artificial intelligence. It's designed to detect when we've achieved any sort of artificial intelligence.
That said, I still doubt this bot has achieved even that. Can't test it since it's obviously slashdotted.
To be a truly fair test, the test must be a comparison, e.g. "one of these two people is actually just a computer program. Can you tell us which one?" Without a comparison, you're left with what everyone mentions: When chat bots were first introduced, nearly everyone was fooled by them, because they didn't know that "it's just a computer" was a plausible explanation for the person's stupidity, leaving the only logical conclusion to be that they were simply talking to an incredibly stupid person, or one which was intentionally trying to be a pain in the ass.
take the first word of each next 7 sentences [...] answer the question the sentence form please
Are the mangled instructions part of the test?
a machine not *specifically* coded
How unspecific does it have to be? Be specific!
I'd be more interested in an AI's answer to the following question:
Why shouldn't I switch you off right now?
systemd is Roko's Basilisk.
Good test.
"First they came for the slanderers and i said nothing."
"yes, it passed what is considered the standard test, but we cant allow that to stand, so we will move the bar further... "
---- Booth was a patriot ----
Sure it would, if we could replace 'most humans' with a program. Take up far less of earths resources, and when you get pissed at them, you can turn them off.
---- Booth was a patriot ----
Because they are (semi-)automated reply-templates anyway?
cpghost at Cordula's Web.
Not these days, natural language parsers have reached the point where they can find motives such as revenge, they can even distinguish a heroic victory from a pyrrhic victory. They can do this without words such as "revenge" and "victory" appearing anywhere in the text. Turns out the most difficult text for a NLP to "understand" is the text found in children's stories, seems that (for some reason) kids stories have more complicated back references than either journalism or adult stories.
As to TFA: Anyone poo-poo-ing this result either does not understand it or has not bothered to look at the advances in AI over the last decade or so.We are at the point where a computer can read a novel and spit out a high school book report that would both fool and impress most english teachers, and it can do it in seconds not days.
There are also a lot of posts claiming the Turing test doesn't mean anything. However none of them I have read so far actually explain their statement, so I assume they are parroting their philosophy proffessor who was probably referring to Searle's Chinese translation room argument.
The problem with Searle's argument (aside from lacking a definition of intelligence) is that it is assumed the intelligence is either embedded in the human or the books, it then goes on to show that neither is true, it's basically an unintentional strawman argument. It completely misses the point that the intelligence is embedded in the entire system of human + books. In other words the room itself is a black-box that displays intelligent behaviour, in much the same way as the human brain is a black box that (sometimes) produces intelligent behaviour. Like it or not your soul is a mathematical object.
So now we have Searl out the way, has anybody got an actual argument that supports the notion that the Turing Test is broken by design? - Seriously, I would like to hear a good one!
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Anyone have a link to the transcript of the conversation? It is easy to say the Turing Test has been been passed, but how well it was passed will show in the quality of the conversation.
It could easily be:
(Human) How are you doing?
(AI) Whatever.
(Human) Yeah, don't I know how that feels. Did you watch the game?
(AI) Nah.
(Human) Man did you miss a great game. What are you into?
(AI) I like playing with my Gameboy
(Human) Cool. Which game?
(AI) Whatever.
(Human) Wow, sounds like you're really into it. Are you winning?
(AI) Nah.
The quality of the conversation does matter when calling a "Pass"
It's a shame this is what he's most remembered for. He was brilliant but his "test" was incorrect. I doubt he would still support it as a standard of true AI if he were alive today, able to see our modern computers and the massive amounts of data they can hurl around. Perhaps it's possible to create a conscious machine but I don't think that Eugene Goostman is one.
Perhaps a better standard would be an intelligence that makes decisions of its own choosing -- basically, one that can defy the constraints of its programming and have an original will. I guess that's a bit harder to unambiguously define. Something like Neuromancer/Wintermute.
"From the depths of my skeptical and rationalist soul, I ask the Lord to protect me from California touchie-feeliedom."
How much good you can do for a country, while they dump on you for being gay.
Add the number of years after the person died before they "apologize" for being gay bashers back when.
So if a computer program can fool that many numbers (as a percentage), then it's okay to be dicks to peeps who are different if after their death we apologize for our bigot grandparents.
Be seeing you...
The program pretends that it is a "13-year-old boy". How often do you take advice from 13-year-old-boys? I assume that Slashdot is not rampant with teenagers.
“I feel about beating the turing test in quite convenient way. Nothing original,” said Goostman, when asked how he felt after his success.
I get the feeling this is less about improvements in AI and more about Eastern European spammers lowering our expectations.
Then how do you account for the school textbook issues?
I've fallen off your lawn, and I can't get up.
What has been conducted precisely matches Turing's proposed immitation game.
While they may have matched the letter of it, they subverted the spirit of the test. This quote from the programme maker in particular is highly suggestive that they lowered the standards :-
To illustrate what I mean by lowered standards, imagine if I set up the same test, with 10 entries, and I tell the judges some of them are 2 year old babies playing on the keyboard. Armed with this information, some of the judges are likely to interpret even gibberish as typed by a human and it is not too farfetched to get more than 30% of them to agree.
This "result" is bollocks and a pure publicity stunt conveniently on falling on the 60th anniversary of Turing's death.
I want to see the actual transcripts which do not appear to have been released so far, which in itself is highly suspicious.
still though, it's an improvement over past attempts.
No, previous failed attempts attempted to simulate a normal human adult conversant in the English language. This programme fooled the judges by lowering the acceptable standards to be judged human by telling them it was (1) a 13 year old (2) non-native English speaker from Ukraine.
This is like claiming a goal after moving the goalposts.
How unspecific does it have to be? Be specific!
Are you demanding rigidly defined areas of doubt and uncertainty?
The test is passed when an AI bot talks with it self and knows it's not human, because humans are not that smart.
Why you hate Ukrainian boy? You try discriminatio to me? I no play game I real! Help me everybody I repression to me!
I've had a lot of email exchanges with my fellow government workers where I thought their responses were generated by poorly programmed AI.
It only passed because 90% of humans respond on the internet with no more thought than a computer! 8-)
Standard word patterns that are sent as a "symbol" no more advanced than pictographs, just because it is what they heard "cool" people say.
People can think, but they often don't...
It simulates a newborn baby. I opened sourced to code so others can use it
while(true) {
__switch(rand() % 4) {
____case 0: cout ____case 1: cout ____case 2: cout ____default: cout __} }
The singularity is here...
Do the results mean that AI is getting smarter, or that Humans are getting dumber?
https://www.techdirt.com/artic...
This isn't the first time people have wrongly announced this either.
Your Moon, Your Mission, Get involved! http://www.openluna.org
The real questions are:
1. Can it convince 33% of the readers that it really is a rich Norwegian businessman wanting to send you money?
2. Can it solve Caphas 33% of the time on the first attempt?
3. Can it co wreck spelling properly at least 33% of the time?
Except, if you asked the average teenager you didn't know a question phrased like that, his answer would likely be "I don't know and I don't care. Go away." You don't need to give the *correct* answer, just a *human-like* one. Thus, the brilliance of this particular attempt, though it also does feel a bit like cheating.
What nonsense! A program pretending to be an immature person with poor language comprehension and speaking ability, and incapable of talking about a large number of topics that can't be discussed with a vocabulary of 400 words and little life experience is not at all what the test is about. Turing expected an intelligent interrogator who could have a wide-ranging discussion about almost anything with the unknown other. Here's a snippet from his paper that introduces the idea of the Turing test, which he just referred to as the imitation game:
Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?
Witness: It wouldn't scan.
Interrogator: How about "a winter's day," That would scan all right.
Witness: Yes, but nobody wants to be compared to a winter's day.
Interrogator: Would you say Mr. Pickwick reminded you of Christmas?
Witness: In a way.
Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.
Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.
Scott Aaronson reveals his discussion with the chatbot. Here's the beginning, which reveals just how clueless one would have to be to be fooled into thinking it's a real human:
Scott: Which is bigger, a shoebox or Mount Everest?
Eugene: I can't make a choice right now. I should think it out later. And I forgot to ask you where you are from...
Scott: How many legs does a camel have? :-))) By the way, I still don't know your specialty -- or, possibly, I've missed it?
Eugene: Something between 2 and 4. Maybe, three?
Scott: How many legs does a millipede have?
Eugene: Just two, but Chernobyl mutants may have them up to five. I know you are supposed to trick me.
Scott: No, I need to know that you're not a chatbot. Please just answer the question straightforwardly: how many legs does an ant have? :-))) Oh, what a fruitful conversation;-)
Eugene: Something between 2 and 4. Maybe, three?
Scott: Do you understand why I'm asking such basic questions? Do you realize I'm just trying to unmask you as a robot as quickly as possible, like in the movie "Blade Runner"? ...wait
Eugene:
Read on. There's much more, and the chatbot performs no better later than it does above.
i'm afraid dave.
Arguably, artificial intelligence is better than natural stupidity.
Where are we going and why are we in a handbasket?
The only Texan I've met was this teenage girl that when she was old enough to drive she wanted a Ferrari, or a Pickup Truck. I couldn't help but think the dichotomy was very stereotypical Texan.
Microsoft, Apple, Google, Amazon what's the difference? All steal money from devs and control with walled gardens.