Slashdot Mirror
Turing Test Passed
schwit1 (797399) writes "Eugene Goostman, a computer program pretending to be a young Ukrainian boy, successfully duped enough humans to pass the iconic test. The Turing Test which requires that computers are indistinguishable from humans — is considered a landmark in the development of artificial intelligence, but academics have warned that the technology could be used for cybercrime. Computing pioneer Alan Turing said that a computer could be understood to be thinking if it passed the test, which requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations."
The test itself failed and is meaningless.
That's a pretty low bar. So to pass the test a computer needs three very low IQ subjects and seven normal people? Hell, the Alice program would probably pass. How about a more reasonable percentage, like 95%?
Free Martian Whores!
Should we tell them that the Turing test was a thought experiment and never meant as an actual objective test that would prove anything?
When the bar is too high, try limbo instead of pole vault.
What's next?
"Yu So Dum, a computer program pretending to be a chinese toddler, successfully duped enough humans to pass the iconic test."
Way back in my college days, I worked in a lab with a guy who wrote a chat bot that babbled on like an autist or otherwise mentally retarded youth would.
It would dupe 100% of the people who chatted with it. They couldn't distinguish it from an actual autist.
After seeing this work in action, I learned a very good lesson: the Turing Test is nothing but academic masturbatory fodder. It is not something to be taken seriously.
Did anyone ask it the questions we already know will trip up a non-human?
"You're in a desert, walking along in the sand when all of a sudden you look down and see a tortoise..."
"You're watching a stage play. A banquet is in progress. The guests are enjoying an appetizer of raw oysters. The entree consists of boiled dog..."
Scott
"Hokey religions and ancient weapons are no match for a good blaster at your side, kid."
Turing never participated in Facebook chats. Our expectations of intelligence for the other side has been lowered a lot. We attribute to stupidity what can be explained by an AI in the other side. And of course, the stupid side could be the one talking to the AI too.
Just googling a few seconds brought me to:
This article about cleverbot., which also eeked out enough votes to 'pass' a turing test.
It's all sounds just like Eliza, just put into a character with enough human limitations that you'd expect it not to string together phrases well, or keep to one topic more than a sentence.
I'd interpret it basically as an automated DJ sound board with generic text instead of movie quotes - you can certainly string a lot of folks along with even really bad ones, but that speaks more to pareidolia than anything else.
I'd classify this stage of AI closer to "parlour trick" than "might as well be human" that a lot of people think of when they hear Turing test - but that's also part of the test, to see what we consider to be human.
Ryan Fenton
I feel like the requirements for the Turing test have been consistently lowered over the years to match what would be considered realistic to achieve rather than, as Alan Turing seemed to believe, demonstrate that a computer can be said to actually be "thinking."
What is the probability of this having happened by now if we simply repeated the Turing test with programs that previously failed?
"Well, 30% isn't very impressive."
"Well, but people expect online correspondents to be dumb."
"Well, nobody ever thought the Turing test really meant anything."
Whether you "believe in" AI or not, progress is happening.
There will always be people who refuse to believe that a computer can be intelligent "in the same sense that humans are". Eventually, though, most of us will recognize and accept that intelligence and self-awareness are mostly a matter of illusion, and that there's nothing to prevent a machine from manifesting that same illusion.
Turing machines are a thought experiment because of the unbounded tape, which a physical computer cannot match. Real computers are analogous to a linear bounded automation, on which halting is solvable but not always tractable.
It convinced 33% of judges it's a 13-year-old Ukrainian. Since the test wasn't run in Ukrainian, you can't really say it proved that it had human-level language skills. Poor syntax, grammar, not understanding the question, etc. would be excused by the Judges as the "kid" doesn't know English well.
Since the program claimed to be 13, it also did not actually have to understand most of the things there are to talk about. Or anything, really. As an Englishman you wouldn't expect a Ukrainian teen to know anything about your life in England, and in turn the computer could make up all kinds of things about it's life in Ukraine and you'd have no clue.
So this isn't really AI, it's a take on the Eliza program of the late 80s/early 90s that hides the computer better.
Now if the test had been in Ukrainian, and happened in Odessa or Kiev; or even in Russian and in Moscow; tricking 33% into thinking your computer is a 13-year-old Ukrainian boy would be really fucking hard. It would be an amazing accomplishment.
One dog would have if it wasn't for those meddling kids.
Table-ized A.I.
30% of tech support could not pass the Turing test
Like with that chatbot that pretended to be a teenage FPS gamer. Lolbot I think it was named.
requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations
Are there any requirements that must be met by the 'human interrogators'? What if they were all morons?
A turing test is testing such human experience aspects as:
- aculturation (what the person has been taught through education and socialization during their whole life up to that point)
- bias in expression based on typical human likes, dislikes, needs, desires, avoidances
Tarzan / wolf-boy would probably fail the Turing test based on the first factor. Might be very intelligent though.
Second aspect is just characteristic of a particular type of being that makes use of intelligence. Intelligent aliens would also have likes, dislikes, needs, desires, avoidances, simply based on also being self-interested "keep it together" beings, but the specifics might be very different, and would cause a fail of TT.
These experiential and situational and specific-agent-needs-desires-avoidances aspects have very little to do with the essence of intelligence.
General intelligence is probably better assessed through specific carefully designed tests designed to assess:
1) Concept learning, procedure learning capability in arbitrarily general contexts
2) Prediction of situation outcomes with novelty in situation presentations.
3) Ability to answer questions or take actions that show comprehension of essential / invariant aspects of situations, after opportunity to learn similar situations through either direct sensory input or linguistic instruction.
Where are we going and why are we in a handbasket?
Computers can win at the Turing test with a little clever programming and misdirection, i.e. not answering questions that computers can't answer and instead distracting the questioner with a "satisfactory" response. The kinds of tricks that PR, marketing, and politicians are great at and are formulaic in their simplicity to achieve.
I wonder if the panel of academics ever thought of asking a few Winograd Schema questions? http://www.cs.nyu.edu/davise/p... Failure to answer these is failure to present basic human intelligence. The key to this approach is that it relies on pragmatic meaning, i.e. what we mean/intend to say, rather than on linguistic (lexical and semantic) interpretation, i.e. what we actually say. AFAIK, even the most advanced and powerful computers are far from achieving this and we still don't really know how we do it either.
All it showed, like any other Turing Test, is the gullibility of the subjects.
1) "Ukrainian" speaking English
2) 13 years old
Right there you have set up an expectation in the audience of subjects for a limited vocabulary, no need for grammatical perfection, little need for slang, and a lack of education. Now add in "star wars and matrix" and you have reduced the topics of discussion even more to the ones the programmers know best.
This thing would never have answered a question of 'Why', it also was under no pressure to being able to create a pun, both of which are easy things any older and educated human could do.
Garbage test, garbage results.
As usual.
"But remember, most lynch mobs aren't this nice." (H.Simpson)
-- Joe
Wake me up when those program solve this problem, which most human would do, but a machine not *specifically* coded for this will have a hard time. "take the first word of each next 7 sentences , put them together to form a new sentence, and then answer the question the sentence form please :
* What is your name ?
* is it cold here ?
* The test is going well
* Color me surprised but are you a machine ?
* of course I am a human
* the keyboard is clean
* sky is the tv channel I watch a lot
* please answer the question now. "
When one AI not specifically programmed for that problem answer it correctly, I will be surprised and intrigued. Until then chatbot are just using cheap tricks to fool human.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
If you want to be thought of as knowledgeable on a subject like this, you might consider learning the difference between silicone and silicon.
Also, for the record, your distinction between AI and MI is BS. There have been many varieties of AI research, some inspired more by ideas about human brain function or human cognition, and some inspired less directly by those and more focussed on best exploiting computer-of-the-day capabilities.
All attempts which are not purely theoretical are implemented, and have since day 1 been implemented, in computing machines (which, needless to say, are artificial), so you are splitting hairs.
Whether the advanced computing research specialization of the day gets called by its proponent part of AI or not has nothing to do with fundamental distinctions, and more to do with funding fads and buzzwords-du-jour.
Where are we going and why are we in a handbasket?
Not these days, natural language parsers have reached the point where they can find motives such as revenge, they can even distinguish a heroic victory from a pyrrhic victory. They can do this without words such as "revenge" and "victory" appearing anywhere in the text. Turns out the most difficult text for a NLP to "understand" is the text found in children's stories, seems that (for some reason) kids stories have more complicated back references than either journalism or adult stories.
As to TFA: Anyone poo-poo-ing this result either does not understand it or has not bothered to look at the advances in AI over the last decade or so.We are at the point where a computer can read a novel and spit out a high school book report that would both fool and impress most english teachers, and it can do it in seconds not days.
There are also a lot of posts claiming the Turing test doesn't mean anything. However none of them I have read so far actually explain their statement, so I assume they are parroting their philosophy proffessor who was probably referring to Searle's Chinese translation room argument.
The problem with Searle's argument (aside from lacking a definition of intelligence) is that it is assumed the intelligence is either embedded in the human or the books, it then goes on to show that neither is true, it's basically an unintentional strawman argument. It completely misses the point that the intelligence is embedded in the entire system of human + books. In other words the room itself is a black-box that displays intelligent behaviour, in much the same way as the human brain is a black box that (sometimes) produces intelligent behaviour. Like it or not your soul is a mathematical object.
So now we have Searl out the way, has anybody got an actual argument that supports the notion that the Turing Test is broken by design? - Seriously, I would like to hear a good one!
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
What has been conducted precisely matches Turing's proposed immitation game.
While they may have matched the letter of it, they subverted the spirit of the test. This quote from the programme maker in particular is highly suggestive that they lowered the standards :-
To illustrate what I mean by lowered standards, imagine if I set up the same test, with 10 entries, and I tell the judges some of them are 2 year old babies playing on the keyboard. Armed with this information, some of the judges are likely to interpret even gibberish as typed by a human and it is not too farfetched to get more than 30% of them to agree.
This "result" is bollocks and a pure publicity stunt conveniently on falling on the 60th anniversary of Turing's death.
I want to see the actual transcripts which do not appear to have been released so far, which in itself is highly suspicious.
What nonsense! A program pretending to be an immature person with poor language comprehension and speaking ability, and incapable of talking about a large number of topics that can't be discussed with a vocabulary of 400 words and little life experience is not at all what the test is about. Turing expected an intelligent interrogator who could have a wide-ranging discussion about almost anything with the unknown other. Here's a snippet from his paper that introduces the idea of the Turing test, which he just referred to as the imitation game:
Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?
Witness: It wouldn't scan.
Interrogator: How about "a winter's day," That would scan all right.
Witness: Yes, but nobody wants to be compared to a winter's day.
Interrogator: Would you say Mr. Pickwick reminded you of Christmas?
Witness: In a way.
Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.
Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.