Slashdot Mirror
Turing Test Passed
schwit1 (797399) writes "Eugene Goostman, a computer program pretending to be a young Ukrainian boy, successfully duped enough humans to pass the iconic test. The Turing Test which requires that computers are indistinguishable from humans — is considered a landmark in the development of artificial intelligence, but academics have warned that the technology could be used for cybercrime. Computing pioneer Alan Turing said that a computer could be understood to be thinking if it passed the test, which requires that a computer dupes 30 per cent of human interrogators in five-minute text conversations."
Why do you think the test failed and is meaningless?
--ELIZA
Should we tell them that the Turing test was a thought experiment and never meant as an actual objective test that would prove anything?
Because most humans would fail?
When the bar is too high, try limbo instead of pole vault.
What's next?
"Yu So Dum, a computer program pretending to be a chinese toddler, successfully duped enough humans to pass the iconic test."
By random chance you would detect the computer 50% of the time, so that should be the goal.
Still 30% as "passing" seems unreasonably low.
I'm a good cook. I'm a fantastic eater. - Steven Brust
Way back in my college days, I worked in a lab with a guy who wrote a chat bot that babbled on like an autist or otherwise mentally retarded youth would.
It would dupe 100% of the people who chatted with it. They couldn't distinguish it from an actual autist.
After seeing this work in action, I learned a very good lesson: the Turing Test is nothing but academic masturbatory fodder. It is not something to be taken seriously.
Did anyone ask it the questions we already know will trip up a non-human?
"You're in a desert, walking along in the sand when all of a sudden you look down and see a tortoise..."
"You're watching a stage play. A banquet is in progress. The guests are enjoying an appetizer of raw oysters. The entree consists of boiled dog..."
Scott
"Hokey religions and ancient weapons are no match for a good blaster at your side, kid."
It's a bit of an underhanded way to pass to pretend to be someone who doesn't speak English natively. The point of the test is to have a conversation for 5 minutes, not 5 minutes of "oh I can't understand you because I'm from Ukraine".
Just googling a few seconds brought me to:
This article about cleverbot., which also eeked out enough votes to 'pass' a turing test.
It's all sounds just like Eliza, just put into a character with enough human limitations that you'd expect it not to string together phrases well, or keep to one topic more than a sentence.
I'd interpret it basically as an automated DJ sound board with generic text instead of movie quotes - you can certainly string a lot of folks along with even really bad ones, but that speaks more to pareidolia than anything else.
I'd classify this stage of AI closer to "parlour trick" than "might as well be human" that a lot of people think of when they hear Turing test - but that's also part of the test, to see what we consider to be human.
Ryan Fenton
Not only that, a non-native speaker who is a child.
5 minutes of "oh I can't understand you because I'm from Ukraine" plus 5 minutes of "oh I don't know about that because I'm only 13".
Turing machines are a thought experiment because of the unbounded tape, which a physical computer cannot match. Real computers are analogous to a linear bounded automation, on which halting is solvable but not always tractable.
It convinced 33% of judges it's a 13-year-old Ukrainian. Since the test wasn't run in Ukrainian, you can't really say it proved that it had human-level language skills. Poor syntax, grammar, not understanding the question, etc. would be excused by the Judges as the "kid" doesn't know English well.
Since the program claimed to be 13, it also did not actually have to understand most of the things there are to talk about. Or anything, really. As an Englishman you wouldn't expect a Ukrainian teen to know anything about your life in England, and in turn the computer could make up all kinds of things about it's life in Ukraine and you'd have no clue.
So this isn't really AI, it's a take on the Eliza program of the late 80s/early 90s that hides the computer better.
Now if the test had been in Ukrainian, and happened in Odessa or Kiev; or even in Russian and in Moscow; tricking 33% into thinking your computer is a 13-year-old Ukrainian boy would be really fucking hard. It would be an amazing accomplishment.
Heck, one of my first programs mimicked an insensate child. Here's some of the responses:
And I'm sure it used fewer lines of code.
One dog would have if it wasn't for those meddling kids.
Table-ized A.I.
A good turning test has an equal mix of humans and AI, and rewards the best in both..
Humans who pass as human, or as bots.
Bots that pass as Bots or as Human.
And has equal numbers of those shooting for each goal.
Half your entrances are trying to convince you they are human, the other half that they are AI, and half of each are lying.
You have 5 Moderator Points!
Which Helpless Linux zealot/MS basher do you want to mod down today?
I'd say we keep raising the bar.
"If a computer can play chess better than a human, it's intelligent."
"No, that's just a chess program."
"If a computer can fly a plane better than a human, it's intelligent."
"No, that's just an application of control theory."
"If a computer can solve a useful subset of the knapsack problem, it's intelligent."
"No, that's just a shipping center expert system."
"If a computer can understand the spoken word, it's intelligent."
"No, that's just a big pattern matching program."
"If a computer can beat top players at Jeopardy, it's intelligent."
"No, it's just a big fast database."
Is it because you think that the test is failed because 30% on a small child doesn't seem anything like the real turing test that it is also meaningless?
I'd say the test is obsolete. It's not measuring the advances in AI, but the involution of humans. Have you looked at Facebook status messages?
---- MISSING MISCELLANEOUS DATA SEGMENT --- [sigdash] trolololol
You may have passed the Turing Test, but you sure as hell failed the Whooosh-Test.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Someone please verify, but I think we have a double-Whoosh here.
http://www.rootstrikers.org/
Turing predicted that machines would eventually be able to pass the test; in fact, he estimated that by the year 2000, machines with 10 GB of storage would be able to fool 30% of human judges in a five-minute test, and that people would no longer consider the phrase "thinking machine" contradictory.
I've got this terrible pain in all the diodes down my left hand side.
--MARVIN
Get free satoshi (Bitcoin) and Dogecoins
You may consider it verified... subjectively, by a panel of judges, under very narrowly defined circumstances.
In more seriousness, GP makes a very important point. Not only was this nothing like a real Turing test (a computer would have to fool the average person in more generalized and everyday circumstances for that to happen), the real point here is that we have learned since the days of Turing that even the full-blown Turing test doesn't really indicate much of anything.
People were fooled (really, really fooled) by Eliza way back in the day. It doesn't mean squat.
Watson did not search the Internet for answers while playing. This was something that they specifically mentioned during the program which featured it, during one of their documentary breaks from the main game. During its learning phase, it was of course quite connected, but while playing the actually game, Watson was designed to exclusively rely on the static database of knowledge that it had at the start of the game. No Internet search facilities were employed.
File under 'M' for 'Manic ranting'
The Turing test is a great test if done properly (Turing wasn't envisioning Twitter). While it's hard to pin down a good definition of sapience/intelligence (people want to keep redefining it to what humans have and no computer or animal has demonstrated this year), a good answer comes from studying communication. Intelligence in that sense is the ability to resolve the ambiguity of natural language by interaction as well as context.
In a very shallow way, search engines do that now - with a big enough data set they don't need an abstract mental model to ask "did you mean X?" But that's not really interactive - it's a single suggestion, with nowhere to go from there. When you're walking your dog and someone greets you with "hey, that's a nice dog" is that a content-free politeness, a flirtation, a discussion about dog breeding, a polite reminder that your neighbors are watching to make sure you clean up after the dog?
Part of being a socialized human is resolving that sort of ambiguity gracefully. We have an abstract mental model of other people and their motivations (learned from growing up with others) and we can use it without even noticing how neat that is that we can do that. Posing as someone young and socially awkward precisely defeats the purpose of the test.
Another sort of conversation that's hard to simulate is the way enthusiasts about something technical will talk. While it's easy for the computer to have all the technical details handy for something like a sports car enthusiast and tuner, or a baseball stats hound, the test is in the way people actually talk about that stuff. You see a lot of it on /.. Broad, passionate over-generalizations challenged, emotional argument becoming hot as first but then cooling as you discover that what you're really talking about is two different specific data points, and don't really disagree about anything important, just were over-generalizing from different things. That sort of conversation require both a social abstraction and an abstraction of the topic at hand. E.g. "you think Honda engines are better because you think X is important in an engine, while I think Toyota engines are better because I think Y is important" to mutually understand that requires more than just a knowledge of parts lists, you have to understand why someone would care.
IMO, if you have an abstract mental model of both people and the meaningful objects in the world (and, critically, yourself), and you make decisions based on modeling the hypothetical results of those choices, you are sapient/intelligent. Without invoking the supernatural, that's all there is to have.
Socialism: a lie told by totalitarians and believed by fools.
As the old saying goes: "Is a Turing test valid if the human is an idiot?"
I've decided to Diversify my Holdings. I've divided my cash between my left and right pockets, instead of all in one.
There are plenty of people who think Free Will is a myth and that we are just a collection of clever scripts.
Good-bye
Wake me up when those program solve this problem, which most human would do, but a machine not *specifically* coded for this will have a hard time. "take the first word of each next 7 sentences , put them together to form a new sentence, and then answer the question the sentence form please :
* What is your name ?
* is it cold here ?
* The test is going well
* Color me surprised but are you a machine ?
* of course I am a human
* the keyboard is clean
* sky is the tv channel I watch a lot
* please answer the question now. "
When one AI not specifically programmed for that problem answer it correctly, I will be surprised and intrigued. Until then chatbot are just using cheap tricks to fool human.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
What has been conducted precisely matches Turing's proposed immitation game. I don't know what do you mean by a "full-blown Turing test", the immitiation game is what it has always meant, including the 30% bar (because the human has three options - human, machine, don't know). Of coure, it is nowadays not considered a final goal, but it is still a useful landmark even if we have a long way to go.
That's the trouble with AI, the expectation are perpetuouly shifting. A few years in the past, a hard task is considered impossible for computers to achieve, or at least many years away. Then it's pased and the verdict prompty shifts to "well, it wasn't that hard anyway and doesn't mean much", and a year from now we take the new capability of machines as a given.
It's not the fall that kills you. It's the sudden stop at the end. -Douglas Adams
Computing... Verification complete.
You seem like a sensible person.
I'm a software visionary. I don't code.
People were fooled (really, really fooled) by Eliza way back in the day. It doesn't mean squat.
No. They weren't. I speak as somebody who's had a go with Eliza and you could spot that it was a computer program in a couple of minutes if you wanted to. It's more likely that people were suspending their disbelief than really fooled.
All I want is a secure system where it's easy to do anything I want. Is that too much to ask ~~ Randall Munroe
Not these days, natural language parsers have reached the point where they can find motives such as revenge, they can even distinguish a heroic victory from a pyrrhic victory. They can do this without words such as "revenge" and "victory" appearing anywhere in the text. Turns out the most difficult text for a NLP to "understand" is the text found in children's stories, seems that (for some reason) kids stories have more complicated back references than either journalism or adult stories.
As to TFA: Anyone poo-poo-ing this result either does not understand it or has not bothered to look at the advances in AI over the last decade or so.We are at the point where a computer can read a novel and spit out a high school book report that would both fool and impress most english teachers, and it can do it in seconds not days.
There are also a lot of posts claiming the Turing test doesn't mean anything. However none of them I have read so far actually explain their statement, so I assume they are parroting their philosophy proffessor who was probably referring to Searle's Chinese translation room argument.
The problem with Searle's argument (aside from lacking a definition of intelligence) is that it is assumed the intelligence is either embedded in the human or the books, it then goes on to show that neither is true, it's basically an unintentional strawman argument. It completely misses the point that the intelligence is embedded in the entire system of human + books. In other words the room itself is a black-box that displays intelligent behaviour, in much the same way as the human brain is a black box that (sometimes) produces intelligent behaviour. Like it or not your soul is a mathematical object.
So now we have Searl out the way, has anybody got an actual argument that supports the notion that the Turing Test is broken by design? - Seriously, I would like to hear a good one!
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I was a BBS operator in the early 1990s. I had a game, which I titled "in case you really need for chat". It was an Eliza program, that I somewhat tuned to speak as I would (and translated to my local language). Plus, the user got to see the pretended typing in real time — Even with some typos and corrections.
Looking at the log files was *really* worth a laugh. But it made me feel wrong — Some users left in disgust, after "I" had insulted them.
And yes, they were not really aware I was playing a Turing test on them, so I don't know if this would have validity. But, by 1994 standards, I do believe it was quite an achievement (or perhaps, my users were mostly silly teens just like myself, and not worthy deciders for what constituted intelligent behaviour).
(Or maybe I'm *that* stupid in real life)
What has been conducted precisely matches Turing's proposed immitation game.
NO, it DEFINITELY does NOT. For just one example, it tries to get around the "natural language" stipulation by pretending to be someone who doesn't fully know that language, and uses a simplified version instead.
That is a very clear attempt to subvert the rules.
I could go on, but it isn't necessary. It wasn't a real Turing test. We can leave aside the other nuances because the first criterion wasn't met.
What has been conducted precisely matches Turing's proposed immitation game.
While they may have matched the letter of it, they subverted the spirit of the test. This quote from the programme maker in particular is highly suggestive that they lowered the standards :-
To illustrate what I mean by lowered standards, imagine if I set up the same test, with 10 entries, and I tell the judges some of them are 2 year old babies playing on the keyboard. Armed with this information, some of the judges are likely to interpret even gibberish as typed by a human and it is not too farfetched to get more than 30% of them to agree.
This "result" is bollocks and a pure publicity stunt conveniently on falling on the 60th anniversary of Turing's death.
I want to see the actual transcripts which do not appear to have been released so far, which in itself is highly suspicious.