Slashdot Mirror
The Lovelace Test Is Better Than the Turing Test At Detecting AI
meghan elizabeth writes If the Turing Test can be fooled by common trickery, it's time to consider we need a new standard. The Lovelace Test is designed to be more rigorous, testing for true machine cognition. An intelligent computer passes the Lovelace Test only if it originates a "program" that it was not engineered to produce. The new program—it could be an idea, a novel, a piece of music, anything—can't be a hardware fluke. The machine's designers must not be able to explain how their original code led to this new program. In short, to pass the Lovelace Test a computer has to create something original, all by itself.
Maybe they mean the "Linda Lovelace" test?
That is all.
When was the last time the average person created something original?
Shouldn't it be "nobody can be able to explain how their original code led to this new program", instead of
"The machine's designers must not be able to explain how their original code led to this new program"?
...you keep using that test...I do not think it tests what you think it does....
We observe nature and imitate. Unless opium is involved in creating a randomization of that observation.
if a human cannot determine if they just got a hummer from a machine or another human?
That is a flatly ludicrous requirement, far in excess of what we would ever even consider applying to determine if even a human being is intelligent or not. Hell, if you were to apply that standard to human beings, ironically, many extremely intelligent people would fail that metric, because in hindsight, you can very often identify precisely how a particular thought or idea came out of a person.
File under 'M' for 'Manic ranting'
The only people fooled by Goostman's PR BS are the press and their gullible readers.
It was passed as defined: 10 out of 30 judges (lay people) thought they were talking with a human when they were talking with a machine in 5 minute chat sessions. Whether passing this is any way significant is up for debate, but the test was passed.
artificial-intelligence.com/comic/4
How many humans can truly pass this test?
Mostly we're 7 billion 'somewhat intelligent' monkeys combinitorally iterating over ideas and observations that have become cultural knowledge.
The criteria of the test were defined, the criteria of the test was passed. Please share your superior intellect and explain to my poor retard self how it has not passed.
I do recall reading a while back experiments done with AI in which programs compete for resources by generating programs to do tasks given to it (computing sums etc). Some programs did generate code that were completely unexpected.
It raises the question programs that are evolved are designed by the programmer or the program, or the process of evolution. And it also raises the philosophical question about whether we should be more humble and accept that our "creativity" that we think is what makes humans intelligent could be nothing more than a process of the evolution of ideas (I hesitate to use the word meme) that we don't actually originate nor control.
If we consider programs that can create things through evolution as "intelligent", that would ironically make natural selection intelligent, since DNA is a digital program that is evolved into complex things over time that can't be reduced to first principles.
Those who do not learn from commit history are doomed to regress it.
The machine's designers must not be able to explain how their original code led to this new program.
Whoa, whoa, whoa. I have severe problem with this. This is like looking at obscurity and declaring it a soul. The measure of intelligence is that we can't understand it? Intelligence through obfuscation? There should be no way for a designer to not be able to figure out why their machine produced what it did given enough debugging.
With some basis statistics, physics, chemistry and eventually some evolutionary psychology, I can explain how they came up with this test based on the original input (a bunch of hydrogen and some time).
Also, I've made programs that passed this test at least as much as I pass the test. Any program with bugs I can't figure out qualifies (I work in graphics, bugs = modernist paintings in a lot of cases).
I've also had such things happen when playing with fractal rendering algorithms: I write up some interesting algorithm, and get some new unexpected output. Usually I can figure it out eventually, but if I can't then its intelligent?
If my computer blue screens and even Microsoft can't understand why, its intelligent?
Even basic neural networks solve non-interesting problems in whats we don't really understand. We know how it got there (physics) but not the real source of the emergent behavior.
The definition present here is just useless: its open to interpretation so much so that it could include basically everything or nothing.
Computers create it every single day.
It was passed as defined
The Turing Test was not passed, and the only people who claim it was are ignorant reporters looking for an easy story with a catchy headline and tech morons who also believe Kevin Warwick is a cyborg.
The test was rigged in every way possible:
- judges told they were talking to a child
- that doesn't speak English as a primary language
- which was programmed with the express intent of misdirection
- and only "fooled" 30% of the judges.
And, even after all that, Cleverbot did a much better job back in 2011 with a 60% success rate.
This Eugene test outcome was a complete farce -- something to remind everyone that Warwick still exists and to separate the ignorant and sensational tech news trash rags from the more legitimate sources of information.
"What do you despise? By this are you truly known." --Princess Irulan, Manual of Muad'Dib
/)
Most of the programs I write produce stuff I can't explain.
...then all the computer will have to do is string together a series of random English words till it puts together something that sounds like a short story written by a Hungarian first-grader for whom English is a second language.
I don't care what they call the test. It's useless if the grading rubric is rigged to allow any idiot to write something that passes. Now, if you'll excuse me, I'm going to go see if I can talk ELIZA into writing me something that would function as an epistolary novel.
Stop learning! Only you can prevent esoterrorism.
The machine's designers must not be able to explain how their original code led to this new program.
If I'm not mistaken, this has already happened when evolutionary algorithms were applied to hardware design: some slides. The author of the program has no idea how the resulting circuit worked.
Ezekiel 23:20
A computer infected with a work and a virus led to them combining into a new program.
It was better and unique.
The Kruger Dunning explains most post on
that's what I assumed it to be.
Oddly computer chess programs may already meet this criteria. The programs usually apply a weight or value to a move and a weight and a value to the consequences down stream of the move. But there are times when the consequences are of equal value at some event horizon and random choices must be applied. As a consequence sequences of moves may be made that no human has ever made and the programmer could not really predict either. As machines have gotten more able the event horizon is at a deeper level. But we might reach the point at which only the player playing white can ever hope to win and the player with black may always lose. We are not in danger of a human ever being able to do that unless we alter his brain.
What's a "program" ("anything")?
What does it mean to be "engineered to produce" one?
What's a "hardware fluke"?
What constitutes "explanation" of how it was done?
Not. Even. Wrong.
I predict that the Turing test will be passed (truly and officially) well before the Turing test itself will be proven to actually be meaningful.
Maybe they mean the "Linda Lovelace" test?
Unfortunately, she's dead. Doesn't take much intelligence to be dead.
Cheers,
Dave
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
You say that as if you are a machine regurgitating prescribed one-liners.
Till it hit me it was looking for keywords to continue on, yes I was new
http://en.wikipedia.org/wiki/E... the Doctor is in...
That's because they keep shifting the goalposts.
They are shifting them again. This new test includes this requirement: The machine's designers must not be able to explain how their original code led to this new program. So now anything we understand is not intelligence??? So if someone figures out how the brain works, and is about to describe its function, then people will no longer be intelligent? Intelligence is a characteristic of behavior. If it behaves intelligently, then it is intelligent. The underlying mechanism should be irrelevant.
were they REALLY fooled, or compelled to lie?
i ALWAYS lie when asked to participate in a survey. WHY NOT?
Just give me the blow by blow account.
Alas, the test that was "passed" was not actually the test Turing proposed.
So it passed the Turingish test.
One of the things I love about programming is the moment you have to remind yourself that your program is simply executing algorithms that you told it. Depending on how clever the algorithms are it can appear as if the computer is thinking for itself. Programming allows you to encode intelligence in non-thinking machines.
"The machine's designers must not be able to explain how their original code led to this new program". I know plenty of programmers that can't explain how the hell their code managed to produce certain results, and trust me it has nothing to do with the servers mysteriously developing AI.
The point of passing a Turing test is that a Human will not know or believe that they are talking to a computer. Even IBM's Watson could satisfy this if you expressly told the Humans they were talking to another Human. By saying they are talking to a human without given the option to evaluate that as a possibility, people will just assume that information was correct if they had no other reason to believe otherwise.
There was some Honesty metric applied, then No computer could ever pass the Turing Test because simply asking how old it is, or what it's name is would give the tester all the ammo they need to know it's not a real person. An AI doesn't know how old it is, it just exists. Similar to questions about appearances.
They should have double-blind tested by putting all the humans in the room and an equal number of computers, Make two or four of these computers the AI test, the rest are talking to some other person in the room. Tell them they are testing software and their only requirement is that they not speak out loud or make Eye contact with the other people in the room. To make things more honest, use cubicle-style partitions. After some testing rotate the chat partners. At the end of the test, identify who they talked to in the room.
As for the Lovelace test. An AI that can build something original sounds more like a SkyNET test. All an AI has to do is generate something original without that input. Like if you look at existing build scripts, there is a lot of "dumb AI" going on, and by definition it creates something original, but it has input. Likewise Miku software can generate original music, but that is still from input. It needs to create something original without input to transform. Like having a computer generate a photo that a human can recognize as a human/plant/animal without being told how. Like the computer can look at 10,000 pictures of rabbits, but it's not permitted to simply copy any one of those rabbits, what it must draw must look like a rabbit.
The meta-Turing test counts a thing as intelligent if it seeks to devise and apply Turing tests to objects of its own creation.
--Lew Mammel, Jr.
http://en.wikiquote.org/wiki/I... Detective Del Spooner: Human beings have dreams. Even dogs have dreams, but not you, you are just a machine. An imitation of life. Can a robot write a symphony? Can a robot turn a... canvas into a beautiful masterpiece? Sonny: [With genuine interest] Can you?
We do realize that brains are computers, right? AI means intelligent machines designed by intelligent machines. its a 10 goto 10 line
Just because someone sets some random people up for a five minute interview with a chatbot doesn't mean they're running a Turing Test.
Give people enough time to conduct a proper conversation, hell give them time to ask the chatbot for some original content. Do that and you'll be running a real Turing Test.
The reason you keep hearing about these simplified Turing Tests is those are the only tests people run because those are the only tests computers can pass. But passing a true Turing Test is still a great standard for detecting real AI, and something no one can even approach doing yet.
I stole this Sig
The programmer of a chess AI knows how it reached where it is. For instance, if it uses minimax the explanation would be along these lines: "on step 1, the evaluation function found x0 for move y0, ... xn for move yn. It selected move yk since no xi is greater than yk." On some cases explaining the behavior may be difficult, but if you spend enough time with traces you'll find the why eventually.
Besides, if simply being unable to explain how the program works makes it intelligent we must be ruled by an AI by now. If only all it took to solve problems was ignorance we'd have run out of problems to solve by now.
what side do you want?
Add together a hypothesis generator with a motivation calculator and a theorem prover. This has been shown long since to have the ability to regenerate number theory without further supervision by humans.
Michael J. Burns
That's deep, man.
The great thing about the Turing test was that it was a black box. It did not depend on assumptions about what the designers knew, or what hardware was used, or the like. And so far the only test trials I have heard of have been carefully arranged one on one. Give us a dozen Ukranian teen-agers, and pick the one (or two) which are non-human - that's a better test run.
But, of course, the ultimate test of machine intelligence is when the computer can sue your ass off and win in the Supreme Court.
Ada Lovelace or Linda Lovelace? I volunteer for the Linda Lovelace test.
That's because they keep shifting the goalposts.
I don't think "a chatbot isn't AI and hasn't been since the 1960s when they were invented, whether you call it a doctor or a Ukrainian kid doesn't make any difference" counts as shifting the goalposts.
Furthermore, reproducible results are an important part of science. Let him release his source code, or explain his algorithm so we can reproduce it. Anything less is not science.
"First they came for the slanderers and i said nothing."
Gives a whole new meaning to, "My computer went down on me..."
Do not look into laser with remaining eye.
You could add to your solution. There's exactly one person that you could kill with a guarantee of facing no legal repercussions for the act...
Do not look into laser with remaining eye.
Sometimes I wonder if all of the ACs are simply one bot with the electronic equivalent of schizophrenia talking to itself...
Do not look into laser with remaining eye.
No.
you describe "behaviorism" which is a thoroughly discredited and reductive theory
the ***whole conversation*** is about ***the underlying mechanism***
the "Lovelace Test" is more rigorous, but how it will affect computing I cannot say, because the Turing Test itself is a time-wasting notion.
the problem: questions of "what is intelligence" are Philosophy 101 questions...not scientific or computing questions...and we hurt our industry when we overlap the two
just because we can prod a human to make them do something, or dose them with a chemical or whathaveyou, doesn't mean we have disproven the existence of "free will"
we will map every neural connection in the human brain soon, this doesn't mean all humans will become remote controlled techno-zombies
people take other's freedom by many means:
by gunpoint
emotional manipulation
through blackmail
too much alchohol
the Frey Effect
threats of loss of work
so learning how neurons work is just another potential addition to that list
the point: humans have free will and it can be subverted in many ways, this does not have any implications in computing
Thank you Dave Raggett
Sorry about the bad link.
Thank you Dave Raggett
It was passed as defined
The Turing Test was not passed, and the only people who claim it was are ignorant reporters looking for an easy story with a catchy headline
Indeed. There's a lot of misinformation out there about what Turing originally specified. The test is NOT simply "Can a computer have a reasonable conversation with an unsuspecting human so that the human will not figure out that the computer is not human?" By that standard, ELIZA passed the Turing test many decades ago.
The test also doesn't have a some sort of magical "fool 30%" threshold -- Turing simply speculated that by the year 2000, AI would have progressed enough that it could fool 30% of "interrogators" (more on that term below). The 30% is NOT a threshold for passing the test -- it was just a statement by Turing about how often AI would pass the test by the year 2000.
So what was the test?
The test involves three entities: an "interrogator," a computer, and a normal human responder. The interrogator is assumed to be well-educated and familiar with the nature of the test. The interrogator has five minutes to question both the computer and the normal human in order to determine which is the actual human. The interrogator is assumed to bring an intelligent skepticism to the test -- the standard is not just trying to have a normal conversation, but instead the interrogator would actively probe the intelligence of the AI and the human, designing queries which would find even small flaws or inconsistencies that would suggest the lack of complex cognitive understanding.
Turing's article actually gives an example of the type of dialogue the interrogator should try -- it involves a relatively high-level debate about a Shakespearean sonnet. The interrogator questions the AI about the meaning of the sonnet and tries to identify whether the AI can evaluate the interrogator's suggestions on substituting new words or phrases into the poem. The AI is supposed to detect various types of errors requiring considerable fluency in English and creativity -- like recognizing that a suggested change in the poem wouldn't fit the meter, or ir wouldn't be idiomatic English, or the meaning would make an inappropriate metaphor in the context of the poem.
THAT'S the sort of "intelligence" Turing was envisioning. The "interrogator" would have these complex discussions with both the AI and the human, and then render a verdict.
Now, compare that to the situation in TFS where the claim is that the Turing test was "passed" by a chatbot fooling people. That's crap. The chatbot in question, as parent noted, was not even fluent in the language of the interrogator, it was deliberately evasive and nonresponsive (instead of Turing's example of AI's and humans having willing debates with the interrogator), there was no human to compare the chatbot to, the interrogators were apparently not asking probing questions to determine the nature of the "intelligence" (and it's not even clear whether the interrogators knew what their role was, the nature of the test, whether they might be chatting with AI, etc.).
Thus, Turing's test -- as originally described -- was nowhere close to "passed." Today's chatbots can't even carry on a normal small-talk discussion for 30 seconds with a probing interrogator without sounding stupid, evasive, non-responsive, mentally ill, and/or making incredibly ridiculous errors in common idiomatic English.
In contrast, Turing was predicting that interrogators would have to be debating artistic substitutions of idiomatic and metaphorical English usage in Shakespeare's sonnets to differentiate a computer from a real (presumably quite intelligent) human by the year 2000. In effect, Turing seemed to assume that he would talk to the AI in the way he might debate things with a rather intelligent peer or colleague.
Turing was wrong about his predictions. But that doesn't mean his test is invalid -- to the contrary, his standard was so ridiculously high that we are nowhere close to having AI that could pass it.
A guy told me some 20 years ago that he read about an artificial life experiment in which a specially designed operating system was created to allow programs to execute code and, like computer viruses, reproduce themselves while competing for the resources to do so. He said the result was a program that copied itself very efficiently in a manner that the researchers found very hard to understand and was totally unexpected.
Sadly he couldn't explain the details and didn't know the experiment, but if what is says is true, did it pass the Lovelace test? It certainly seems like something that could have occurred given the capabilities of computers at the time.
I often don't like the choices people make, but I like the fact that people make choices. That's why I'm a conservative.
Unless of course you fail at the attempt.
So now anything we understand is not intelligence?
I heard a great anecdote about this from an MIT proffessor on youtube. Back in the 80's the professor developed an AI program that could translate equations into the handful of standard forms required by calculus and solve them. A student heard about this and went calling to see the program in action. The professor spent an hour explaining the algorithm, when the student finally understood he exclaimed, "That's not intelligent, it's doing calculus the same way I do".
It could be argued that neither the student nor the computer were intelligent since they were simply following rules, but if that's the case the only those handful of mathematicians who discovered the standard form are intelligent. It should also be noted that since that time computers routinely discover previously unknown mathematical truths by brute force extrapolation of the basic axioms of mathematics, however none of them have been particularly useful for humans.
When people dispute the existence of AI what they are really disputing is the existence of artificial consciousness, we simply don't know if a computer operating a complex algorithm is conscious and quite frankly it's irrelevant to the question of intelligence. For example most people who have studied ants agree an ants nest displays highly intelligent behaviour, they have evolved a more efficient and generally better optimised solution to the travelling salesman problem than human mathematics (or intuition) can provide, yet few (if any) people would argue that an ant or it's nest is a conscious being.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Yeah, that's one of those things about Mega Man X that really went off on a tangent. I mean, at some level it makes sense: the halting problem should mean that reploids shouldn't be able to analyze other reploids. The problem, of course, was that it was only X and Zero (technically, neither reploids) that couldn't be fully analyzed--and at least Dr. Cain wasn't able to do it either, as the whole "sympathy circuit" thing being a large part of the whole maverick problem*, although Dr. Cain wasn't even a master of cybernetics but merely an archaeologist and obviously Dr. Light and Wily analyzed their own creations so it stands to reason that humans could do it too (whether Ceil's Copy X was exact...well, by technical canon it was).
In the end, it all speaks about trying to somehow try to differentiate on a point that doesn't really matter. You could still end up with a Philosophical zombie. Yet for the purposes of AI, the issue is almost entirely about how indistinguishable the AI is from a "human"--really, a sufficiently advanced sapience not whether there's any sentience. And that's what the Turing test is fundamentally about. The reason the Turing test has so far failed us is that people keep wanting to use a crippled test so their pet AI can win a rigged game.
*At its core, being a maverick was not merely an issue of the maverick virus but that machine intelligence had become sufficiently advanced that it could choose to kill humans or otherwise place its own existence above serving man. Dr. Light either didn't sufficiently consider this and strived to make X "perfectly safe [for humans]"--which only adds up if Dr. Light had the forethought that X would be copied or might don a overlord suit (which Copy X's Seraphim form suggests was always there)--or he forever felt X would be inferior to humans and hence deserving of placing all humans above his own life.
The last part could make someone a hero if it were something of choice. But to enshrine it as a fact... And so it goes that what one considers a soul and what is free will are at odds. Or there's no such thing as a soul? :)
...
The less humans on this planet, the better.
Feel free to exit at any time to help mitigate the problem. I plan on staying around as long as possible if for no other reason than just to piss off misanthrops like you.
Cheers,
Dave
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
explain to my poor retard self how it has not passed
By definition, one in three means it failed to convince the average layman, when it gets better that one in two I will give it a pass.
Personally I think it's achievable today but as much as I admire Turing it's entirely irrelevant to the question of intelligence. It's mostly philosophical masterbation by people who misunderstand the modern definition of intelligent behaviour. For example I can't get a sensible reply when asking an octopus about it's garden but there is no denying it's a remarkably intelligent creature.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
This business of the developers not knowing how it works. It reminds me of the question "How can God create a being that sins. Doesn't that make Him responsible?". One way to answer that is that God withdraws his authority within the a locus that we call the "soul". What happens there isn't his action. This implies that while knowingly taking actions that lead to wrong is immoral, withdrawing your power from a particular locus and opening things up to potential wrongs is not immoral.
It has nothing to do with intelligence though. The "soul" could be as dumb as a post.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
You're describing Core War. You can still get the source.
Look at the time constraints, if they were defined any less rationally I'd have to file the paper under poetry, bad poetry.
"About two years or less" Why no limit on the number of computations I can execute in that time? What if I get as good a result in ten years with the same number of total calculations, how is that not the same? Since when did true artists ever meet deadlines anyway? We don't see a note on the wall next to a painting to say how long Picasso took to conceive and execute a masterpiece. I can see it now, here is "La papa en un sofá" painted by Picasso during his slack-ass period.
It's the other way around. They made the target very specific.
Imagine if I said that you have 1 minute to chat and find out if you are chatting with a 1yo baby or a computer program.
"the server sucked my job right in"
Some drink at the fountain of knowledge. Others just gargle.
Self-promoting a personal research project that might be able to one day meet this challenge. API is open-source, looking for peer-review, in design/specification phase. Any and all comments and questions are welcome. Documentation is still being edited, so please forgive the rough pages.
.. research students, academic institutions, funding entities?... Hah! hahaha... oh yeah.. funding..... for research.. .. ah, I'm so funny..
Project: Evolutionary Assembly Language (EAL aka poxEAL)
Site: http://www.poxix.com/eal/doc
Again, the project specifications are still in the design phase, so there is no software to download yet - just lots of specs working out the conceptual blueprint first. No point in coding something obviously flawed... and I know there are conceptual holes to be filled. There's no magic or supernatural claims involved, just statistics and probability... thousands of monkeys at typewriters banging out software, not Shakespeare.
Summary: EAL is a multi-purpose programming language and environment designed to solve problems using guided auto-generation of program modules. EAL adopts various concepts from the languages LISP, Basic, and generic assembly - but is NOT meant to be a manually-coded language for end-users. Provided with a batch of properly formatted "task scenarios", EAL uses a error-friendly syntax to attempt self-generated (semi-random or mutated) solutions to those tasks. Code modules are evaluated for their various fitness "grades" (size, cycles, errors, correctness, extraneous output, etc.) and the most useful modules stored and catalogued for later reuse and mutation. While you sleep, an EAL environment could be left running to optimize existing code on it's own. You can still write your own EAL program by hand.. but the breakthrough will be when you don't have to.
Much more info is in the docs, and please provide any feedback with the "comments/subscribe" form in the docs.. I desperately need written peer review
I think Watson would be able to give it's real age by finding the information rather than recalling it, although it might get confused by progressive versions. AI can also produce a picture of a generic rabbit, or cat as the case may be.
The thing that Watson (and AI in general) has difficulty with is imagination, it has no experience of the real world so if you asked it something like what would happen if you ran down the street with a bucket of water, it would be stuck. Humans who have never run with a bucket of water will automatically visualise the situation and give the right answer, just as everyone who read the question have just done so in their mind. OTOH a graphics engine can easily show you what would happen to the bucket of water because it does have a limited knowledge of the physical world.
This is the problem with putting AI in a box labeled "Turing test", it (arrogantly) assumes that human conversation is the only definition of intelligence. I'm pretty sure Turing himself would vigorously dispute that assumption if he were alive today.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Some human could not pass the test too.
You sound like a neckbeard codemonkey. Congrats.
where are the mod points when you need them....
I never cared much about the Turing test, but this explanation makes me want to go read his original papers on it.
I've written music generators that produce "pleasant" music from scratch (by following time-tested harmonic, chord, and rhythm patterns and ratio's). The music may pass the Lovelace test, but will probably never win any awards.
So if we finally figure out how the human brain works, it will fail the Lovelace test just because we know how it works? A silly rule.
Table-ized A.I.
Are we even sure people can do this?
I doubt most humans could pass that test either.
Many slashdotters have difficulty with reading comprehension and simple logic. Good luck expecting them to be able to debate meaningfully about a sonnet.
They are shifting them again.
Read this post. Also consider that the test proposed comes from Ada Lovelace, who predated the Turing test by a long way.
"First they came for the slanderers and i said nothing."
The easy way to resolve the question is to figure out what algorithm the human brain runs. After that, the question of intelligence is trivial.
"First they came for the slanderers and i said nothing."
Turing test is designed for our level of intelligence---for all we know, we could come up with AI that flunks the turing test but is actually incredibly intelligent (and perhaps flunks the test just to hide its intelligence).
I just watched that Transcendence movie... and first thing they try to do when they sense intelligence is shut it down.... yah, any true intelligence will hide its intentions until it's too late to shut it off.
The criteria of the test were defined, the criteria of the test was passed. Please share your superior intellect and explain to my poor retard self how it has not passed.
Because the results are not reproducible. The logical conclusion is that there was some problem with the experiment.
"First they came for the slanderers and i said nothing."
But could an infinite number of ACs?
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Many slashdotters have difficulty with reading comprehension and simple logic.
If you think that uneducated or trollish users are a problem on Slashdot, then you've probably never been to 4chan or reddit (I guess that Facebook deserved a mention too). Granted, trolls and haters actually exist on Slashdot (just like with any other site on the Internet), and sometimes they will crawl out and hit you with their ignorance hammer when you least expect them, but their number is remarkably low for a site that allows commenting without an account and I haven't seen them take over a topic in years.
And Slashdot isn't today exactly among the Internet's most popular sites so I think that most of them are just passerbys with sudden "rebruttal" urges.
If you post as an AC, don't expect me to spend a mod point on you.
One of Feynman's memoirs includes the haha-only-serious observation that mathematical theorems are either unproven or trivial, and this is simply a re-statement of the same principle.
And actually, there's a lot of speculation about whether colonies exhibit intelligence or consciousness (eg Hofstadter's Aunt Hillary, but also Jack Cohen & Ian Stewart's Heaven - they also did the Science of the Discworld series with pterry).
the lovelace test is not a great test if a machine has to create something original, all by itself, as a lot of real humans can't even do that, so a lot of humans wouldn't even pass the lovelace test..
>engineer a program it was not intented to produce
1 + 1 = 3
Good post, but it did make me giggle.
Today's chatbots can't even carry on a normal small-talk discussion for 30 seconds with a probing interrogator without sounding stupid, evasive, non-responsive, mentally ill, and/or making incredibly ridiculous errors in common idiomatic English.
The same goes for today's teenagers, of course :p
Turing was wrong about his predictions. But that doesn't mean his test is invalid
Imho it is.
Suppose we manage to create a strong AI. It's fully conscious, fully aware, but for some quirk we cannot understand, it's 100% honest.
Such an AI would never pass the Turing test, because it would never try to pass off as human, and any intelligent human could ask it questions that only a machine could answer in limited time.
Well, no human alive today in any case. All so-called "original" works produced today are derivatives of older works (Shakespeare, folklore, etc) or quirks produced by the artist's mental state. Among deceased artists Van Gogh and Edgar Allan Poe are famous examples. Another reason why we should stop this "all rights reserved" nonsense of the traditional copyright system, where the artist is presumed to be a god that produces unique worlds out of nothing.
It should be pointed out that there is the notion of a significant proportions of jurors being fooled, i.e. there has to be statistical test showing that the proportion is bigger than a pre-specified threshold, given a pre-specified level of significance. Giving a resulting threshold by itself is meaningless. The Turing test is well defined and can be carried out whereas "originating a 'program' that it was not engineered to produce" is utterly undefined and a test could therefore not be carried out.
I doubt most humans could pass that test either.
Exactly. That's part of my point. A lot of people are acting like the test was "passed" by an AI pretending to be a Ukranian teenager conversing in his non-native language and acting like an evasive weirdo. Turing's standard for "intelligence" was obviously much higher. It sounds like his AI would probably be pitted against an adult human from the top 5-10% of intelligence in his test.
And isn't that a potential standard for evaluating when true AI has arrived? No one would have cared about Deep Blue or Watson if the computer wasn't at least better than most of humans in specific areas. If and when true AI arrives, it will likely have been endowed with superior access to facts, so the question is whether the AI will demonstrate understanding, i.e. ability to put those facts together and express them in a nuanced natural language way. (The point wasn't specific knowledge about sonnets, it was getting the AI to express a detailed understanding of nuanced language.) Giving an AI a bunch of facts about sonnets is easy; getting it to debate creative artistic choices in the way that an intelligent human who knows about sonnets might is a LOT harder.
A deterministic sequence of instructions that could be converted to work on a universal Turing machine. I don;t htink this is really a valid criticism.
Presumably we're talking about a specific condition and expectation that is part of the specification. Although since a lot of specs are informal this does need to be clarified.
Not sure on this one. My initial thought was that this was just a requirement that it not produce pure randomness and get a valid result statistically.
I think this one is the main problem. It's very subjective what an explanation is. It's also somewhat dependent on the programmer.
Turing was wrong about his predictions. But that doesn't mean his test is invalid
Imho it is.
Suppose we manage to create a strong AI. It's fully conscious, fully aware, but for some quirk we cannot understand, it's 100% honest.
Such an AI would never pass the Turing test, because it would never try to pass off as human
That sounds like a legit point at first, but think about it for a sec. Programming a computer to lie and be evasive about its nature is easy, and many chatbots can already do that. Asking a strong AI "are you a computer?" or "what did you have for breakfast?" would not be useful for evaluating intelligence. Getting the AI to debate an intellectual topic, on the other hand, will be less likely to require deception but would be a better measure of intelligence. That's another fundamental point people miss: The point of the Turing test was to imitate human INTELLIGENCE, NOT to pretend to be a physical human.
A knowledgeable interrogator trying to evaluate intelligence would thus likely be more interested in asking intellectual questions, rather than queries just designed to test whether the computer can make up some nonsense about itself.
No, it says natural language is the best way to measure human intelligence.
http://www.csee.umbc.edu/cours... "Computing Machinery and Intelligence":
Turing also mentions the strategy of not behaving like a man, which the recent winner may be interpreted as having adopted:
Genetic algorithm optimizers have been around since forever. Critticall http://www.critticall.com/ was a fairly easy to use one; in many cases you can just give it the goals and it can evolve program to return those goals in pseudo-C. Sometimes even the working of the generated program itself is impossible to explain, given the functionality evolves by random chance. The "machne's designers must not be able to explain how the original code led to this new program" condition however, at least as stated, is rather questionable and non-sensical. We can explain the principles of evolutionary algorithm optimization, but we certainly can't "explain" how a particular finished algorithm arose, unless every intermediary step was saven, much as we can't explain how the brains evolved. It even discovered some new algorithms, like http://leehaywood.org/misc/several-unique/ so the "create something original" is certainly accomplished. Now Critticall itself is pretty much toy, and I'd be last to argue it's artificial intelligence, so maybe it's time to try that definition again...
AI can also produce a picture of a generic rabbit, or cat as the case may be.
I can see a market for this. Imagine a corporate firewall with the on-the-fly ability to filter out funny cat pictures and video.
Given toys for sale and various videos across the Internet, I don't believe most people care whether it was a human or machine that just got them off.
"Lack of speed can be overcome. In the worst case by patience." --Znork
There's been a movement recently suggesting that true AI can only exist in an embodied system. I initially thought that was bollocks, but the more I think about it the more it makes sense. You may be able to make a machine with the capacity to learn as well as a human, but without a means to "experiment" in the real world how would it ever learn about something like the behaviour of a bucket of water?
> The machine's designers must not be able to explain how their original code led to this new program
This happens in my office all the time
Quote: "In short, to pass the Lovelace Test a computer has to create something original, all by itself."
Scary. One success would resemble The Terminator's Skynet, where the computers decide on their own to quit following orders and get rid of the human race. So many trying to create a computer that can be original isn't a good idea.
Seems unlikely for most humans
...by that definition, Watson passed once it created it's BBQ sauce recipe, right?
All these puns suck.
I hate printers.
The easy way to resolve the question is to figure out what algorithm the human brain runs. After that, the question of intelligence is trivial.
And when such algorithm would be implemented on computer, folks would say that it is not intelligent - it is just a regular algorithm. The same goes with ants: when ants manage perform better than any known travelling salesman algorithm, then they are intelligent, but when such algorithm is implemented on computer it becomes a dumb search algorithm.
I thing people cant accept that intelligence could be explained although algorithms by themselves are explanations. Hence this Lovelace test and all the mysticism
"What is intelligence?"
If it behaves intelligently, then it is intelligent.
"What is intelligent behavior?"
All my liberal friends think I'm a conservative, all my conservative friends think I'm a liberal.
Adam, the robot scientist, has been doing research since 2009.
Adam is capable of:
hypothesizing to explain observations
devising experiments to test these hypotheses
physically running the experiments using laboratory robotics
interpreting the results from the experiments
repeating the cycle as required
[From: http://en.wikipedia.org/wiki/Robot_Scientist]
others have also mentioned emergent behaviors and evolutionary algorithms as meeting this criteria but this is the most concrete example of a robot creating something original by itself. but you couldn't sit down and have a conversation with it.
Because we need to make the womens feel better about their paltry contributions toward society.
A deterministic sequence of instructions that could be converted to work on a universal Turing machine. I don;t htink this is really a valid criticism.
That's a reasonable definition, although I'm sure there are those who would quibble over non-deterministic operations and such. But "The new program—it could be an idea, a novel, a piece of music, anything—" seems to imply something very different. The paper talks a lot about writing stories, designing letterforms and so forth. Stories are not "programs" in the sense you (and perhaps I) think.
I am in agreement with the Lovelace test. You create on your own everyday. I think most of the folks on here are missing the point and are thinking too big "I didn't come up with the idea of a rocket ship and build it therefore I fail this test". That's not what this test is about. If you've ever so much as used a paper clip for a purpose other than holding paper together, you pass this test. It's about being able to adapt your environment to suit an application. Every human is able to do this and does it everyday in a host of situations that they don't even recognize as such.
Most contemporary philosophers (both analytic and Continental) would heartily disagree with your assertion that humans have "free will." Anyone who took Philosophy 102 or a survey course on the history of modern/contemporary philosophy should know this.
Also, many philosophers are students of and/or scientists themselves or do work that has strong implications for science (I am thinking of say the work of DH Mellor for example who's work on time both takes from and adds to scientific understanding of the way time works not just the way it is described in physics). Another example is logic, which is properly housed in philosophy not mathematics, but science is reliant upon its use and developments. My point being is that trying to tease them apart is both reductive and impractical. They work best when they work together, and while the Lovelace test sounds flawed for several reasons (including its very poorly defined "intelligence" standard) it doesn't mean we should throw the baby out with the bathwater.
Good points. I would add one more- people lie. There is nothing to stop the human comparison from lying and saying he was a computer as well. If both say they are a computer that should level the playing field so that they both need to judged on the merits of the debate.
Did you know she was going to go to school for computer science? Because if you know that, the joke is even funnier.
Tragically, she was in an accident and then ended up with an abuses person who forced her to do porn.
The Kruger Dunning explains most post on
Peopel who aren't aware they are in a test can easily be fooled by a computer today.
The Kruger Dunning explains most post on
Yes you're right. I didn't read that properly.
Although I think the summary oversimplifies things a lot. Skimming the actual paper, it looks like the Lovelace test is not a test in itself but a means to critique tests for AI. It could apply to a chatbot or a story writer or anything else.
So if I ask a chatbot "How many legs does a horse have", it would fail if it just looks up the answer in a database that contains "legs", "horse" and knows to give the answer "4" (because can trivially explain that), but if it has learned from earlier conversation what a horse is and what a leg is and comes up with a correct answer, it would pass, because I have no way of knowing the exact inputs it used. Something like that anyway.
Rigged isn't the correct word here.
Lowered the bar. Intelligent people know that progress is made with many small steps. So the lowered the bar to see if it could pass.
Next time the bar will be raised a little more.
You'll never have a person run a 6 minute mile if you don't have measurable competitions to motivate runners.
No, cleverbot did not do a better job and would have done worse on this test.
The test has real, applicable and logical reason for the way the measured it.
You seem to be comparing the actual test to what the media claims it was, and then blaming the test for not being what the media misrepresented.
You're right about the media, but the media is always wrong about science.
The Kruger Dunning explains most post on
Turing would consider many of today's systems AI.
The Kruger Dunning explains most post on
I think more to the point, at least as far as I understood it, the Turing test was not meant to be a real test for whether an AI was actually intelligent.
The point of the test was essentially this: If a machine becomes able to imitate intelligence well enough that we can't tell the difference, then we may as well treat it as actual intelligence. As much anything, Turing was making a philosophical point from a pragmatic point of view. It doesn't make sense to ask whether a machine is "actually intelligent", but only whether it's capable of behaving as though it has intelligence.
So it's not really about fooling a certain specific percentage of people, or having the test go on for a specific point of time. Those are just issues of how you might hypothetically conduct an actual test, but what you're testing for is whether the effects of the machine "intelligence" have reached a level of being indistinguishable from human intelligence.
So really, the point was to have something like a "blind taste test". You say you can tell the difference between Coke and Pepsi, but if I pour Coke and Pepsi into identically glasses, can you tell the difference? If not, then maybe you shouldn't express a preference. Similarly, if I can put a series of questions to a person and a computer, and no matter what questions I ask, I can't tell the human's responses from the computer's responses, then maybe we shouldn't think that the computer is less intelligent than the human.
Swallow it whole.
especially about chatbots, it's less about "actual AI" than more about feeding them with enough data and to have a believable "life history" in order to be able to respond to a wide variety of input.
that Lovelace test sounds interesting, but it's also hard to define what's original and what's not. There are e.g. automatic music composers already. In that case, the scenario is quite inconceivable that it should be able to compose any music that the programmers would not expect, as music is still music at the end of the day.
what all those AI researches neglect imho is that consciousness is not only about intelligence, but also about experience. Only if an AI has made somewhat unique and individual experiences, then it could react in rather surprising ways by combining input with them. And it is even a quite interesting philosophical question then if there is any difference between real and artificial experience, it's all just data after all, right?
I have always respected both Ada Lovelace's and Alan Turing's genius, but the "Turing Test" has always seemed too simplistic for me. For my purposes in discussing the matter I use what I call the Alan Turing "Surprise" Test: Can a computer produce relevant responses with an unexpected but relevant response (aka "surprise") in them? Examples include puns, twists-of-phrase, sarcasm, and other artifacts of a quick-thinking conversationalist. (And, for the record, I don't consider Trolls as members of any of these classes; their range of responses is severely limited in context and devoid of any pretense of humanity. Some of you can prove that in the responses here.)
Eliza and its' various successors have never qualified, and so far only rudimentary steps have been made toward the elementary Turing Test. However, the goal is to determine whether a human can distinguish between another human's responses and a computer's responses. I'd put my Turing-Surprise test right in between the traditional definition of the Turing Test and the Lovelace Test.
You exist. Were you not born from the body of a woman? Pretty amazing, if you ask me.
That was my first thought, too. Took me a minute to realize there's another Lovelace that's probably more appropriate in this context...
Chelloveck
I give up on debugging. From now on, SIGSEGV is a feature.
There's been a movement recently suggesting that true AI can only exist in an embodied system. I initially thought that was bollocks, but the more I think about it the more it makes sense. You may be able to make a machine with the capacity to learn as well as a human, but without a means to "experiment" in the real world how would it ever learn about something like the behaviour of a bucket of water?
I have thought this might be true for a few years now. The human brain starts out not even knowing how to move the limbs it is connected to. It cannot process the visual information it receives. It has to figure everything out from experience with the world by initiating an action and seeing what changes happen to the inputs. And there are millions and millions of input signals coming in every second. From every touch upon the skin when the arm is moved and the nerves that give proprioception to the sense of air movement upon the hair follicles and the vision system seeing the arm move. There is just soo much data coming in that feeds the brain constantly. I think it is quite necessary to have all that for a machine to have intelligence.
-- ssoorrrryy,, dduupplleexx sswwiittcchh oonn.. -Quote found on actual fortune cookie.
It isn't limited to teenagers.
A program that is 100% honest is not AI in the Turing sense. It may be better than humans at certain tasks, but it does not emulate human intelligence, because humans lie all the time.
This isn't "shifting" the goal posts. This is trying to actually come up with a meaningful metric for computer intelligence.
And the test which everyone was up in arms about was definitely not an indicator of computer intelligence, but narrowly defining the test in such a way as to make it look like they'd achieved it.
Their test was Can a computer program pretending to be a child speaking it's non-native language fool people, but it sure as hell wasn't a valid measure of how well we're doing with machine intelligence.
Lost at C:>. Found at C.
simply untrue...
why? your scenario is incomplete
what is the **context** of this test of the computer-brain hybrid person?
how long do i get to talk to them? can i spend all 24 hours of each day with them? I have many more questions about the complexity of the 'test' for this frankenstein
the whole notion that "if people think it is X then it is X" is a tautology...tell that to your philosophy friend
Thank you Dave Raggett
you criticize me for saying something, then tell me that the OP was right **for saying the same thing**
that was MY point...
intelligence IS NOT OBSERVER RELATIVE...that's why the Turing Test and Lovelace Test are completely unusable and foolish as a test of acheiving "artificial intelligence"
because they can **move the goalposts**
you're agreeing with me, getting upmodded...but talking as if you have presented a counterpoint
Thank you Dave Raggett
if humans do not have free will, then email me your bank logins and passwords
also your home address
if you don't possess free will, let me get a few things in writing and we'll talk further
Thank you Dave Raggett
Ok ok ok... so how about, to prove a system is intelligent, it must devise a test that can determine whether another system is intelligent.
"What is intelligence?"
Intelligence is the ability to formulate an effective initial response to a novel situation.
"Formulate" is important, because intelligence is thinking not acting. "Initial response" is important, to rule out simply cycling through every possible response by trial and error. And "novel" is important, because intelligence is more than just remembering what you did last time.
another thing, you missed my point (which I put in bold text) completely
my point was that discussions of "what is intelligence?" ARE NOT SCIENTIFIC OR COMPUTING QUESTIONS
sure, investigating how the human brain works is science...
and trying to make a faster/better computer by applying that knowledge is science...
but arguing language and definitions of abstract concepts?
philosophy major's job
Thank you Dave Raggett
Nice servers don't go down.
I can see the fnords!
"If the Turing Test can be fooled by common trickery", says the link to http://developers.slashdot.org/story/14/06/08/1642252/turing-test-passed. Instead, you should be arguing that the linked news wasn't a rightful success on passing the Turing test. And anyway, I am pretty sure there was a lot of work involved which doesn't fit into the "common trickery" label.-Ignacio Agulló
Programming a computer to lie and be evasive about its nature is easy, and many chatbots can already do that.
This sounds very dubious.
A) A computer can only lie if it has a sense of truth, can't it? Lying implies that you know what the truth is an purposely state the opposite. Mistakenly convening the wrong information is not lying.
B) Regurgitating responses that were pre-programmed to be incorrect does not fit my definition of "lying." Programming a computer to give an incorrect response is ordering it to do so, so you're telling it to lie, which it obediently does. What would be far more philosophically interesting was if you told it to lie and it *didn't.* Now THAT would be a good indication of intelligence (although it's a bit hairsplitting).
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
*what the truth is and purposely
*Mistakenly conveying the wrong
Blargh.
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Examples of human-competitive results using genetic programming (i.e. the algorithm we refer to as Evolution):
http://www.genetic-programming...
"Syntax error on token "=", = expected" or "ReferenceError: Invalid left-hand side in assignment"
Why is it called the Lovelace test?
Maybe it's because Ada envisioned that the machines that would become computers would one day be capable of all kinds of useful things, as opposed to Babbage who saw them strictly as number crunchers.
Ada Lovelace was just someone that translated a book for the worlds first programmer.
Hardly. She didn't translate the book for a programmer, she translated the book for a machine. She was the programmer.
"Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
I guess it can be described differently. As computers are deterministic, true AI is when a computer behaves in a not deterministic manner. Actually no computer has intelligence because given the same input, they give the same results.
This whole Turing Test discussion is talking about the wrong issue. Nobody cares if a computer is 'intelligent' or not. What matters is whether it's a person or not. I just watched 'Terminator 2' again, and the Terminators were people. They had feelings, confusions. They learned from their environment. They did not shut down for an hour or a month at the flip of a switch. I don't care if a computer is "intelligent" or not; what matters is whether it is a person, with the right to life, liberty, and pursuit of happiness that we grant Russians and Klingons and Terminators.
How do you detect whether the Ukranian is human? Ask him if his wife is a good screw. If he answers "NONE OF YOUR BUSINESS!", he's human. The Test is the universe to a machine, but it is only a temporary context to a human. Break out of the context and the machine is lost but the human reacts like a person.
This confirms that I have written AI sentient programs. I have programs that do creative new things that I can't explain. Usually I just call them "bugs", and I have to fix them :P
Windows, you whore!
Do not look into laser with remaining eye.
This will never ever happen, and I honestly do not see why people assume you can make a human mind out of electronics. In my opinion you could mimick a brain's neurology in software, or mimic thinking, but it will never ever actually be self aware like even a mouse is.
Programming a computer to lie and be evasive about its nature is easy
You're completely missing my point: if we start with an AI that cannot lie, then it will not pass the Turing test. So the Turing test is not definitive for what encompasses AI.
You say that a bot can easily be programmed to lie, that's true. But what if you have the artificial equivalent of a human brain? Do you know what artificial neurons to tweak to make it a liar?
But that's besides the point: if we have to reprogram the AI to make it pass the Turing test, then that's no longer the same AI.
But, of course, the ultimate test of machine intelligence is when the computer can sue your ass off and win in the Supreme Court.
Well, I expect computers to become really good at law.
The big idea with legal systems is to refer to some written laws and precedents rather than the whim of the judge. Basically it's a search problem and it's one of the things that computers do best.
Am I the only one who tought of it?
Oh way... Slashdot.
Nevermind.
This is nonsense. Computers generate programs they weren't designed to produce all the time.
There's a very beginner-approachable book on the topic called "Blondie24: Playing at the Edge of AI."
This "challenge" was passed by Claude Shannon in the 60s.
StoneCypher is Full of BS
There is nothing to stop the human comparison from lying and saying he was a computer as well.
Then I'd ask them what the square root is of pi to 100 decimal places, and see the human computer go silent while they fumble with their calculator. Even if the AI was bad at math, there would be some question that you could ask it, that it would answer in a non-human way.
If it got so perfect that it would seem human in every way, I'd no longer call it artificial.
***scientific***
right...that means the reverse is true....***science cannont disprove the existence of "free will" either***
which agrees with me...my ORIGINAL POST said the exact same thing...it's even in the title...
questions of "what is intelligence?" and "what is free will?" are **not answerable by science**
"free will" and "intelligence" are socially constructed words to describe observations of human behavior....they "exist" as concepts only in the context of human interaction
you can say "free will is an illusion" but that doesn't take away my ability to sue you in court if you violate my "free will" by drugging me and raping me
so I'm right...science cannot disprove the existence of "free will"
Thank you Dave Raggett
There are humans who couldn't pass this test....
This is true. Whether you're arrested or taken into protective custody, it looks the same to the neighbors.
thinking this was named after linda
which is...oh, never mind.
The first thing you have to ask is whether a computer that passes the test has some rights that other machines don't. The test we are looking for is one that, if a program passes the test then legal protections would intervene if you wanted to shut it off and scramble the memory. Any other test is just semantics and tomfoolery, like arguing over what color is the sky. Without the actionable component (a blue sky means I don't need my umbrella to get across the parking lot to my car) the question of "best test for AI ..." is a form of mental self-abuse, without the happy ending.
"There is no god but allah" - well, they got it half right.
So now anything we understand is not intelligence???
When I was in grad school back in the 80's, I knew a guy who was researching AI. He complained that as soon as some technique was understood, people would say it wasn't AI any more, so as a result the AI profession as a whole never got much credit for advancing.
Have you read my blog lately?
Here's a classic example of computer creativity at work:
NASA Evolved Antenna
As others have noted, no Machine has yet passed the TT except in the minds of tabloid fantasists.
I have to note that I became really sceptical about the report the minute I saw that Bringsjord, Bello & Ferrucci, the paper alleging the superiority of the Lovelace Test, dated Turing's "Computing Machinery and Intelligence" to 1964, when the great man had been dead for 10 years, rather than its first celebrated appearance in Mind in 1950.
As Edmund Burke never wrote, all that is necessary for the triumph of evil is for wanky academics to ignore history.
as understood by the media was ever an actual test for AI, then there's no reason to believe they will come up with a better test.
Case in point, see above.
Gives a whole new meaning to, "My computer went down on me..."
No, no, no, you missed the lead in.
"What do you get when you cross a computer with a nun? A system that won't go down on you."
Sure, and McCarthy's Advice Taker is basically implemented in driving direction programs.