Slashdot Mirror
EFF Launches New AI Progress Measurement Project (eff.org)
Reader Peter Eckersley writes: There's a lot of real progress happening in the field of machine learning and artificial intelligence, and also a lot of hype. These technologies already have serious policy implications, and may have more in the future. But what's the ratio of hype to real progress? At EFF, we decided to find out.
Today we are launching a pilot project to measure the progress of AI research. It breaks the field into a taxonomy of subproblems like game playing, reading comprehension, computer vision, and asking neural networks to write computer programs, and tracks progress on metrics across these fields. We're hoping to get feedback and contributions from the machine learning community, with the aim of using this data to improve the conversations around the social implications, transparency, safety, and security of AI.
Today we are launching a pilot project to measure the progress of AI research. It breaks the field into a taxonomy of subproblems like game playing, reading comprehension, computer vision, and asking neural networks to write computer programs, and tracks progress on metrics across these fields. We're hoping to get feedback and contributions from the machine learning community, with the aim of using this data to improve the conversations around the social implications, transparency, safety, and security of AI.
Based on the amount of time it takes me to solve ReCaptchas lately, I think the # of ReCaptchas/second that can be solved by a human is probably a good cross-domain proxy. When it takes me 1 hour to solve a ReCaptcha, I'm pretty sure SkyNet is already real at that point and just biding its time until Judgement Day.
You can just mark 0% for progress now. Playing Go or Chess or any game is NOT AI. Neither is Siri or facial or voice recognition or autonomous driving. They are just programs. Computers are good at Go and Chess because they have strict rules to follow. Computers love rules. Computers are less good at autonomous driving because the rules aren't as clearly defined.
If those that are trying to make a fast buck off of the unsuspecting using AI, wouldn't those same scary geniuses want an AI that wouldn't treat them the same? Why not a category titled, "3 Laws Safe?"
And during my short journey into studying AI I've notice that once a beloved AI system figured out a solution, then other folks would optimize it using some variant of the C language. At the time, collecting these solutions was not feasible, but they are today. That would make an interesting category, "Accessing Solutions to Problems".
I am also reminded of a battery like device discovered at an archeological site in the Mesopotamia region. Funny, the secret of the battery was kept secret for thousands of years. Now we treat it like a short term toy.
steered.. bushwhacked, hoodwinked, trampled, sad enough.. hats off to eff providing real help for a thankless crowd... good sports, stuff that really.. never mind.. cease fire stand down.. sing along.. https://www.youtube.com/watch?v=P1t6bvY1aWY .. probably has her own planet by now?
The AI Singularity is nearly upon us, but to be fair, there's a lot of AI hype out there, too.
I comment occasionally so that I can mod others -1 overrated or -1 offtopic.
I am sure that Professor Jefferson [a critic of AI] does not wish to adopt the extreme and solipsist point of view. Probably he would be quite willing to accept the imitation game as a test. The game (with the player B omitted) is frequently used in practice under the name of viva voce to discover whether some one really understands something or has "learnt it parrot fashion." Let us listen in to a part of such a viva voce:
Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?
Witness: It wouldn't scan.
Interrogator: How about "a winter's day," That would scan all right.
Witness: Yes, but nobody wants to be compared to a winter's day.
Interrogator: Would you say Mr. Pickwick reminded you of Christmas?
Witness: In a way.
Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.
Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.
And so on, What would Professor Jefferson say if the sonnet-writing machine was able to answer like this in the viva voce? I do not know whether he would regard the machine as "merely artificially signalling" these answers, but if the answers were as satisfactory and sustained as in the above passage I do not think he would describe it as "an easy contrivance."
That's an example of what Alan Turing expected of the "Turing Test." And the issue isn't knowledge of sonnets or English lit here or whatever -- it's being able to parse and understand and respond reasonably to demonstrate such understanding. That was Turing's definition of AI. The kind of AI that he predicted by the year 2000 would be able to fool a skilled "interrogator" specifically trying to trip up the AI and identify the computer when an AI would be put up against a human in the "imitation game" test.
When a chatbot can do this, call me. Otherwise, all of this talk about "artificial intelligence," "deep learning," "neural networks," etc. is just fancy words for slightly more powerful statistical tools and adaptive algorithms. Maybe chaining billions of such things together could eventually lead to something that could carry on a conversation like Turing's example, but I've never encountered a chatbot with anything close to that. Most chatbots can't understand a pronoun reference to the previous sentence, let alone make abstract connections as shown in the above quotation.
You seem to contradict yourself “Computers are less good at autonomous driving because the rules aren't as clearly defined”, and yet here we are with self driving cars already and soon to be affordable by the masses. Computers are becoming better at dealing with messy data. They are getting better at just about everything across the board and yet you would mark their progress at 0%, because evidently they can only follow rules. Is a neural network just following rules when it teaches itself to play Go? I’m assuming you would say yes. How about this, the neurons in your brain are just following rules when they sum action potentials across your synapses. I would concede that computers are not highly self-aware (yet). Are all animals highly self aware? I remember when people use to lament that computers where not as smart as a mosquito. I think we are probably at least to reptilian levels of self awareness and intelligence by now. AI is progress far faster than evolution did in creating human intelligence and only seems to be accelerating. No one thought 15 years ago that we’d have self driving cars by now, that computers could parse speaker independent speech, and identify objects in pictures even the species and bread of animals in pictures. By denying that there is true AI now, you seem to imply any true breakthrough in self awareness and self motivation are decades away if not impossible. I suspect self awareness will coalesce as AI gains more skills in more domains. I don’t mean coalesce on its own, but knowledge feeds on knowledge and AI workers will eventually crack problems that seem insurmountable now. Seems they get no credit for doing things that just a few years ago where suppose to be decades away.
Letter To Iran
Depends on whether you render AI as "artificial intelligence" (dumb and tired) or "automagic induction" (smart and wired).
Automatic induction is rocking out, lately, with important applications constructed using general purpose learning algorithms, mounds of data, and very little hand-crafted (expensive) feature logic.
Feature engineering is pretty much a dead career already.
But if you're satisfied spending the rest of your life griping about scant progress at clearing the One Ridiculously High Bar to Rule Them All (1950s-style), go right ahead.
Why isn't this 'mixed realities' in keeping up with the pluralisation of all the words?