IQ Test Pegs ConceptNet 4 AI About As Smart As a 4-Year-Old

An anonymous reader writes "Artificial and natural knowledge researchers at the University of Illinois at Chicago have IQ-tested one of the best available artificial intelligence systems to see how intelligent it really is. Turns out–it's about as smart as the average 4-year-old. The team put ConceptNet 4, an artificial intelligence system developed at M.I.T., through the verbal portions of the Wechsler Preschool and Primary Scale of Intelligence Test, a standard IQ assessment for young children. They found ConceptNet 4 has the average IQ of a young child. But unlike most children, the machine's scores were very uneven across different portions of the test." If you'd like to play with the AI system described here, take note of the ConceptNet API documentation, and this Ubuntu-centric installation guide.

Comments

More like autistic-savant 4 year old

[schneidafunk]

From the article: “If a child had scores that varied this much, it might be a symptom that something was wrong,” said Robert Sloan, professor and head of computer science at UIC, and lead author on the study.

Re:More like autistic-savant 4 year old

[Anonymous Coward]

Being as intelligent as a 4 year old human on an IQ test is not even remotely related to having the learning abilities of a 4 year old human.

Misleading crap

[Anonymous Coward]

We are nowhere near getting an AI that can navigate the world at the level of a 4 year old. All the program can do is simple tasks in vocabulary and such with no real understanding of those words. Nothing to see here.

Re:Misleading crap

[Anonymous Coward]

And you married her. How smart does that make you? About as smart as someone linking his FaceBook account to his Slashdot account.

Re:Misleading crap

[ebno-10db]

We are nowhere near getting an AI that can navigate the world at the level of a 4 year old. All the program can do is simple tasks in vocabulary and such with no real understanding of those words. Nothing to see here.

The headline is the usual attention grabbing junk, but the article itself does a decent job of explaining it:

Sloan said ConceptNet 4 did very well on a test of vocabulary and on a test of its ability to recognize similarities.

“But ConceptNet 4 did dramatically worse than average on comprehension—the ‘why’ questions,” he said.

One of the hardest problems in building an artificial intelligence, Sloan said, is devising a computer program that can make sound and prudent judgment based on a simple perception of the situation or facts–the dictionary definition of commonsense.

Commonsense has eluded AI engineers because it requires both a very large collection of facts and what Sloan calls implicit facts–things so obvious that we don’t know we know them. A computer may know the temperature at which water freezes, but we know that ice is cold.

“All of us know a huge number of things,” said Sloan. “As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don’t appreciate having their tails pulled. Life is a rich learning environment.”

IQ tests mean little enough for a human being, for AI they're little more than cute. Most 4 year old's know if someone is mad at them (expression, tone of voice, etc.) and, from past experience, often know why someone is mad at them. They're also clever enough to pretend they don't know why someone is mad at them. Most importantly (and practically), they know to start acting cute before somebody kills them. Let me know when an AI program can do that.

P.S. This is not to disparage the AI work, just to keep things in perspective.

I think they have it backwards

[shadowrat]

They didn't assess how intelligent this AI is. They assessed the IQ test and found it to be a poor indication of intelligence.

IQ tests only apply to humans

[Anonymous Coward]

These tests don't tell us much about the power of an AI and here is why. If you give a human test with a million questions, then giving one more question is not going to tell you much more. You could probably remove some of the questions too without removing much information about how smart the person is. It turns out some of the questions are much more valuable when it comes to figuring how smart someone is. If you put enough statistics work into that, you'll be able to condense those million questions into a quite short list of questions that can be administered in an hour or so, to a human, yet still tell you almost as much information as the million question test did. That's what an IQ test is.

The problem is, if you give that test to an AI, then the IQ number you get at the end won't tell you how well the AI would have done at a million/billion/trillion question test. You do get that information for a human because the test has been carefully constructed to be like that. For an AI, all you learn is how well the AI does at the questions in the test, which is much less interesting than the information you get from a human taking an IQ test.

Re:But

[flyingfsck]

I'm wondering whether your post deserves a whoosh, or a huh? or whether you are a 4 year old AI...