Slashdot Mirror
Software Takes On School Science Tests In Search For Common Sense
holy_calamity writes: Making software take school tests designed for human kids can help the quest for machines with common sense, says researchers at the Allen Institute for Artificial Intelligence. They've made software called Aristo that scores 75 percent on the multiple choice questions that make up most of New York State's 4th grade science exam. The researchers are urging other researchers to pit their best software against school tests, too, to provide a way to benchmark progress and spur competition.
Good sense is no longer common.
Star Trek, there maybe hope.
There's no such thing as "Common sense". It's just a myth to oppress those with different opinions.
And if you actually depend on people's common sense for things to work, you're doing it wrong./p.
i'm in!
“What’s difficult for humans is very different from what’s difficult for machines,” says Davis, who also works on giving software common sense. “Standardized tests for humans don’t get very good coverage of the kinds of problems that are hard for computers.”
"First they came for the slanderers and i said nothing."
Am I the only one who is not impressed with a 75% score on a 4th grade science multiple choice test? I'm not sure that AI is any better than:
Load answers into an array A
Load question into string B
Search Google for string B
Count number of times each index value of array A appears (eg count the number of times choice a appears, choice b, choice c, etc)
return value of index with the highest count.
Actually using these as benchmarks would bring "teach the test" to a whole new level.
75%? Everybody knows that you get 25% when you just guess randomly... So being able to add another 50% isn't all that amazing.
Understand how they do this though... They have taken the existing study guides and have constructed an algorithm that does basic word association. Multiple choice tests are written to have one right answer, one plausible answer and two answers which are distracters, designed to trick you. So the trick to multiple choice when you don't know the answer is to identify the distracters and pick from the remaining answers. So armed with their word association, they eliminate the distractors by finding the answers that have the more closely associated words with the question seen in the study guides and throwing out the rest. this will get you easily to a pretty good solution, and where it isn't conclusive it will easily get you to a 50/50 choice.
Problem is, this isn't how people do this. They've not invented a "common sense" way to do this that works for humans, but a way that is more suited to machine "learning". It's about pattern matching and possible associations, not knowledge of 4th grade science. They've not taught the computer 4th grade science, far from it, they've only figured out a way to winnow down the more likely answers in a multiple choice test and the computer then just guesses based on probabilities. While this is interesting, it has nothing to do with human "common sense" and is basically pointless.
Now, if they only would teach KIDS how to take multiple choice tests using similar techniques, THAT would be something worthwhile....
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
... fail at this once a week while watching, Are You Smarter Than A 5th Grader.
It little behooves the best of us to comment on the rest of us.
Standardized test scores will go up so the education establishment will look good. Plus they can fire all the teachers and replace them with hourly contract workers with 1-HB visas who will work even more unpaid overtime then current teachers. English fluency not required.
Naturally, there will be no cost savings to the public, because all the profit will go into the pockets of the outsourcing companies. That's what already happened with for profit higher education, and the public and the students are left holding the bag. But not too worry. Only the taxpayers, students and general investors were screwed. The insiders already got out with their guaranteed profit. It's the American way.
Why is Snark Required?
Common sense is dead. As evidence of this claim, I submit:
http://news.slashdot.org/story/15/09/09/2110218/researcher-the-us-owes-the-world-4-trillion-for-trashing-the-climate?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Slashdot%2Fslashdot%2Fto+((Title)Slashdot+(rdf))
Test taking skills do not equal common sense.
Multiple choice tests aren't that hard to pass, even if you don't know the material. Typically, there are four choices. Two are usually so obviously NOT the answer, that they can be easily discounted. Then it's just a matter of guessing which of the remaining two is more likely to be correct.
If I can use this technique to pass a test on a subject I know nothing about, then a machine certainly doesn't have to have common sense to duplicate the feat.
My son's score was 95%. And he was just in grade 2 ;)
...I would suggest we make electing human beings to public office illegal.