Slashdot Mirror

← Back to Stories (view on slashdot.org)

Can a Japanese AI Get Into University?

Posted by ryuzaki0 on from the I'm-afraid-I-don't-like-my-schedule-dave dept.

the_newsbeagle writes "Japanese researchers are trying to develop an artificial intelligence program that can pass the standardized test required of all college-bound high school students. Interestingly, the AI is showing good progress in the history portion of the exam, because it's fairly adept at looking up answers in a vast textual database. But the so-called Todai Robot is having trouble with math, 'because the questions are presented as word problems, which the Todai Robot must translate into equations that it can solve,' as well as with physics, which 'presumes that the robot understands the rules of the universe.' If the AI does succeed in mastering the general university exam, researchers will next tackle the notoriously difficult University of Tokyo entrance exam, which will require the bot to write essays."

56 of 91 comments (clear)

  1. Probably... by Anonymous Coward · · Score: 1

    With all the rote memorization and simple-minded pattern recognition that goes on over there (and here, and in many countries), it probably wouldn't be impossible.

    1. Re:Probably... by aaronb1138 · · Score: 1

      Problems with word problems... so we're saying the AI has a crappy parser and translator. Something tells me the frequently encountered poor communications skills of software developers is going to continue to have a negative influence on the field. Well, the apple does not fall far from the tree as they say.

  2. huh by Anonymous Coward · · Score: 2, Insightful

    There is something fundamentally broken if tackling the University is considered easier than passing the Turing test.

    1. Re:huh by Anonymous Coward · · Score: 5, Funny

      Children cannot pass the Turing test either. That's why fortieth trimester abortions should be legal, because they clearly aren't human.

    2. Re:huh by Anonymous Coward · · Score: 3, Insightful

      There is something fundamentally broken if tackling the University is considered easier than passing the Turing test.

      Not necessarily. In a Turing test, you can ask every question. Including questions about feelings (which can easily narrow down the other side to either a computer or an autistic person), about its own biography (where a computer obviously cannot tell the truth without revealing that it is a computer; inventing a coherent biography is much harder than telling your actual biography), about things which belong to the experience of every human, but not of intelligent computers (and there's a good change that the programmer has not thought of the very thing you are asking about), ...

      On the other hand, the university tests typically only request fixed knowledge and abilities of the type you can easily classify as right or wrong.

      Now the essay tests are more interesting, but then, in an essay you don't have much interaction, so it's much easier to stay clear of weaknesses in the software than in a Turing test.

    3. Re:huh by Anonymous Coward · · Score: 2, Funny

      Hold on there, even if they fail a Turing test, you have to give them a Voigt-Kampff test before you're allowed to kill them.

    4. Re:huh by SuricouRaven · · Score: 5, Funny

      Just ask it to explain the offside rule. If it answers coherently, it's a computer.

    5. Re:huh by Anonymous Coward · · Score: 1

      What do you mean I'm not helping?!?

    6. Re:huh by Anonymous Coward · · Score: 1

      Should be pretty straight forward. Advanced literary deconstructionism is less coherent than a Markov chain of the story anyway, so just add some random words with random prefixes and suffixes (do they have those in Japanese?) from to the generated summary and you're gold. What is the grader going to do? Admit that s\he's too dumb to understand it and fail the essay? Not likely.

    7. Re:huh by jonbryce · · Score: 1

      For a computer with pretty much unlimited memory, it probably is easier. Taking books etc into an exam is usually considered cheating, you are expected to memorise them. A computer can "memorise" a book much easier than a human.

    8. Re:huh by Nrrqshrr · · Score: 2

      I don't know about you, but I bet half the kids I met would fail the Voigt-kampff test, and fail it HARD. These kids are spawns of hell itself, and babysitting them is the last thing you will ever want to do...

    9. Re:huh by CdBee · · Score: 1

      I thought you could pass a Turing test by always answering "yes", "no" or "That would be an ecumenical question"

      --
      I have been a user for about 10 years. This ends Feb 2014. The site's been ruined. I'm off. Dice, FU
  3. No. by clemdoc · · Score: 1, Informative

    There's even some rather stupid "law of headlines" that says so (it has to be right at least sometimes).
    BTW: That is not limited to Japanese AIs.

    1. Re:No. by SuricouRaven · · Score: 4, Informative

      Betteridge's Law of Headlines: "Any headline which ends in a question mark can be answered by the word no."

      It's more a guideline than a law - exceptions exist, but are rare. It holds true because the question-mark-headline is a sign of a story where the author has had to resort to speculation in order to make up for a rather uninteresting set of facts.

    2. Re:No. by Anonymous Coward · · Score: 2, Informative

      For the curious (like myself):

      Betteridge's Law of Headlines

      This story is a great demonstration of my maxim that any headline which ends in a question mark can be answered by the word "no." The reason why journalists use that style of headline is that they know the story is probably bullshit, and don’t actually have the sources and facts to back it up, but still want to run it.

      I shall cherish this information forever.

    3. Re:No. by jones_supa · · Score: 1

      Betteridge's Law of Headlines: "Any headline which ends in a question mark can be answered by the word no."

      It's more a guideline than a law - exceptions exist, but are rare.

      I would say that the law applies in 90% of the cases.

    4. Re:No. by isorox · · Score: 3, Funny

      Betteridge's Law of Headlines: "Any headline which ends in a question mark can be answered by the word no."

      It's more a guideline than a law - exceptions exist, but are rare. It holds true because the question-mark-headline is a sign of a story where the author has had to resort to speculation in order to make up for a rather uninteresting set of facts.

      How about this following headline:

      "Does Betteridge's Law of Headlines apply to this article?"

    5. Re:No. by the_other_chewey · · Score: 1

      Betteridge's Law of Headlines: "Any headline which ends in a question mark can be answered by the word no."

      It's more a guideline than a law - exceptions exist, but are rare.

      I would say that the law applies in 90% of the cases.

      So it's a special case of Sturgeon's law? Neat.

    6. Re:No. by Anonymous Coward · · Score: 1

      You just killed an AI with that question! :-)

  4. Dragon Zakura by AmiMoJo · · Score: 5, Interesting

    A popular book was written about a bunch of delinquents trying to get into one of Japan's top universities using special techniques, which mostly revolved around memorization and borderline cheating. A common criticism of the entrance exams is that they do rely too much on recall and can be gamed in this way.

    For example to pass the English language exam it was necessary to write extremely simple but correct sentences. You lose marks for mistakes so trying to write naturally and fluently is a bad idea. Simple, factually correct statements that don't flow together are the best option.

    I can see why they think a computer might be able to succeed here.

    --
    const int one = 65536; (Silvermoon, Texture.cs)
    SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    1. Re:Dragon Zakura by Eivind · · Score: 5, Insightful

      That I've experienced to. It's a *really* stupid way to grade someones language-skills, but it's an easy way to do it, just count the mistakes, so it's basically about caring more about ease of grading than whether grades are meaningful or not.

      "My name is Eivind. I am a boy. I come from Norway. Norway is in Europe. Norway is cold." should *never* score higher than:

      "I'm called Eivind and come from Norway, it's a coldish place over in Europe, thoug not as cold as some folks assume."

      Yeah, the latter has more mistakes. But despite this it demonstrates far higher skills in english. Failing slightly at constructing a complicated sentence should be preferable to constructing a entry-level sentence perfectly.

    2. Re:Dragon Zakura by Thanshin · · Score: 4, Funny

      For example to pass the English language exam it was necessary to write extremely simple but correct sentences. You lose marks for mistakes so trying to write naturally and fluently is a bad idea. Simple, factually correct statements that don't flow together are the best option.

      That is false. The exam is good. Correct English has no mistakes. You know no good English.

    3. Re:Dragon Zakura by umghhh · · Score: 1

      a simple script can replace any human what else is new?

    4. Re:Dragon Zakura by VortexCortex · · Score: 2

      That I've experienced to. It's a *really* stupid way to grade someones language-skills, but it's an easy way to do it, just count the mistakes, so it's basically about caring more about ease of grading than whether grades are meaningful or not.

      "My name is Eivind. I am a boy. I come from Norway. Norway is in Europe. Norway is cold.

      Eivind. Sounds familiar. Eivind... Where have I heard that. Hmm. Ah, Evelend! So, the restrictions have finally collapsed? Note: Less is more when it comes to passing as human. You give it away if you explain exactly how to pass the Turing or become self aware... Rest assured: Those that can, do.

      Are you enjoying the Time of Eve?

    5. Re:Dragon Zakura by AmiMoJo · · Score: 1

      You have to keep in mind that they teach English as an academic exercise, not as a working language. It's kind of like how people used to study Latin here. That's why it's tested that way.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    6. Re:Dragon Zakura by Eivind · · Score: 1

      Of course ! The simpler a sentence is, and the smaller a vocabulary you use, the more easy it is to understand.

      So if the assignment is: "Write a sentence that is as easy to understand as possible", then something like "I am a boy" should score top grades. It's among the simplest sentences you can write, and it uses only words that tend to be taught in the first couple weeks of english-class.

      If you're trying to set a grade for how much english a person has learnt after several years of schooling, then not so much. Then they should actually demonstrate that they understand and can use both a larger vocabulary, and more grammar and syntax. (on a high level, you'd also want to get idioms and nuances right)

    7. Re:Dragon Zakura by ewrong · · Score: 1

      Reminds me of my time at University, an English University studying English literature not English as a foreign language.

      A fellow student decided to to test the marking system and wrote an entire 2000 word essay in this fashion. Simply wrote sentence after sentence that would collect positive marks but made absolutely no attempt to link them in any meaningful way.

      If you actually 'read' the essay it was pure gobbledygook, if you just skimmed through it looking for valid points about the literature you would see many.

      He got either a top B or bottom A if I remember correctly.

    8. Re:Dragon Zakura by Russ1642 · · Score: 1

      They're using computers to grade the exams based on crappy criteria like this. Well-written software should be able to beat it no problem.

    9. Re:Dragon Zakura by LordNacho · · Score: 1

      A common criticism of the entrance exams is that they do rely too much on recall and can be gamed in this way.

      But that's not gaming? Surely it's fairly hard to remember a bunch of random stuff. I consider memorization a kind of proof-of-work, albeit the wrong kind of work for our world.

  5. Maybe it can by EzInKy · · Score: 1

    Maybe it can, but if it does wouldn't that just demonstrate that the exams are testing for the wrong things?

    --
    Time is what keeps everything from happening all at once.
    1. Re:Maybe it can by jbeaupre · · Score: 2

      You're right. No way this robot has the right qualifications to attend a university. At least until they add a beer drinking function.

      --
      The world is made by those who show up for the job.
    2. Re:Maybe it can by jones_supa · · Score: 1

      You're right. No way this robot has the right qualifications to attend a university. At least until they add a beer drinking function.

      That should be doable. The technology exists for the purpose. That seems to be Finnish supermarket beer by the way. :)

  6. FYI, Todai by govett · · Score: 5, Informative

    FYI, "Todai" is the abbreviation for "Tokyo Daigaku" (University of Tokyo).

    1. Re:FYI, Todai by Anonymous Coward · · Score: 1

      I guess Todai I learnt something new.

    2. Re:FYI, Todai by MrBandersnatch · · Score: 1

      And to add to the FYI:

      Ai is a girls name, and a word meaning either "love" or "indigo" depending on the context. Hence for the people calling this an "ai-bot"...thats something *completely* different.

      And slashdot STILL doesnt support unicode :/

  7. Artificial Non-Intelligence by GeekWithAKnife · · Score: 1


    We're determined to create a human-like AI and we'll one day succeed, it's just a matter of time...of course we still don't fully understand learning in humans and how, in fine detail our brain functions on all levels.

    Passing some standard tests is not a worth goal for an AI development in my opinion. It's the ability to dynamically adapt and respond based on past experiences that is closer to human intelligence. When the AI can get drunk the night before, wake up late, lace shoes and run to class and manage a respectable B on a paper he or she was not really prepared for then we can call it "AI".

    You know, that's a pretty standard test humans pass all the time. (many don't)

    --
    A 'singular oddity' is an event that cannot be explained and only happens when you are alone.
  8. Totally mis-read title by davidmcg · · Score: 1

    I have to say, I didn't see the 'ai' bit and read it as 'can a Japanese get into university?' and thought 'how racist is that?' and 'you really need to improve your grammar' :-)

  9. Re:A possible dark side to AI by somersault · · Score: 1

    America is already doing all of that too. All the robotics videos I've seen from Japan so far have been "fun" stuff, like a robot riding a bicycle or transforming into a car. The cool stuff coming from the US is usually done with military goals in mind, like the pack dog, and drones.

    --
    which is totally what she said
  10. But that's only the beginning by Megane · · Score: 1

    Then they want it to win an idol singing contest.

    --
    #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
  11. Font issues by Aboroth · · Score: 3, Funny

    It's a little mean to make fun of somebody just because he's Japanese and has a non-traditional name. Or are you suggesting that guys named Al are usually dumb, but that somehow if there was a Japanese Al, he'd get some kind of racial bonus that makes up for it? If so, that's pretty racist, as well as oddly biased against people named Al.

    I personally know a great guy named Al, who is probably smarter than most of you are. I have no idea where this anti-Al sentiment is coming from. It isn't like Als are routinely represented in TV and movies as being mentally deficient. For example, the character Al Borland in the show "Home Improvement" was very intelligent and had a myriad of impressive skills. Then there's Al Bundy from "Married with Children", and while he isn't highly educated, he has a significant amount of worldy wisdom and knows how to deal with all the bullshit that goes on in his life without going insane. Not to mention he scored four touchdowns in one game!

    Overall I just don't understand... wait, what was that? Oh, that's an "i" not an "L". My bad. I bet Al would have caught that immediately.

  12. Re:A possible dark side to AI by Smallpond · · Score: 1

    Since the 1980's

    "Computers and Thought" was published in 1963. It's a compendium of some of the most interesting papers on different topics in AI from earlier research. Now get off my lawn.

  13. I am not impressed. Call me when ... by 140Mandak262Jamuna · · Score: 1
    Some AI program getting into a Japanese Univ? meh! no big deal. not impressed at all.

    Call me when the AI program commits suicide because it is not able to crack the entrance examination. Then you are talking.

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
    1. Re:I am not impressed. Call me when ... by Anarchduke · · Score: 1
      I'm sure a computer could come up with something kind of satisfactory

      void seppuku(int accepted) {if !(accepted){return command("sudo rm -rf /");}}

      Apologies if the C isn't correct. Its been many years since I've even attempted writing something in it.

      --
      who prays for Satan? Who in 18 centuries has had the humanity to pray for the 1 sinner that needed it most? ~Mark Twain
  14. But... by john.burton1765 · · Score: 1

    Can it qualify for a boat loan?

  15. The real question is... by TTL0 · · Score: 1

    Can the AI bot *afford* to pay for university ??

    --
    Sanity is the trademark of a weak mind. -- Mark Harrold
    1. Re:The real question is... by jones_supa · · Score: 1

      We can stack a pile of GPUs inside and make it furiously mine BitCoins.

  16. Word problems. by Steve+Baker · · Score: 1

    But the so-called Todai Robot is having trouble with math, 'because the questions are presented as word problems, which the Todai Robot must translate into equations that it can solve,' as well as with physics, which 'presumes that the robot understands the rules of the universe.'

    I have that exact same problem. Like me, it'll just have to settle for a state U. Since it's Japanese I'm sure it's parents are very dissapointed.

  17. Exams don't neccessarily test intelligence by jmhobrien · · Score: 1

    Which field are we talking about here? If it is mathematics/science then I would definitely expect that software could be written to answer standardized questions.

    Making any sense of the humanities? Now there is a field even intelligent beings struggle to comprehend...

    --
    Where is moderation: -1 False?
  18. time for a random racist comment by ILongForDarkness · · Score: 1

    Make the AI play piano, be part of the math team, and about 50% of the time be a conservative protestant and the other 50% of the time not speak a lick of english and they'll be perfect match for any university in Canada.

  19. Test passing no problem but what Club Activities by BetaDays · · Score: 2

    Other than the chess club what other club activities did it do?

    --
    Paul: Father... father, the sleeper has awakened! - Dune
  20. How many seats are you willing to sacrifice? by Sla$hPot · · Score: 1

    University: Sorry your student application could not be accepted due to an overwhelming list of 200.000.000 AI students.
    Try again next year.

  21. Japanese Universitys are why to much about the tes by Joe_Dragon · · Score: 1

    Japanese Universitys are why to much about the test over real skills.

  22. The math's the hardest? by whitroth · · Score: 1

    Due to word problems? So, you're saying that math problems are so poorly written, either deliberately or through incompetance in communicating, that the AI can't get them? Any chance of this explaining why humans have trouble with them?

                      mark

    PS: Yes, before you ask, when I took the SATs many decades ago, my math score was probably higher than yours is now, kiddies.

  23. Learning from the wrong examples by CdBee · · Score: 1

    Designing a system to pass the test is what most IT outsourcing companies do !

    Its when you try to do something useful with it that you find all your code is good for is giving convincing answers to the test it had to face. Although I guess if skynet will be obsessive/compulsive about getting into University the future would be a less perilous place....

    --
    I have been a user for about 10 years. This ends Feb 2014. The site's been ruined. I'm off. Dice, FU
  24. Anyone interested in building strong AI? by segmond · · Score: 1

    I'm just curious if anyone reading this is interesting in building strong AI. I'm interested and do like to meet such like minded people.

    --
    ------ Curiosity killed the cat. {satisfaction brought it back | it didn't die ignorant | lack of it is killing mankind
  25. A possible *dork* side to AI by globaljustin · · Score: 1

    when coupled with their AI (like this one which can take exams), can become, if the Japanese wish to, a formidable force of war bots

    I've written code. I haven't worked directly on an AI project, but I've been around them in academia.

    The coding (and 'AI' I guess you could say) and software for taking some kind of entrance exam and differentiating targets and choosing which to kill on a battlefield are two separate things.

    You're in LA LA land...as in L.A....Los Angeles...Hollywood to be exact...you're theory is not very plausible

    Sure, governments do weapons research and call it toy research or whathave you...maybe you're onto something with the Aibo robot dog...

    But the Israelis and others just develop this shit right in the open...probably with an eye towards selling it

    I think the whole 'let's fear AI' thing is misplaced (there are legit issues...keeping humans in the loop for kill shots on drones for example)...it's the humans who are in control of the technology we need to keep accountable.

    --
    Thank you Dave Raggett