Slashdot Mirror


AI Beats Humans at Reading Comprehension (bloomberg.com)

In what is being called a landmark moment for natural language processing, Alibaba and Microsoft have developed AIs that can outperform humans on a reading and comprehension test. From a report: Alibaba Group put its deep neural network model through its paces last week, asking the AI to provide exact answers to more than 100,000 questions comprising a quiz that's considered one of the world's most authoritative machine-reading gauges. The model developed by Alibaba's Institute of Data Science of Technologies scored 82.44, edging past the 82.304 that rival humans achieved. Alibaba said it's the first time a machine has out-done a real person in such a contest. Microsoft achieved a similar feat, scoring 82.650 on the same test, but those results were finalized a day after Alibaba's, the company said.

6 of 171 comments (clear)

  1. No Kidding by tsqr · · Score: 3, Insightful

    Based upon the knee-jerk quality of many comments posted on /. this should not be a surprise to anyone.

  2. Doug Lenat's Test by sycodon · · Score: 4, Insightful

    “Mary saw a bicycle in the store window. She wanted it.”

    Does Mary want the bike, the store, or the window?

    --
    When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
    1. Re:Doug Lenat's Test by amalcolm · · Score: 5, Funny

      Obviously, she was gagging for sex. The window shopping was just a distraction

      --
      Time for bed, said Zebedee - boing
    2. Re:Doug Lenat's Test by nospam007 · · Score: 4, Funny

      Two guys were watching a dog licking his balls.
      First guy says to the second: I'd like to do that too.
      The second guy replies: You'd better pet him first or he might bite you.

  3. Re:Comprehension, M'FR, Do You Read It ?!? by alvinrod · · Score: 3, Insightful

    It's okay in a lot of areas, it's just that no one really gives a shit about the inner city school districts and so they've gone to absolute shit. If you remove that from the equation, the U.S. as a whole is quite comparable to other western democracies. The U.S. has seems more content to let this problem fester and to deal with the consequences rather than tackle it head on so the problem just goes from bad to worse in a lot of ways.

    On a side note, if there weren't so many useless (not as in they suck at their jobs, just useless in that their jobs don't improve educational outcomes in any measurable way) administrators soaking up money, we could pay teachers a hell of a lot more. The U.S. spends more on education as a percentage of GDP than other countries that do as well or better than us, and over time our spending on education as a percentage of GDP has increased. Even though you hear about cutbacks all the time (who pays attention when funding is increased?) the trend has been moving upward over time. So it's not strictly a money problem.

    Here's a good report (PDF warning) that has looked into how public education has changed in the U.S. over time. The increase in administrative staff has done nothing to improve outcomes and removing the excess would allow for an additional ~$11,000 in yearly salary for every teacher.

  4. Re:This says little about AI by djinn6 · · Score: 3, Informative
    The questions are nothing like that. Here's the reading material:

    Packet mode communication may be implemented with or without intermediate forwarding nodes (packet switches or routers). Packets are normally forwarded by intermediate network nodes asynchronously using first-in, first-out buffering, but may be forwarded according to some scheduling discipline for fair queuing, traffic shaping, or for differentiated or guaranteed quality of service, such as weighted fair queuing or leaky bucket. In case of a shared physical medium (such as radio or 10BASE5), the packets may be delivered according to a multiple access scheme.

    And here's the questions:

    How are packets normally forwarded?
    Answer: asynchronously using first-in, first-out buffering, but may be forwarded according to some scheduling discipline for fair queuing

    How is packet mode communication implemented?
    Answer: with or without intermediate forwarding nodes

    In cases of shared physical medium how are they delivered?
    Answer: according to a multiple access scheme

    So the test taker only needs to find a selection of the original text that answers the question.

    The way I see it, the real issue with the "reading comprehension" quiz is that you don't need to actually comprehend the text to answer it. A better question than "How are packets normally forwarded?" would be something like "What are some situations where packets are not forwarded in the fifo order?" The first question only requires you to find the words "packets", "normally" and "forwarded" in the paragraph and answer with the rest of the sentence. The second question requires you to understand that the text is presenting 2 options, one is "normal" and the other isn't.

    There's also some official answers that are just plain incorrect. The answer to "How is packet mode communication implemented?" is the entire rest of the paragraph, not just "with or without intermediate forwarding nodes".