Slashdot Mirror


IBM Computer Program To Take On 'Jeopardy!'

longacre writes "I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward. ... The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly. Such a program would have enormous economic implications. ... The proposed contest is an effort by I.B.M. to prove that its researchers can make significant technical progress by picking "grand challenges" like its early chess foray. The new bid is based on three years of work by a team that has grown to 20 experts in fields like natural language processing, machine learning and information retrieval. ... Under the rules of the match that the company has negotiated with the 'Jeopardy!' producers, the computer will not have to emulate all human qualities. It will receive questions as electronic text. The human contestants will both see the text of each question and hear it spoken by the show's host, Alex Trebek. ... Mr. Friedman added that they were also thinking about whom the human contestants should be and were considering inviting Ken Jennings, the 'Jeopardy!' contestant who won 74 consecutive times and collected $2.52 million in 2004."

19 of 213 comments (clear)

  1. Leap Forward? by ShieldW0lf · · Score: 3, Insightful

    I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward.

    In what way would this be a leap forward? Looking up trivial facts in a database and spitting them out is easy, and not particularly significant...

    --
    -1 Uncomfortable Truth
    1. Re:Leap Forward? by Red+Flayer · · Score: 3, Insightful

      Did you even read the summary?

      The leap forward is not in being able to look up facts in a database, it is in being able to interpret written questions properly.

      There's a lot involved in interpreting natural language, and so far computers have been a far cry from being able to do it well. It says something that these algorithms are being tested against Jeopardy answers, since those are not completely natural language either -- they've been screened and edited to remove ambiguity.

      --
      "Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
    2. Re:Leap Forward? by derGoldstein · · Score: 5, Insightful

      Doubly so since the clues and questions in a Jeopardy! game are usually at least somewhat obfuscated, contain puns, double entendres, etc...

      This is exactly why this sounds so implausible to me. You often have to take the category name and weave it in with the question (or rather, answer). A lot depends not on the knowledge, but on the phrasing of the "queries". Give me one example of translation software which can translate entire paragraphs well.

      It makes me wonder how much "stress testing" they've done, by taking old Jeopardy questions and seeing if the output would be considered "correct" by a human arbiter.

      --
      Entomologically speaking, the spider is not a bug, it's a feature.
    3. Re:Leap Forward? by Bandman · · Score: 4, Insightful

      The step forward will be parsing the english language.

      I hope it remembers to phrase its answers in the form of a question.

    4. Re:Leap Forward? by clickety6 · · Score: 2, Insightful

      Parsing the questions in natural language,

      Natural language?

      Outside of a Jeopardy! gameshow, I have never heard anybody use the this type of phrasing.

      "This is a reason for you not handing in your homework Johnny"

      "Why is because my dog ate it, sir"

      Yeah, sounds very natural :-)

      --
      ----------------------------------- My Other Sig Is Hilarious -----------------------------------
    5. Re:Leap Forward? by Bandman · · Score: 2, Insightful

      That's the sort of thing that makes me believe that this team may be able to succeed.

      When Deep Blue went up against Kasperov, who could it practice against? Nobody.

      There are tens of thousands of Jeopardy! questions to go through before they start making up their own.

    6. Re:Leap Forward? by lelitsch · · Score: 2, Insightful

      When Deep Blue went up against Kasperov, who could it practice against? Nobody.

      That this got modded Insightful is the best argument yet for adding tags to /.

    7. Re:Leap Forward? by damburger · · Score: 5, Insightful

      Being able to beat a human at Jeapordy is a fairly substantial subset of the Turing test sorted.

      Natural language processing is an absolute and total bitch - take it someone who has studied it. One of my AI professors once explained it to me such; the human brain tricks you into believing the hardest tasks it accomplishes are the easiest. Stuff like language, walking, and so on take up far much more of your neural hardware than what you would consider 'thinking' - but it all happens subconsciously.

      No, it isn't Artificial Intelligence per se - there is no real 'understanging' or 'intelligence' behind it -but it is a very serious technical challenge. There is a lot more to it than simply dumping Jeopardy questions into a standard search engines.

      Don't take my word for it. Load up your favourite editor or IDE and start coding a simple chat bot. The difficulties that IBM must have overcome are best discovered through experience.

      --
      If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
    8. Re:Leap Forward? by damburger · · Score: 2, Insightful

      Yup. Natural language processing is a piece of piss until you actually *try* it. Along similar lines, I once heard an anecdote that computer vision was first attempted when it was given to a graduate student whilst all the professional researchers were busy with board games. It hadn't occurred to anybody it might be difficult.

      --
      If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
    9. Re:Leap Forward? by Anonymous Coward · · Score: 1, Insightful

      Computer: I'll take this has been done before in the 60s for $2000, Alex.

      Alex: And here's the clue: It makes little sense to parse English when this technique can be used instead, at a lower cost of development and higher success rate.

      Computer: What is anchoring the words "this/that" in the clue to obtain the "type" of the response (i.e "this author", "this composer", "this country"), and reply with the database entry with the highest correlation to other words in the clue.

      Alex: Correct. We'll be back after this messages.

      p.s. I'd like to see the computer solve one of the video daily doubles.

    10. Re:Leap Forward? by derGoldstein · · Score: 2, Insightful

      You don't need a human to determine who won a chess match. Winning is absolute.

      You need a human to judge if the answer to a question, its phrasing, and its context were correct.

      --
      Entomologically speaking, the spider is not a bug, it's a feature.
  2. I'd take Jennings by Paul+Pierce · · Score: 4, Insightful

    That guy will beat anyone.

    The problem they might run into is the speed of pressing the button to respond. I would imagine the computer would be able to beat the human every time it knew the answer.

    1. Re:I'd take Jennings by OwnedByTwoCats · · Score: 2, Insightful

      It would be perfectly fair. If it doesn't come up with the answer, it loses money. Same rule the human players are under.

  3. Comment removed by account_deleted · · Score: 4, Insightful

    Comment removed based on user account deletion

  4. Comment removed by account_deleted · · Score: 4, Insightful

    Comment removed based on user account deletion

  5. Re:Jeopardy really that challenging? by thesandtiger · · Score: 2, Insightful

    Actually, yes, it is. See, it's not just general knowledge, but, as about 9 billion other people in this thread pointed out, there are puns and other wordplay often involved.

    What do you think the proper Jeopardy answer to this question is (in the category "Much Ado" for $100):

    "It's the spirit that gets things done."

    Answer: What is "can do"

    The $500 version might be something like, "This recently hip-again party favorite was first created in New York."

    Answer: What is "fondue"

    Both of those are pretty easy examples. Both require the computer to "get" the wordplay in the topic (aDO), one requires that it understand that pronunciation of the word in the answer is the key, and numerous other things that I probably take for granted but are rather non-trivial things for a computer to do.

    If they can get the machine doing reasonably well on those kinds of questions for Jeopardy, I'd love to see them go after "You Don't Know Jack" next. I've seen many reasonably bright people completely unable to handle that game when it came to the punny or obfuscated questions.

    --
    Since I can't tell them apart, I treat all ACs as the same person.
  6. Can I have theatrics for $2000, Alex? by code65536 · · Score: 2, Insightful

    So... why Jeopardy? IBM is trying to demonstrate software that can parse text for meaning. That's great. But there are plenty of other places/formats/etc. that you can demonstrate this technology. There are certainly far more useful applications of this sort of technology.

    I'm guessing that they they are going after J! because...
    1) The warm spotlight of a well-known TV show
    2) There is still a lot of structure and form on J! that it's easier to achieve "success" than if they had the machine do something more free-form... e.g., read a novel and generate a plot summary or, heavens forbid, actually understand real human conversation
    3) The computer could have other advantages, like impeccable buzzer timing (which is sometimes more important than actual knowledge, especially in the Tournament of Champions) and having memorized the material beforehand (the NYT indicates that it would have "read" study materials before the match), which also helps increase the likelihood of "success"

    And to pile on the criticism of grandstanding, the machine will be fed electronic text. So no video camera to perform text recognition? No speech recognition (IBM afraid of the "wreck a nice beach" vs. "recognize speech" problem tripping up their theatrics?). And what use would this be? At least the AI text research done at Google is being put to good use, like improving their machine translation services. Aside from getting IBM's name plastered in the media, what exactly is this going to do?

  7. Re:Not AI just Google with a filter by damburger · · Score: 3, Insightful

    No argument will show you how wrong you are - please just try to code something similar. Or even a chat bot. Things like language processing and image recognition seem easy to humans because most of your brains activity is hidden from your conscious mind. Try making a computer do these things, and you will discover that what IBM is nontrivial.

    Neither was selling calculating equipment to the Nazis either, like you said.

    --
    If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
  8. Comment removed by account_deleted · · Score: 4, Insightful

    Comment removed based on user account deletion