Slashdot Mirror

← Back to Stories (view on slashdot.org)

How IBM Plans To Win Jeopardy!

Posted by Soulskill on from the 011010-for-400-alex dept.

wjousts writes "Technology Review is reporting on IBM's plans to take on Trebek at his own game. The 'Watson' computer system uses natural-language processing techniques to break down questions into their structural components and then search its database for relevant answers. A televised matchup with Trebek is planned for next year. 'David Ferrucci, the IBM computer scientist leading the effort, explains that the system breaks a question into pieces, searches its own databases for "related knowledge," and then finally makes connections to assemble a result. Watson is not designed to search the Web, and IBM's end goal is a system that it can sell to its corporate customers who need to make large quantities of information more accessible.'"

36 of 154 comments (clear)

  1. Dealing with Layered Problems by eldavojohn · · Score: 5, Funny

    I wonder how they plan to do with categories that have implications for all the answers. I've seen categories where words must be so many letters in length or perhaps start with certain things and Alex will interject while reading the category such as "'Cats'--and that means all the words in this category start with 'Cat'." Now, with that in mind, a clue could come in as "They are the popular makers of earth moving equipment." Might prompt Watson to find the most popular makers of earth moving equipment--Who is John Deere? The category of 'Cats' would do nothing for Watson without the aid of Alex's interjection ... thus failing at finding "Who is Caterpillar?" (bonus points if you also thought of "Who is Bobcat?" but that answer doesn't start with Cat).

    As a fairly avid though novice crossword puzzler, my mind explodes with questions. Could Watson discern a four letter word for "Pleasant French city" (Nice)? Or what about a four letter word for "Beefy Laker" (Kobe)?

    Lastly, will Watson have something inane and boring to talk about during the break?

    Alex Trebek: Now, Watson, it says here that you are named after Thomas J. Watson who forbade his employees to drink and even frowned upon it while off the job?
    Watson: That is correct. It is against IBM regulation 4-245 Section 8 to consume alcohol on the premises of any facility.
    Alex Trebek: Fascinating, I'm sure you've never broken that strict regulation, ha ha.
    Watson: Good sir, I am a computer, drinking is not within my capacity.
    Alex Trebek: Um, right. So could you tell us something interesting about yourself?
    Watson: *pauses to search records* During the fabrication of my circuitry, several engineers went months without sleep. Leading one to go insane and killed his wife and kid before taking his own life in a double homicide/suicide case.
    Alex Trebek: How unfortunate. Well, I wish you the best of luck today in Jeopardy.
    Watson: Thank you, my snide game show master.

    --
    My work here is dung.
    1. Re:Dealing with Layered Problems by wjousts · · Score: 2, Interesting

      Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.

      An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don't know.

    2. Re:Dealing with Layered Problems by eldavojohn · · Score: 5, Interesting

      Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.

      An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don't know.

      Yes, there are categories which require the contestant to have an active imagination and it's these categories I wish the article had addressed instead of a vanilla one. And I believe it's these categories that makes Jeopardy fresh and new after decades.

      In retrospect, I should have broke out the conversation into a different post so that this wasn't modded +5 Funny. I'm seriously interested in how IBM plans to address things that require the natural speech recognition of Alex Trebek. Does it take into account other answers in the same category to "catch on" like some contestants obviously do?

      Then there's the folks running Jeopardy who could pick some categories that would wreck Watson and give the humans the creative advantage. I hope they exploit this creative ability humans have and write an entire category in ... oh, say Pig Latin!

      In reality, they stand to have much more to gain if the machine comes close to winning ... as they could make this into an annual competition drawing fans and viewers much like the quest to beat the world chess grand masters.

      --
      My work here is dung.
    3. Re:Dealing with Layered Problems by Ethanol-fueled · · Score: 3, Insightful

      It's possible that the questions for that particular show will be specifically chosen to be more explicit and less ambiguous (avoiding the show's characteristic punny wordplay) to put the machine on a more level playing field, keeping its score closer to those of the contestants', which will make the episode more exciting to watch.

    4. Re:Dealing with Layered Problems by eldavojohn · · Score: 5, Insightful

      It's possible that the questions for that particular show will be specifically chosen to be more explicit and less ambiguous ...

      Yes, clues like "It's the cube root of 474552" would level the playing field.

      Isn't the purpose of this to let Jeopardy be Jeopardy? And see if a computer can compete at what the show is?

      --
      My work here is dung.
    5. Re:Dealing with Layered Problems by gnick · · Score: 4, Interesting

      Without speculating on the specifics of tweaking the AI, my guess is that IBM has tried to think through these things. Having put together a few AI bots myself (purely recreationally - you know, just for kicks), I know that I let them play in the real world for quite a while to work out the kinks before unveiling them to nerdy friends and family to show them off and demonstrate just how much time and sleep I'd wasted. My poker-bot played thousands of games in free online rooms before I told anyone that I was even working on him.

      IBM has probably been feeding Watson DVR'd episodes for a while now so that they could identify (if not fix) the kind of gotchas that you're thinking about.

      --
      He's getting rather old, but he's a good mouse.
    6. Re:Dealing with Layered Problems by sexconker · · Score: 2, Interesting

      Speech recognition?
      The machine will be receiving a text file of the question.

      Hell, I bet the thing is always the first to the buzzer too.

    7. Re:Dealing with Layered Problems by weszz · · Score: 3, Interesting

      It may not be... would it hit the buzzer and hope it had time to compute the answer like many people on those shows, or would it wait until it had time to compute, and then ring in only if it has the answer?

    8. Re:Dealing with Layered Problems by sexconker · · Score: 5, Informative

      Assume it's a perfect cube.
      x^3 is 6 digits, so we're looking at numbers from about 50 to 100.

      x^3 = 4XX
      6^3 = 216
      7^3 = 343
      8^3 = 512

      70 < x < 80

      x^3 ends in an 2, so the cube root must end in an 8.
      78.

      Seriously though, square roots are easy peasy.
      Cube roots let you use the awesome property that:

      0 - 0
      1 - 1
      2 - 8
      3 - 7
      4 - 4
      5 - 5
      6 - 6
      7 - 3
      8 - 2

      So you can always figure out the last digit of the cube root of a number VERY easily (no, you don't need to memorize that list).

      Then you use the size of the number to get a range, and then estimate. If you're feeling ballsy, you can go for it. Spend the first few seconds (before people buzz in) and get your range down. Then buzz in and spend a couple seconds estimating, then answer (just say "what is..." right when you buzz in). If someone else buzzes in first, more time for you to think.

      4th powers are just doing the square root twice.

      The list for 5th power roots is neat, too.

      0 - 0
      1 - 1
      2 - 2
      3 - 3
      4 - 4
      5 - 5
      6 - 6
      7 - 7
      8 - 8
      9 - 9
      0 - 0

    9. Re:Dealing with Layered Problems by skelterjohn · · Score: 2, Interesting
      Since you bring up crosswords as an example of this sort of issue, let me point you to http://www.oneacross.com/proverb/

      Its an automated crossworld puzzle solver. How it works (and my advisor led the project, though I don't work on anything remotely similar) is that it has a large number of solver modules that are each good at a certain kind of clue. One might be really good at looking up famous people based on keywords. Another might be good at... I dunno some other type of crossword clue.

      Then each of these modules made lists of possible answers for each clue (subject to length and letter constraints), complete with the confidence they had in various answers.

      A central "merger" then collected the candidate answers for each clue from the different modules, and then did lots of tricky search-like algorithms to find a set of answers that seemed the most cohesive.

      This system, PROVERB, was at least the best computer system for solving crosswords, at one time, and did fairly well in competitions in which other humans competed too.

      With Jeopardy! something similar could approach this issue, as well, except without the added constraint that questions to different answers have to relate to each other on the level of spelling.

  2. A test with Wolfram|Alpha by Anonymous Coward · · Score: 2, Interesting

    I fed all the Jeopardy questions into Wolfram|Alpha and it got every single one right.

  3. Jesus by BadAnalogyGuy · · Score: 2, Funny

    What was an extra-terrestrial?

    1. Re:Jesus by treeves · · Score: 2

      I know you were going for funny, but in reality Jeopardy never uses one(or two)-word answers. There are too many possible questions for a one-word answer.

      Example:
      A: Caesar

      Q: Who crossed the Rubicon and said 'the die is cast'?
      Q: Who said 'Et tu Brute?' when he was assassinated?
      Q: Which Roman emperor adopted his great nephew Octavian who later became Augustus Caesar?
      Q: What salad is made with Romaine lettuce, anchovies, garlic and lemon?
      etc. etc.

      Instead you use the long answer and get a short question:

      A: This Roman emperor adopted his great nephew Octavian who later became Augustus Caesar.
      Q: Who is Julius Caesar?

      --
      ...the future crusty old bastards are already drinking the Kool-Aid.
  4. Only if... by weszz · · Score: 5, Funny

    It can answer in Sean Connery's voice and make your mother jokes at him.

    Otherwise I'll probably pass and look up old SNL skits on youtube instead.

    1. Re:Only if... by SterlingSylver · · Score: 5, Funny

      So I think IBM's plans here are to
      Use a high-tech set of
      Computers to create a
      Knowledge processor that can be monetized.

      I think
      That wanting

      To use such a
      Rediculously advanced
      Engineering marvel to make Sean Connery jokes would
      Be a waste of
      Everone's time, energy, and
      Karma

    2. Re:Only if... by Ogive17 · · Score: 2, Funny

      IBM> You'll rue the day you crossed me Trebek!

      --
      "Action without philosophy is a lethal weapon; philosophy without action is worthless."
  5. Suck it Trebek! by Anonymous Coward · · Score: 5, Funny

    I wonder how well it'll do at Anal bum cover.

    1. Re:Suck it Trebek! by Volante3192 · · Score: 4, Funny

      Clearly, some mod has never seen SNL.

      (That's 'An Album Cover', Connery!)

    2. Re:Suck it Trebek! by weszz · · Score: 2, Funny

      or the other classics...

      Connery - "I'll take the PEN IS mightier for 300..."
      Trebek - "That's the Pen Is Mightier..."

      Connery - "I'll take Famous Tities for 500"
      Trebek - "That's Famous Titles..."

      It's not so easy when you don't have the answers to look at is it Trebek?

    3. Re:Suck it Trebek! by 0100010001010011 · · Score: 2, Informative

      Thank Goodness for Hulu (instead of finding dead Youtube Links)

      The wiki entry is also pretty good: http://en.wikipedia.org/wiki/Celebrity_Jeopardy!_(Saturday_Night_Live)

      "Catch These Men" for "Catch the Semen".
      "Things Trebek Sucks" over the actual category, "Potpourri".

  6. I get it now... by click2005 · · Score: 4, Funny

    sell to its corporate customers who need to make large quantities of information more accessible.'"

    They want to replace the call centres in India with call computers.

    "Hello you're speaking to Susan Blue Gene how can I help you?"

    --
    I am a free slashdotter. I will not be modded, blogged, DRM'd, patented, podcasted or RFID'd. My life is my own.
    1. Re:I get it now... by Pie+Pan · · Score: 3, Funny

      I'm suddenly reminded of this: http://www.penny-arcade.com/comic/2000/01/28/

    2. Re:I get it now... by StikyPad · · Score: 2, Funny

      Ah, but how are they to be mimicking the bizarre structure of grammar I am having to become accustomed to?

      --
      https://www.eff.org/https-everywhere
  7. Jesus by eldavojohn · · Score: 5, Funny

    What was an extra-terrestrial?

    How tastelessly incorrect. Extra-terrestrials don't come back to life. Watson would cross reference The Bible with many recent movies and come up with the correct question we were looking for: "What was a zombie?"

    --
    My work here is dung.
  8. Wordplay by ooutland · · Score: 5, Insightful

    A lot of Jeopardy questions are wordplay-dependent, something AI doesn't have the hang of yet (unless IBM has been toiling in secret on something truly amazing). Categories like "Rhyme Time" and questions like "Qhat does a Pharoah need when he has a cold?" (Answer: an Egyptian Prescription) are beyond the ken of a data search.

    Many Jeopardy "answers" have the key to the answer within the question, though in some cases it may be enough to throw the program off. IE in a category like "Musicals" an answer like "Unlike his other hits, this musical wasn't 'the cat's meow' on Broadway." Raw data crunching will pair musicals, Broadway and "cats" but won't know where to go with "unlike." Only an aficionado will know that Andrew Lloyd Weber's "Starlight Express" tanked on Broadway.

    So the writers, given any knowledge of the limitations of AI, can set a challenge which will be nearly impossible for current AI to meet. John Henry will live another day.

    --
    I'm the queer the atheists sent here to take away your gun!
    1. Re:Wordplay by Another,+completely · · Score: 2, Informative

      Sorry, I used information from outside the original article without a citation. This is from the team's web info:

      ... just like human competitors, Watson will not be connected to the Internet or have any other outside assistance.

  9. Waste by Tablizer · · Score: 2, Interesting

    IBM is laying off American citizens, but hiring in Asia, and yet are spending all this money on gimmicks. This is the kind of thing that gives big companies bad names. Hopefully, as a consolation prize, the laid-off Americans can watch their former company go down in smoke on the game show, hoping it starts smoking and sparking like a cheesy Trek android meltdown.

    --
    Table-ized A.I.
    1. Re:Waste by wjousts · · Score: 2, Interesting

      It's not a gimmick. It is very important research on AI and natural-language processing. Jeopardy! just happens to give them a very difficult problem to tackle. If they can develop a system that can handle Jeopardy!, it'll be a huge break through for other fields.

    2. Re:Waste by glwtta · · Score: 4, Informative

      They're spending money on research, gimmicks just help pay for it.

      --
      sic transit gloria mundi
  10. CATS MEANING ALL RESPONSES START WITH CAT by tepples · · Score: 4, Insightful

    Alex will interject while reading the category such as "'Cats'--and that means all the words in this category start with 'Cat'."

    Then the bot would read the closed caption that the category is "CATS MEANING ALL RESPONSES HAVE A WORD THAT STARTS WITH CAT" and include that in its reasoning. Then the clue "They are the popular makers of earth moving equipment" becomes something like "They are the popular makers of earth moving equipment, starting with 'CAT'".

  11. Bad editors! by sootman · · Score: 4, Funny

    The summary clearly should have been titled "How does IBM plan to win Jeopardy?"

    --
    Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
  12. Re:Why is "Watson" such a popular choice of name? by grouse · · Score: 5, Informative

    Because Thomas J. Watson was the man who turned IBM into a global empire, and Thomas J. Watson Jr. brought it into computers. They successively held the top position at IBM for 57 years. So it's a very important name at IBM, and the connection with Sherlock Holmes is serendipitous.

  13. I want to see how it responds to this question... by PJ1216 · · Score: 3, Funny

    What is the answer to life, the universe, and everything?

  14. Video Daily Double by Comboman · · Score: 4, Interesting

    hoping it starts smoking and sparking like a cheesy Trek android meltdown.

    Alex: "Here to present the Video Daily Double is Harry Mudd, who always lies."

    Harry: "I am lying."

    --
    Support Right To Repair Legislation.
  15. Re:I want to see how it responds to this question. by Bigjeff5 · · Score: 2, Insightful

    Well, we already know that, it's 42.

    The real question, is what is the real question for which 42 is the answer? That one is the tough one.

    I suggest we build a planet, who's sole purpose is to calculate that question...

    --
    Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
  16. Comment removed by account_deleted · · Score: 3, Funny

    Comment removed based on user account deletion