How IBM Plans To Win Jeopardy!

wjousts writes "Technology Review is reporting on IBM's plans to take on Trebek at his own game. The 'Watson' computer system uses natural-language processing techniques to break down questions into their structural components and then search its database for relevant answers. A televised matchup with Trebek is planned for next year. 'David Ferrucci, the IBM computer scientist leading the effort, explains that the system breaks a question into pieces, searches its own databases for "related knowledge," and then finally makes connections to assemble a result. Watson is not designed to search the Web, and IBM's end goal is a system that it can sell to its corporate customers who need to make large quantities of information more accessible.'"

Comments

Dealing with Layered Problems

[eldavojohn]

I wonder how they plan to do with categories that have implications for all the answers. I've seen categories where words must be so many letters in length or perhaps start with certain things and Alex will interject while reading the category such as "'Cats'--and that means all the words in this category start with 'Cat'." Now, with that in mind, a clue could come in as "They are the popular makers of earth moving equipment." Might prompt Watson to find the most popular makers of earth moving equipment--Who is John Deere? The category of 'Cats' would do nothing for Watson without the aid of Alex's interjection ... thus failing at finding "Who is Caterpillar?" (bonus points if you also thought of "Who is Bobcat?" but that answer doesn't start with Cat).

As a fairly avid though novice crossword puzzler, my mind explodes with questions. Could Watson discern a four letter word for "Pleasant French city" (Nice)? Or what about a four letter word for "Beefy Laker" (Kobe)?

Lastly, will Watson have something inane and boring to talk about during the break?

Alex Trebek: Now, Watson, it says here that you are named after Thomas J. Watson who forbade his employees to drink and even frowned upon it while off the job?
Watson: That is correct. It is against IBM regulation 4-245 Section 8 to consume alcohol on the premises of any facility.
Alex Trebek: Fascinating, I'm sure you've never broken that strict regulation, ha ha.
Watson: Good sir, I am a computer, drinking is not within my capacity.
Alex Trebek: Um, right. So could you tell us something interesting about yourself?
Watson: *pauses to search records* During the fabrication of my circuitry, several engineers went months without sleep. Leading one to go insane and killed his wife and kid before taking his own life in a double homicide/suicide case.
Alex Trebek: How unfortunate. Well, I wish you the best of luck today in Jeopardy.
Watson: Thank you, my snide game show master.

Re:Dealing with Layered Problems

[eldavojohn]

Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.

An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don't know.

Yes, there are categories which require the contestant to have an active imagination and it's these categories I wish the article had addressed instead of a vanilla one. And I believe it's these categories that makes Jeopardy fresh and new after decades.

In retrospect, I should have broke out the conversation into a different post so that this wasn't modded +5 Funny. I'm seriously interested in how IBM plans to address things that require the natural speech recognition of Alex Trebek. Does it take into account other answers in the same category to "catch on" like some contestants obviously do?

Then there's the folks running Jeopardy who could pick some categories that would wreck Watson and give the humans the creative advantage. I hope they exploit this creative ability humans have and write an entire category in ... oh, say Pig Latin!

In reality, they stand to have much more to gain if the machine comes close to winning ... as they could make this into an annual competition drawing fans and viewers much like the quest to beat the world chess grand masters.

Re:Dealing with Layered Problems

[eldavojohn]

It's possible that the questions for that particular show will be specifically chosen to be more explicit and less ambiguous ...

Yes, clues like "It's the cube root of 474552" would level the playing field.

Isn't the purpose of this to let Jeopardy be Jeopardy? And see if a computer can compete at what the show is?

Re:Dealing with Layered Problems

[sexconker]

Assume it's a perfect cube.
x^3 is 6 digits, so we're looking at numbers from about 50 to 100.

x^3 = 4XX
6^3 = 216
7^3 = 343
8^3 = 512

70 < x < 80

x^3 ends in an 2, so the cube root must end in an 8.
78.

Seriously though, square roots are easy peasy.
Cube roots let you use the awesome property that:

0 - 0
1 - 1
2 - 8
3 - 7
4 - 4
5 - 5
6 - 6
7 - 3
8 - 2

So you can always figure out the last digit of the cube root of a number VERY easily (no, you don't need to memorize that list).

Then you use the size of the number to get a range, and then estimate. If you're feeling ballsy, you can go for it. Spend the first few seconds (before people buzz in) and get your range down. Then buzz in and spend a couple seconds estimating, then answer (just say "what is..." right when you buzz in). If someone else buzzes in first, more time for you to think.

4th powers are just doing the square root twice.

The list for 5th power roots is neat, too.

0 - 0
1 - 1
2 - 2
3 - 3
4 - 4
5 - 5
6 - 6
7 - 7
8 - 8
9 - 9
0 - 0

Only if...

[weszz]

It can answer in Sean Connery's voice and make your mother jokes at him.

Otherwise I'll probably pass and look up old SNL skits on youtube instead.

Re:Only if...

[SterlingSylver]

So I think IBM's plans here are to
Use a high-tech set of
Computers to create a
Knowledge processor that can be monetized.

I think
That wanting

To use such a
Rediculously advanced
Engineering marvel to make Sean Connery jokes would
Be a waste of
Everone's time, energy, and
Karma

Suck it Trebek!

[Anonymous Coward]

I wonder how well it'll do at Anal bum cover.

Jesus

[eldavojohn]

What was an extra-terrestrial?

How tastelessly incorrect. Extra-terrestrials don't come back to life. Watson would cross reference The Bible with many recent movies and come up with the correct question we were looking for: "What was a zombie?"

Wordplay

[ooutland]

A lot of Jeopardy questions are wordplay-dependent, something AI doesn't have the hang of yet (unless IBM has been toiling in secret on something truly amazing). Categories like "Rhyme Time" and questions like "Qhat does a Pharoah need when he has a cold?" (Answer: an Egyptian Prescription) are beyond the ken of a data search.

Many Jeopardy "answers" have the key to the answer within the question, though in some cases it may be enough to throw the program off. IE in a category like "Musicals" an answer like "Unlike his other hits, this musical wasn't 'the cat's meow' on Broadway." Raw data crunching will pair musicals, Broadway and "cats" but won't know where to go with "unlike." Only an aficionado will know that Andrew Lloyd Weber's "Starlight Express" tanked on Broadway.

So the writers, given any knowledge of the limitations of AI, can set a challenge which will be nearly impossible for current AI to meet. John Henry will live another day.

Re:Why is "Watson" such a popular choice of name?

[grouse]

Because Thomas J. Watson was the man who turned IBM into a global empire, and Thomas J. Watson Jr. brought it into computers. They successively held the top position at IBM for 57 years. So it's a very important name at IBM, and the connection with Sherlock Holmes is serendipitous.