Slashdot Mirror
How IBM Plans To Win Jeopardy!
wjousts writes "Technology Review is reporting on IBM's plans to take on Trebek at his own game. The 'Watson' computer system uses natural-language processing techniques to break down questions into their structural components and then search its database for relevant answers. A televised matchup with Trebek is planned for next year. 'David Ferrucci, the IBM computer scientist leading the effort, explains that the system breaks a question into pieces, searches its own databases for "related knowledge," and then finally makes connections to assemble a result. Watson is not designed to search the Web, and IBM's end goal is a system that it can sell to its corporate customers who need to make large quantities of information more accessible.'"
I fed all the Jeopardy questions into Wolfram|Alpha and it got every single one right.
Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.
An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don't know.
IBM is laying off American citizens, but hiring in Asia, and yet are spending all this money on gimmicks. This is the kind of thing that gives big companies bad names. Hopefully, as a consolation prize, the laid-off Americans can watch their former company go down in smoke on the game show, hoping it starts smoking and sparking like a cheesy Trek android meltdown.
Table-ized A.I.
Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.
An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don't know.
Yes, there are categories which require the contestant to have an active imagination and it's these categories I wish the article had addressed instead of a vanilla one. And I believe it's these categories that makes Jeopardy fresh and new after decades.
... oh, say Pig Latin!
... as they could make this into an annual competition drawing fans and viewers much like the quest to beat the world chess grand masters.
In retrospect, I should have broke out the conversation into a different post so that this wasn't modded +5 Funny. I'm seriously interested in how IBM plans to address things that require the natural speech recognition of Alex Trebek. Does it take into account other answers in the same category to "catch on" like some contestants obviously do?
Then there's the folks running Jeopardy who could pick some categories that would wreck Watson and give the humans the creative advantage. I hope they exploit this creative ability humans have and write an entire category in
In reality, they stand to have much more to gain if the machine comes close to winning
My work here is dung.
Without speculating on the specifics of tweaking the AI, my guess is that IBM has tried to think through these things. Having put together a few AI bots myself (purely recreationally - you know, just for kicks), I know that I let them play in the real world for quite a while to work out the kinks before unveiling them to nerdy friends and family to show them off and demonstrate just how much time and sleep I'd wasted. My poker-bot played thousands of games in free online rooms before I told anyone that I was even working on him.
IBM has probably been feeding Watson DVR'd episodes for a while now so that they could identify (if not fix) the kind of gotchas that you're thinking about.
He's getting rather old, but he's a good mouse.
Speech recognition?
The machine will be receiving a text file of the question.
Hell, I bet the thing is always the first to the buzzer too.
Alex: "Here to present the Video Daily Double is Harry Mudd, who always lies."
Harry: "I am lying."
Support Right To Repair Legislation.
It may not be... would it hit the buzzer and hope it had time to compute the answer like many people on those shows, or would it wait until it had time to compute, and then ring in only if it has the answer?
Its an automated crossworld puzzle solver. How it works (and my advisor led the project, though I don't work on anything remotely similar) is that it has a large number of solver modules that are each good at a certain kind of clue. One might be really good at looking up famous people based on keywords. Another might be good at... I dunno some other type of crossword clue.
Then each of these modules made lists of possible answers for each clue (subject to length and letter constraints), complete with the confidence they had in various answers.
A central "merger" then collected the candidate answers for each clue from the different modules, and then did lots of tricky search-like algorithms to find a set of answers that seemed the most cohesive.
This system, PROVERB, was at least the best computer system for solving crosswords, at one time, and did fairly well in competitions in which other humans competed too.
With Jeopardy! something similar could approach this issue, as well, except without the added constraint that questions to different answers have to relate to each other on the level of spelling.