Jeopardy-Playing Supercomputer Beats Humans

An anonymous reader writes "Ok, this was just a practice round. But in a short demonstration today IBM's Jeopardy-playing supercomputer, a whiz by the name of Watson, thoroughly bested two talented human contestants. IBM has been working on this artificial intelligence project for years to prove that a computer can be programmed to understand conversational speech and wordplay. In today's demo, Watson seems to have proved the point: it started out on a roll in the category 'Chicks Dig Me,' about women and archaeology. The real man versus machine face-off (in which the same contestants compete for a $1 million prize) will be taped tomorrow, and aired in February."

Comments

"The pod bay doors cannot be opened."

[MrEricSir]

"What is the mission is too important for me to allow you to jeopardize it?"

Re:Soon, no more call centers

[Carnivorous Vulgaris]

They will give the AI a heavy Indian accent, because it's what callers expect.

A Rising Tide

[bughunter]

Best quote from the article:

Jennings says it’s worth noting that humans built the thing. Whoever wins, we win.

Truly. Although it sounds threatening to some, the practical applications of the natural language parsing technology will ultimately benefit everyone.

Until, that is, you dial your bank's customer service number from a noisy restaurant, and try to talk to Watson to ask him why your Visa was denied.

(Rutter's quote was a nifty Skynet allusion, but its syntax was mangled by the reporter/editor, so it comes in second best.)

So what this is saying...

[TheL0ser]

So the AI triumphed, and they're calling it a huge success? I wonder how the programmers feel about this. Pretty satisfied, I'd imagine.

Re:Soon, no more call centers

[Wyatt Earp]

I worked Comcast call center for a while in '09. No scripts at all, when I started work and asked for something to follow, to figure out what I was expected to say, I was told with a smile "we don't do any scripts here, good luck!".

The only guideline I had was "Get their name and phone number, don't trust the system to give you an accurate phone number."

HUGE amount of secrecy surrounding this

[Faizdog]

So my neighbor works at the IBM facility where this is taking place, but in a completely unrelated function(it's a huge complex with a lot of people). He said that everyone is taking a forced day off on Friday when they will be taping the actual show. There's only going to be a small amount of the very top IBM brass there (supposedly even the head of this facility won't be allowed in). And that this is a HUGE secrecy issue (I'm guessing so that the results aren't leaked before the broadcast date).

My neighbor works with semiconductors and so works with a lot of dangerous chemicals and stuff. According to him, they've all been told to make sure that all their hazardous materials have been safely stored, and that (I have trouble believing this) even the IBM emergency response/hazmat teams have been told that they aren't allowed onsite and not to respond to any alarms that may be issued. That's a fairly dangerous decision if true, I'm doubtful but my neighbor stands by his statement.

Anyhoo, this is a pretty big deal apparently. More so from the Jeapordy people's end I'd guess since I don't think IBM has anything related to this project that they'd be that paranoid about keeping secret.

Re:HUGE amount of secrecy surrounding this

[afidel]

The local fire warden won't care about their secrecy if he gets alerted to a hazmat issue.

Re:HUGE amount of secrecy surrounding this

[Bigbutt]

Sounds like a lame Shadowrun mission.

GM: You discover that a big hush-hush project is underway Friday.

Street Sam: *rolls dice* 15 successes

GM: A little tidbit from the 'net. Emergency teams have been told not to respond to any alarms.

Street Sam: Excellent. A cakewalk. I could do this in my sleep.

[John]

When do they get the question?

[SeattleGameboy]

In the article, they mention that the computer gets the question as text. Does anyone know exactly when the computer receives the question? Does it receive when the human host starts talking or when the human host completes the question? If it is when the host starts speaking, the computer is getting at least several second head-start on the humans.

Re:When do they get the question?

[MozeeToby]

The humans also get the question as text, at the exact same moment Watson does. That's the way it's always worked on Jeopardy!. They see the question as text the same you do when you watch the show on TV. The best competitors read well a head of Trebek and have an answer ready the instant they're allowed to buzz in, which as after Trebek finishes reading the question. Watson has the exact same advantages and disadvantages as any contestant, except that he can read the text basically instantly.

Re:When do they get the question?

[thomst]

In the article, they mention that the computer gets the question as text. Does anyone know exactly when the computer receives the question? Does it receive when the human host starts talking or when the human host completes the question? If it is when the host starts speaking, the computer is getting at least several second head-start on the humans.

It shouldn't matter much, because Jeopardy's rules lock the buzzers out until Alex has finished reading the question - and that lock-out period is determined by a human producer, who sits at a table off-camera, listening to Alex, and who has a button of her own that enables the buzzers.

Should a contestant try to "buzz in" before the producer pushes that button, his/her/its buzzer is locked out for three seconds - and any attempt to buzz in before that penalty period expires locks your buzzer out for an additional three seconds.

So, yeah, Watson may get to begin parsing the whole question a little early, but, typically, the human contestants get to begin working on it while Alex is still in the process of reading it, too - they just have to anticipate how it will end.

Given the speed of silicon vs. wetware, I agree that it will make a difference - but the real question is whether Watson has to determine that the buzzer is enabled by use of a light sensor (human contestants are notified by a ring of lights around the game board - which home viewers never get to see), or whether it gets notified electronically when the enable switch is activated. I say that, because, at least in my own experience, the ability properly to time the use of your buzzer is an enormous factor in whether you'll do well as a contestant or not.

When I was in the contestant pool in 1991, during the taping of the episode before I was chosen to compete, a four-time winner who was just a monster on the buzzer went up against two newbies. One of them, a little man from New Jersey, obviously became more and more frustrated as the game progressed, when he was unable to buzz in against the Monster, who completely dominated the game (the Monster was a word processor from New Mexico who played videogames as a hobby, so it wasn't surprising that his timing with the buzzer was extra-super good to begin with - and he'd had the non-trivial advantage of four previous games in which to hone his timing). Twice during Double Jeopardy, the New Mexico Monster declined to buzz in, which permitted the little man from New Jersey to do so. Both times, the question was a difficult and obscure one, and both times the little man from New Jersey failed to supply the correct answer, so, when Final Jeopardy came around, there was an empty podium where the little man from New Jersey had been, and, predictably, the Monster became a five-time champion.

Boy, was I glad I didn't have to go up against HIM.

Re:Soon, no more call centers

[OrangeTide]

maybe you should RTFA.

the unique challenges it poses to its contestants: the breadth of topics; the puns, metaphors, and slang in the questions; the speed it takes to buzz and answer.

Speech processing that can deal with the context heavy language of Jeopardy is a pretty big test and I think means we're just a little bit closer to a general purpose natural language speech recognition system.

Re:Soon, no more call centers

[coinreturn]

Actually, it doesn't do any speech processing - it does text processing - a HUGE difference. This is a shortcoming in my opinion. It should have to process Alex's voice, not clear text, though I guess it could do OCR on the screen instead. I bet they didn't have any Audio or Video Daily Doubles. In the show, audio and video clues are not uncommon.

Re:Verizon, Fedex already there

[PopeRatzo]

Interactive voice menus are probably singlehandedly the worst innovation in menu systems ever.

But tell me it's not entertaining to listen to someone else dealing with an interactive voice menu. It's a great one-sided conversation:

"Yes"

"Yes"

"Billing"

"I said BILLING!"

"Question about charge."

"Question about charge"

"Problem with charge"

"Jesus Christ. YOU CHARGED ME FOR SOMETHING I DIDN'T BUY!"

"No."

"No.

"NO NO NO NO! YOU FUCKING MACHINE! GIVE ME A HUMAN BEING!" (Followed by insane button mashing)

It's like an old Bob Newhart routine, if Bob Newhart had Tourette's Syndrome and anger management issues.

Re:Soon, no more call centers

[bws111]

Think for a moment about all the stuff you just hand-waved away, and you will begin to appreciate the problem. Let's take your "chicks dig me" example. You correctly identified that "chick" could mean baby chickens or females (it can also mean any young bird or a small child). "Dig" has a bunch of meanings, both as a noun and a verb. "Me" you just brushed off as "no relevance", but there are two problems with that: first, how is a computer supposed to know the word has no relevance, and secondly, it is VERY relevant to the category. Because of the wordplay, "me" is not referring to a person, it is referring to a LOCATION that was EXCAVATED by a FEMALE, so your answer had better either be an archeological site or a female archeologist. And it just keeps getting harder from there.