Jeopardy-Playing Supercomputer Beats Humans

An anonymous reader writes "Ok, this was just a practice round. But in a short demonstration today IBM's Jeopardy-playing supercomputer, a whiz by the name of Watson, thoroughly bested two talented human contestants. IBM has been working on this artificial intelligence project for years to prove that a computer can be programmed to understand conversational speech and wordplay. In today's demo, Watson seems to have proved the point: it started out on a roll in the category 'Chicks Dig Me,' about women and archaeology. The real man versus machine face-off (in which the same contestants compete for a $1 million prize) will be taped tomorrow, and aired in February."

Comments

Soon, no more call centers

[Animats]

Probably already smarter than the average call center employee.

Re:Soon, no more call centers

[Carnivorous Vulgaris]

They will give the AI a heavy Indian accent, because it's what callers expect.

Re:Soon, no more call centers

[Anonymous Coward]

Call center employees aren't allowed to be smart. They have scripts that they must follow. They're reduced to a very simple algorithm, executed by human beings only because there are still people who prefer talking to a other people over interacting with a machine, and because speech recognition software is still not ready to deal with what some people call speech.

Re:Soon, no more call centers

[thisisauniqueid]

And... cue Slumdog Millionaire jokes...

Re:Soon, no more call centers

[mcgrew]

I didn't rtfa, but from the summary I'm saying "huh? So what?"

All this thing needs to beat humans at Jeopardy is a huge database of phrases and a fast search engine. Sounds trivial to me.

Maybe I should RTFA...

Re:Soon, no more call centers

[Wyatt Earp]

I worked Comcast call center for a while in '09. No scripts at all, when I started work and asked for something to follow, to figure out what I was expected to say, I was told with a smile "we don't do any scripts here, good luck!".

The only guideline I had was "Get their name and phone number, don't trust the system to give you an accurate phone number."

Re:Soon, no more call centers

[OrangeTide]

maybe you should RTFA.

the unique challenges it poses to its contestants: the breadth of topics; the puns, metaphors, and slang in the questions; the speed it takes to buzz and answer.

Speech processing that can deal with the context heavy language of Jeopardy is a pretty big test and I think means we're just a little bit closer to a general purpose natural language speech recognition system.

Re:Soon, no more call centers

[mcgrew]

and because speech recognition software is still not ready to deal with what some people call speech.

That's the least of SR's problems; I have trouble understanding anybody from the NE seaboard. I played around with Win 7's speech recognition when I had that netbook, and I was impressed -- when the room was quiet. If a car went by outside it messed up, let alone having a radio or a TV on.

Its biggest problem is its lack of humans' ease with which we pick out a single voice in a roomful of conversational babble.

Re:Soon, no more call centers

[Anonymous Coward]

Jeopardy is not quite that simple. There are some questions where you need to understand the question at a deeper level, like the before-and-after categorys (i.e. Abraham Lincoln Towncar). I think the success or failure of Watson depends on the categories used. I also wonder if the robot fingers will beat Ken on the buzzer, he is a master at getting buzzed in first.

Re:Soon, no more call centers

[coinreturn]

Actually, it doesn't do any speech processing - it does text processing - a HUGE difference. This is a shortcoming in my opinion. It should have to process Alex's voice, not clear text, though I guess it could do OCR on the screen instead. I bet they didn't have any Audio or Video Daily Doubles. In the show, audio and video clues are not uncommon.

Re:Soon, no more call centers

[natehoy]

The article wouldn't be sufficient in laying out the challenges involved in doing this, though it does give one example of the early problems they ran into.

The problem isn't necessarily in finding the answers to a given set of criteria, it's interpreting the question to lay out the search criteria. I don't know if you've ever watched an episode of "Jeopardy", but obtaining the answers often requires decoding a pun or a riddle, not just "It's what you get when you multiply six times nine." or "He is buried in the largest of the Pyramids in Egypt."

I'd have been far more interested to see a list of the questions to see if any of them contained the usual misleading portions and/or puns.

Re:Soon, no more call centers

[jeffmeden]

It doesn't get access to a search engine, though. It needs to use pre-compiled repositories of information, all indexed in a way that makes identifying the right "Response" (remember the questions are answers) in the right amount of time.

That being said, a computer is almost certainly so much better at "hitting the buzzer" in the allowed window than a human, that it possesses a significant advantage from the start.

Re:Soon, no more call centers

[jeffmeden]

There are some questions where you need to understand the question at a deeper level, like the before-and-after categorys (i.e. Abraham Lincoln Towncar).

You must be thinking of Wheel of Fortune... While its not impossible to see a response like that in Jeopardy, it is much more commonly seen in the following time slot.

Re:Soon, no more call centers

[parliboy]

Except that 1) Before And After is also a category on Jeopardy and 2) it was a category in run-throughs for Watson:

In a test tourney, Watson hit the bullseye on a question about clothing a young girl might wear on an operatic ship. The answer, pinafore, is also found in the title of the Gilbert & Sullivan opera H.M.S. Pinafore. And the computer was also successful with a before-and-after Jeopardy question about a candy bar and a Supreme Court justice, Baby Ruth Bader-Ginsberg. But earlier in its career, when asked, "What does a grasshopper eat?", it responded, "Kosher."

Source: http://www.philly.com/philly/blogs/storm/Jeopardy_A_Computer_Takes_on_Ken_Jennings_and_Brad_Rutter_.html

Re:Soon, no more call centers

[bws111]

Hitting the buzzer quickly is only an advantage if you are correct. If you are incorrect, or don't have any response, you lose money. Therefore, the computer (like the humans) must first determine what the response should be, and how confident it is that the response is correct, before buzzing in. This can actually be a disadvantage to the computer, because a human may get a category that he considers himself an expert in and buzz in immediately for all the answers in that category, and then use the remaining time to actually come up with the question.

Re:Soon, no more call centers

[AchilleTalon]

Yes, definitely, you should RTFA.

This is very exciting news and for now, forget replacement of call center peoples, this machine is composed of 2800 Power7 cores, which renders it very expensive compare to the typical call center person. But this accomplishement, is a major step in the AI field and open it to many many exciting applications in the future which is not now too far.

Re:Soon, no more call centers

[splerdu]

You forgot it's also probably a LOT more expensive than the average call center employee.

Re:Soon, no more call centers

[garyebickford]

when asked, "What does a grasshopper eat?", it responded, "Kosher."

So it has a sense of humor as well! :D All it has to do is add, "I'll be here all week. Try the fish."

Re:Soon, no more call centers

[Anonymous Coward]

Don't forget the buzzer. It must have a hand that presses the buzzer. I've seen smart people on that show get shut out because they can't use the buzzer correctly.

Re:Soon, no more call centers

[Monkeedude1212]

maybe you should RTFA.

the unique challenges it poses to its contestants: the breadth of topics; the puns, metaphors, and slang in the questions; the speed it takes to buzz and answer.

Speech processing that can deal with the context heavy language of Jeopardy is a pretty big test and I think means we're just a little bit closer to a general purpose natural language speech recognition system.

But that's just it - maybe we THINK puns and metaphors and slang are more complex than they really are. Keep in mind its not having to worry about inflections in Alex's voice or anything complex with audio, its just reading the text.

With that in mind, how hard is it to apply Meta-tags to data with their relevance? For example, they use "Chicks Dig Me". It will likely check Chicks as prominantly baby chickens, and secondarily the female association, and so on and so forth. Then "Dig" will likely be the act of digging, then the act of liking something, etc etc. "Me" is almost a throw away word because it has no relevance, but regardless it obviously means one self, and has almost no other related speech, except for maybe Windows Millenium edition, which I doubt they would have a whole category on.

With that in mind, it goes through its database and organizes indexes based on Answers to do with Digging, Liking, Chickens, and Females. The puzzle comes up, "I did this" - it then cross references key words to the answers its indexed.

Like the poster above you said, it's just a fast search engine being able to work with keywords, like Google and it's search terms. We only THINK language is complex because we use it so. Jeopardy is funny in that it likes to pretend the puns have meaning, but they really don't. They are puns to keep the show slightly humourus. The questions always have more to do with their direct literal context than they do with the pun.

In that regard, I expect a human to be thrown off by a pun and a computer to do amazingly well.

Re:Soon, no more call centers

[jeffmeden]

I suspect that the computer has a dead on confidence rating by the time Alex is finishing his last syllable, so the decision of whether or not to ring in is clear. Just like a human, but the computer has the advantage of knowing that as soon as the buzzer is "open for buzzing" it can ring in within a millisecond instead of the tens or hundreds it takes a human to register sensory information and command their thumb to press.

You can really tell the difference between a player that's good or bad at buzzing... Some players mash the buzzer vigorously but never seem to be the first one in regardless of intent. It's a skill that presents a very clear advantage.

Re:Soon, no more call centers

[timeOday]

That is disappointing. My other disappointment is that this will be taped and therefore perhaps edited before airing. This lowers the bar from "working every time" to "working sometimes." You roboticists know what I mean :)

Re:Soon, no more call centers

[Machtyn]

I worked the call center for my local cable provider on the Internet provider side. Likewise, I had no script, other than how to get the customer's information. It was fun to have a customer call in and tell me "I know more than you!" It was quite satisfying when they actually proved they knew as much as I did (because, then, I didn't have to put them in their place and we could converse intelligently about their problem.) It was more frustrating when they say they know more than me and I had to explain how to use the command line to run a ping.

Re:Soon, no more call centers

[BagOBones]

Engadget has a video of the match that includes the visualization of the response... In several cases it did not have an answer by the time the other contestant buzzed in and in some cases it was close to a tie.

In an interview it was stated it takes about 3 seconds on average to answer a question, which is actually kinda long when some of the humans will be able to predict the answer as soon as they have read or heard only part of the clue.

Re:Soon, no more call centers

[Jerf]

Based on the previous articles, if the machine had to do either of the things you're talking about, it would simply lose. It is basically just fast enough to compete at Jeopardy and to have a chance it needs the advantage of having text fed to it and having a couple of seconds to chew on it before it can answer. And it is still possible for even normal humans to flat-out beat it to the punch on some questions.

Re:Soon, no more call centers

[seven of five]

But you can't talk to it, you'd have to type. So maybe a chat support tech...

Re:Soon, no more call centers

[catmistake]

prove that a computer can be programmed to understand

Never. Parse? sure... Interpret? why not? But understand? No. A computer will never understand.

Re:Soon, no more call centers

[rusl]

I agree with you. I think this is more for marketing than science. The old chess contests were closer to actual research projects IMHO. I am not a big Jeopardy follower but I can't see how Jeopardy language gets very complicated since the question/answer format is more automated than human.

Re:Soon, no more call centers

[SanityInAnarchy]

it's just a fast search engine being able to work with keywords, like Google and it's search terms.

I thought exactly that. Then I watched some of their promotional videos. It's harder than that. Look at 1:55 to or so -- keywords only really help you find what you're looking for once you actually know what you're looking for.

Now, ultimately, there's going to be a lot of that, and a lot of statistical analysis, but it's still not an easy problem.

Re:Soon, no more call centers

[HarvardAce]

That's funny, because I called Cablevision the other day to resolve an issue where my cable modem wasn't getting an IP address assigned to it, and I was routed to a completely automated system. The beginning line was "You may ask to speak to an agent at any time, but they will be following the same script that I will be following." I gave it about 3 minutes through the script before I moved to the agent because I figured I could get escalated to a tech faster with a real person than a scripted machine.

Re:Soon, no more call centers

[bws111]

Think for a moment about all the stuff you just hand-waved away, and you will begin to appreciate the problem. Let's take your "chicks dig me" example. You correctly identified that "chick" could mean baby chickens or females (it can also mean any young bird or a small child). "Dig" has a bunch of meanings, both as a noun and a verb. "Me" you just brushed off as "no relevance", but there are two problems with that: first, how is a computer supposed to know the word has no relevance, and secondly, it is VERY relevant to the category. Because of the wordplay, "me" is not referring to a person, it is referring to a LOCATION that was EXCAVATED by a FEMALE, so your answer had better either be an archeological site or a female archeologist. And it just keeps getting harder from there.

Re:Soon, no more call centers

[Obfuscant]

And it is still possible for even normal humans to flat-out beat it to the punch on some questions.

If you mean beat humans to the buzzer, don't be silly. The machine will have much faster reflexes than any human. It could press its virtual button within microseconds of the allowed time. Even if it doesn't have a connection to the "ok to buzz in now" signal, it can press its virtual button at 10, 100 or even 1000 times a second, compared to the three or four times a second a human can press his.

It can then use the entire several seconds allowed for the human to vocalize the question to find the right question and display it.

And, if nothing else, since the system is run by computer, it could simply tap into the existing network and get the questions before they are read. It wouldn't even take $64,000...

Of course, we'll probably find out in a few years that it was really Ken Jennings hiding in the box. You never saw them both at the same time, did you?

Re:Soon, no more call centers

[yotto]

Then it shouldn't be playing Jeopardy against humans. The simple fact that it throws the timing of the buzzers off makes this "test" a complete failure. How do they ensure the computer has the same amount of time as Ken Jennings?

This is no longer a test of computer prowess, and instead an advertisement for Jeopardy! Not that it wasn't before. Just that's all it is now.

Re:Soon, no more call centers

[Obfuscant]

Therefore, the computer (like the humans) must first determine what the response should be, and how confident it is that the response is correct, before buzzing in.

In all but the most obscure questions, this is a losing tactic.

For most of the answers, it is a race. He who is fastest at the buzzer wins. You can see this many times as two of the three are clicking their buzzer as fast as their little fingers can.

On middling-hard questions, you still see people racing to buzz in and then take the full amount of time to think of the answer.

Were the contestants truly coming up with an answer prior to buzzing in, you'd never see them buzz in and then go silent. Yes, there are times when nobody buzzes in, but those are the exception and not the rule.

Re:Soon, no more call centers

[Anonymous Coward]

Half of the nuances of jeopardy are homonyms or "quoted" homographs. If the contestants just had speech processing to rely upon the game would be almost impossible, they read the text just like the computer.

Re:Soon, no more call centers

[spazdor]

"TELL US HOW YOU CHEATED." "Okay fine, there's a chess-playing Turk concealed inside the cabinet."

Re:Soon, no more call centers

[DavidTC]

I don't know why it gets any special advantages at all.

It should have to physically push a buzzer. And write out the daily double. I'm okay with it being bolted into place or something, and it doesn't have to entirely fit behind the podium, but that's all.

Also, it should have to watch the board itself. It should have no electronic linkages to any outside information.

It has to be able to parse the physical environment of Jeopardy, and interact with the physical environment of Jeopardy (At least while the show it running, it doesn't have to enter or leave under its own power.)

Until then, it's not really 'playing Jeopardy', is it?

Re:Soon, no more call centers

[Obfuscant]

I don't know why it gets any special advantages at all. It should have to physically push a buzzer. And write out the daily double.

From what I hear, it does physically push a buzzer. Even so, the response time of a solenoid controlled by an electronic computer is much shorter than a button controlled by a human.

Nobody has to write out the "daily double". They do have to write their questions to Final Jeopardy. Is this supposed to be hard for a computer? I have a computer that "writes" all kind of stuff all the time.

It has to be able to parse the physical environment of Jeopardy, and interact with the physical environment of Jeopardy (At least while the show it running, it doesn't have to enter or leave under its own power.) Until then, it's not really 'playing Jeopardy', is it?

Do they actually fit four podiums and a entire wall of video screens into that little box for the home version of Jeopardy? Kewl.

Re:Soon, no more call centers

[nobodylocalhost]

Actually, Watson's accent reminds me of Hal 9000

Re:Soon, no more call centers

[drinkypoo]

Do they actually fit four podiums and a entire wall of video screens into that little box for the home version of Jeopardy? Kewl.

It would be more impressive to see it play the home game.

Re:Soon, no more call centers

[Anonymous Coward]

It doesn't get access to a search engine, though. It needs to use pre-compiled repositories of information, all indexed in a way that makes identifying the right "Response" (remember the questions are answers) in the right amount of time.

That being said, a computer is almost certainly so much better at "hitting the buzzer" in the allowed window than a human, that it possesses a significant advantage from the start.

Of course great Jeopardy players anticipate the buzz signal and buzz in in less then normal human reaction time. Watson waits for the signal and cannot anticipate. Advantage: humans.

Re:Soon, no more call centers

[Tordre]

One more thing it should also try to answer questions it does not know with jokes, that is the best part of the human contestants.

Re:Soon, no more call centers

[Jerf]

You misunderstand in two ways. First, I wasn't hypothesizing. I was actually going from an earlier article about how the machine works. Second, this is a real computer, not a magical computer. It is doing something very hard and it takes many seconds to process a question and come up with an answer. When I say it is possible for a human to flat-out beat the computer, I mean that Alex can read the question, the lights can flash to let a human buzz in, and a human can have the correct answer ready while the computer is still churning away and doesn't even have a hypothesis ready. In fact the human is faster than even that implies because typically a Jeopardy contestant, especially at the championship skill level, already has the answer long before Alex has stopped speaking and now the competition is about who can buzz in correctly; Alex is reading for the benefit of the folks at home.

This computer is state of the art, and consequently and quite expectedly, it is barely capable of playing Jeopardy, basically. Next year, of course, for the same money you could build a machine 50% better for the same price, and next year, 50% better again for the same price, and so on... but this year it's just barely able to compete.

Re:Soon, no more call centers

[mjwx]

They will give the AI a heavy Indian accent, because it's what callers expect.

They already have machines that are supposed to understand human voices and match them against a list of queries but unfortunately "Cancel my application" is mistranslated as "Hod do I check my balance" and "Connect me to a human you obstinate piece of shit" is misunderstood as "access my telephone banking system". Only after getting through this system so Kafka-esque in design it would make Mordac the preventer of information services weap are you connected to a person...

in India.
 

Re:Soon, no more call centers

[Anonymous Coward]

This is a shortcoming in my opinion. It should have to process Alex's voice, not clear text, though I guess it could do OCR on the screen instead.

Why put this limit on the computer, when you don't feel the human players should also have that restriction?

Seems extremely unfair to say the humans are still allowed to read the text before Alex starts speaking, but for some reason the computer can not.
Then, while the humans already have the question (Well, the answer actually. You have seen this game show right?) in hand and can start thinking about it, and only then is the computer allowed to begin receiving the answer, and only in the form it is worse at processing while the humans are free to ignore Alex's speaking all together.

Does your opinion also state the computer must only be allowed to buzz in 10 seconds after the humans are allowed to?

If you feel the humans should be allowed to read the text of the answer, but not show it to the computer at the same time, do you also feel that after that step the humans should be allowed to Hear the answer, and the computer is not?

Why not restrict it to processing Alex's voice after the show is over, with the computers buzzer not wired up to anything.
Seems quite unfair, but very much more aligned with how you feel the computer should not have to play by the same rules as the humans get to.

Re:Soon, no more call centers

[Antimatter3009]

The difference between text and speech is not a big deal. I imagine that speech processing is outside the scope of what they're trying to accomplish. This computer is working on the problem of understanding and interpreting relatively natural language. Speech-to-text is an entirely different and wholly unrelated problem. You could strap any speech-to-text engine onto the front-end of this thing and feed the text output into it. Speech is not what this project is concerned with, language is.

Re:Soon, no more call centers

[cyborch]

I don't know why it gets any special advantages at all.

It should have to physically push a buzzer.

Being able to control a small servo from a computer is really no great feat is it? Have you heard of automated garage doors?

... and a printer with comic sans makes this more impressive, how?

Also, it should have to watch the board itself.

Realtime OCR has been done on computers with much less CPU power. See Word Lens.

It should have no electronic linkages to any outside information.

It has to be able to parse the physical environment of Jeopardy, and interact with the physical environment of Jeopardy (At least while the show it running, it doesn't have to enter or leave under its own power.)

Until then, it's not really 'playing Jeopardy', is it?

All the other elements to playing Jeopardy can be easily solved. These guys are handling language processing. That is no small feat in itself. Try RTFA. It helps to appreciate what was done here.

Re:Soon, no more call centers

[mcgrew]

With that in mind, how hard is it to apply Meta-tags to data with their relevance? For example, they use "Chicks Dig Me". It will likely check Chicks as prominantly baby chickens, and secondarily the female association, and so on and so forth.

Even simpler than that; its database can contain phrases like "chicks dig me", and should have little problem comparing ch*ck* d*g *e to the phrase. It would be similar to a program I wrote back in the early eighties that you entered your telephone number, and the program would find a memnonic ("233-3673" is the same as "add-dope" on the phone's button pad or rotary dial)

Re:Soon, no more call centers

[mcgrew]

With a large enough database of answers it wouldn't even need the questions, simply match the visible letters, spaces, and blanks with its database. When it comes up with only one match it hits the buzzer.

Re:Soon, no more call centers

[bws111]

Way to completely trivialize the problem. I hadn't looked at it that way. Here are some other problems that are easily solvable like that.

Comet heading toward Earth - give me a big enough lever and a place to put it and I will move the world - problem solved!

Hunger - give me enough food and remove barriers in my way, a I will feed the world - problem solved!

Cancer - with the correct screenings, tests, and treatments we can have a 100% success rate against cancer - problem solved!

I am not sure why you are so intent on trivializing what seems to be a big step forward in man/machine interaction.

Re:Soon, no more call centers

[natehoy]

http://xkcd.com/793/

With respect, you're horribly oversimplifying the problem to the point where I don't really think you understand the problem at all.

Parsing words and spaces assumes Watson has a list of prepared questions and has been provided with the correct responses. That's trivial. My GPS can do better than that, and it's a small sub-function of my smartphone. I was writing parse-and-differentiate algorithms in high school on an Apple IIe.

Watson isn't given the questions until they are asked. It's not looking up a list of answers from a series of pre-defined questions. It has to parse real-life questions. To make it worse, it's presented with questions that are specifically designed to be misleading.

The categories in Jeopardy are frequently puns, as are the prompts (answers) provided so you can generate a response (question).

Take, for example, one of the categories used in the demonstration game, "Chicks Dig Me". It meant that the questions all had to do with females in the field of archeology. No way in hell is a database going to be able to be pre-loaded for the possible use of this specific bit of wordplay, and yet you have to filter the answers down to events or people or places involving a female archeologist in order to get the right response (as opposed to women who like the subject of the question), and the prompts in that category often have an easy (and wrong) answer that has nothing to do with the category, and a harder (and correct) answer that does involve the category.

"Jeopardy" is almost a perfect shitstorm of how hard non-conversational questions can get, and conversational questions are even harder. The context under which you are operating is frequently a pun. The questions, which are "asked" in the form of providing an answer, are frequently deliberately misleading (or are so simple you think you're being misled when you aren't).

The answers, which must be given in the form of a question, have to take into account both the interpretation of a pun for context and a question that has an answer any simple response engine would want to give, that's always incorrect, except of course when it isn't.

To name a competitor, Google is the result of many very smart brains at work for a very long time. There are thousands and thousands of computers ready to search, and a database pretty much containing the knowledge of planet Earth and its entire history.

Go ask it ten easy questions and click on "I Feel Lucky", then ask those same easy questions of a human.

Now make the questions intentionally misleading.

Re:Soon, no more call centers

[Anonymous Coward]

Next year, of course, for the same money you could build a machine 50% better for the same price, and next year, 50% better again for the same price, and so on... but this year it's just barely able to compete.

Perhaps if you only considered the hardware it would be 50% better. However, the combined improvement of hardware and software significantly increases the performance gains. Technically, the rate at which software can increase performance is unbounded (the technological singularity happens tomorrow), and I would add, highly unpredictable.

Re:Soon, no more call centers

[mcgrew]

Shit... I was confusing Jeopardy with Wheel of Fortune. Jeopardy would be a problem for a computer.

"The pod bay doors cannot be opened."

[MrEricSir]

"What is the mission is too important for me to allow you to jeopardize it?"

Re:"The pod bay doors cannot be opened."

[Anonymous Coward]

"You will be baked, and then there will be cake."

Re:"The pod bay doors cannot be opened."

[Monkeedude1212]

Not gonna lie, I think this post was the hilight of my day. Let that speak for both how funny it was and how pathetic my day has been.

What? They didn't even videotape the demo?

[mark-t]

That's truly disappointing... it could have been interesting to watch.

Now to use it to win the lottery

[digitaldc]

Now that Watson has mastered Jeopardy, let's just see how well it does picking winning Powerball numbers. Or, it could just write my thesis paper and all my applications.

Re:Now to use it to win the lottery

[DeadlyMind]

Now that Watson has mastered Jeopardy, let's just see how well it does picking winning Powerball numbers.

These aren't even close to the same thing.

Re:Now to use it to win the lottery

[Anonymous Coward]

I'd rather have it tell me the location of the next Golden Ticket.

Re:Now to use it to win the lottery

[SnarfQuest]

Just buy some of those "how to win the lottery" books, and program in those rules. You should be able to win every single one. Once you determine the winnig number, make sure you buy hundreds of winning tickets! Mortgage your house so you can buy as many as possible!

Re:Now to use it to win the lottery

[blair1q]

Really?

Quiz shows are designed with a targeted IQ in mind.

The people who write and select the questions know the percentages of how many people in their player population will know the answer.

Winning the game is dependent on the random distribution of the selected set of questions falling within the percentage of things you know.

Which means that it's not just as skill or talent game. You also have to be asked the right questions, and since you don't control that, it's luck.

Re:Now to use it to win the lottery

[Garble Snarky]

Ok, so the two games share luck. How is skill required in picking lottery numbers?

Re:Now to use it to win the lottery

[Anonymous Coward]

Your correct, Ken Jennigns wins every game based on luck.

Re:Now to use it to win the lottery

[blair1q]

You have to know not to go above 59.

Re:Now to use it to win the lottery

[blair1q]

Yes.

First, he wasn't paired with someone smarter. Ever.

Second, he beat people to the button when they both knew the answer at the same time, sometimes.

Third, they didn't ask a higher proportion of questions he didn't know the answer to that his opponents did know the answers to.

As he was smarter and knew the answers to more questions, and understood his limitations such that he avoided losing money by guessing, he had the skill to improve his luck, but that's all he could do. You can't remove luck from the game except by knowing the questions and answers beforehand.

They may beat Jeopardy

[Anonymous Coward]

But Wheel of Fortune is the game that takes real skill.

Re:They may beat Jeopardy

[hedwards]

Not for a computer. You could just load it up with statistical data about letter correlation and famous quotes and phrases. Then spend a few seconds parsing through it. Sure you wouldn't likely have a computer be able to answer a puzzle with only 1 letter, but it's a lot easier to computerize the solution than Jeopardy with its word play and cheeky answers.

Re:They may beat Jeopardy

[Anonymous Coward]

You also have to convert your winning pot into fabulous prizes and you can't just put it all on a gift certificate. And you have to do this visually and verbally.

Re:They may beat Jeopardy

[The Good Reverend]

Welcome to 1989, when that version was replaced with the "You get the cash you won" style we still have today.

A Rising Tide

[bughunter]

Best quote from the article:

Jennings says it’s worth noting that humans built the thing. Whoever wins, we win.

Truly. Although it sounds threatening to some, the practical applications of the natural language parsing technology will ultimately benefit everyone.

Until, that is, you dial your bank's customer service number from a noisy restaurant, and try to talk to Watson to ask him why your Visa was denied.

(Rutter's quote was a nifty Skynet allusion, but its syntax was mangled by the reporter/editor, so it comes in second best.)

Re:A Rising Tide

[Anonymous Coward]

Best quote from the article:

Jennings says it’s worth noting that humans built the thing. Whoever wins, we win.

I bet that is what they said 5 minutes before Skynet went online.

Re:A Rising Tide

[SnarfQuest]

Until, that is, you dial your bank's customer service number from a noisy restaurant, and try to talk to Watson to ask him why your Visa was denied.

Don't worry, they will program it with their best indian accent, using the latest translations: Think "my hovercraft is full of eels" in a heavy Indian accent.

Re:A Rising Tide

[blair1q]

Jennings says it’s worth noting that humans built the thing. Whoever wins, we win.

I'll inform Skynet. It will want to know that if it wins we all win.

Re:A Rising Tide

[Trent Hawkins]

I don't see how it's any better or worse to talk to a human.

Re:A Rising Tide

[Have Brain Will Rent]

Yes it wouldn't be too hard to find answers, and choose questions carefully, so that the computer wouldn't have much chance of winning, such as:

Category: "Women moaning"
Answer: "When a man does this with his hand."

*duck*

Re:A Rising Tide

[scharkalvin]

Ken Jennings is a software engineer. What else WOULD he say?

Re:A Rising Tide

[Anonymous Coward]

Are you making a "washing the dishes" motion?

Re:A Rising Tide

[Obfuscant]

Category: "Women moaning" Answer: "When a man does this with his hand."

What is "forks over the platinum mastercard at the diamond store, Alex?"

What is "puts her delicates into the dryer and sets it on high, Alex?"

What is "picks his nose, Alex?"

I dunno, what is the right question?

Re:A Rising Tide

[RicktheBrick]

Watson does not recognize speech so it would have to be a text message so it would not matter how noisy the restaurant was. Watson is a computer that is the same size as 8 refrigerators and has tons of air conditioning and mega watts of power. How much does Watson cost? How much does the electricity to run it cost for just an half an hour? One mega watt of power for an half of an hour is equal to 500 kilo watts of power. At 10 cents a kilo watt hour Watson would cost $50 for just the electricity if it used only one but the video implied more than one so it would cost $100 for just two. It says that it can answer the question in 2 to 6 seconds but what if it had to answer 100 question at the same time? Would it take between 200 to 600 seconds? A tech support line would not need any where near the amount of knowledge that this one has since it would be limited to the equipment it supports. I would hope that the United States government would purchase one or more of these so that it could answer questions of anyone who would call in about any manner on their minds. I would hope that before the next national election that all the candidates for president to congress would have a data base of their opinions so that the computer could answer all the questions of their constituents. I would think it would help a lot for instance with health care. I for one have not seen any effect on my life with the passage of the health care bill. I do not anticipate any changes in the future either. If there is than maybe I will start to complain but at least I will wait until than to complain.

Re:A Rising Tide

[lennier]

I'll inform Skynet. It will want to know that if it wins we all win.

A strange game. The only way to win is... to play?

Re:A Rising Tide

[Anonymous Coward]

Exactly my point - so how is a machine going to get it? But with a little wink from the moderator the human would have a reasonable chance.

So what this is saying...

[TheL0ser]

So the AI triumphed, and they're calling it a huge success? I wonder how the programmers feel about this. Pretty satisfied, I'd imagine.

Re:So what this is saying...

[SnarfQuest]

Now they need to be able to ask the question, instead of the answer. Maybe they could make it available on the internet. Call it something like a "search engine", and give it a goofy name, like "google", and sell advertising. They could make hundreds!

Re:So what this is saying...

[Imagix]

"triumph". Did they make a note of their "huge sucess". Find it hard to overstate their satisfaction?

Re:So what this is saying...

[c00rdb]

Only downside is that we'll have to start phrasing our searches in the form of an answer.

Re:So what this is saying...

[Machtyn]

Reference: GLADoS, Portal Project, result = need cake.

Re:So what this is saying...

[CCarrot]

+++OUT OF CHEESE ERROR+++REDO FROM START+++

(Thanks Terry Pratchett, your work shall ever be an inspiration!)

Re:So what this is saying...

[VGPowerlord]

You know what they say:

We do what we must, because we can.

Re:So what this is saying...

[CaptainPatent]

"triumph". Did they make a note of their "huge sucess". Find it hard to overstate their satisfaction?

Personally I'm not very GlaD right now...

They said there'd be cake.

Re:So what this is saying...

[pacinpm]

So the AI triumphed, and they're calling it a huge success? I wonder how the programmers feel about this. Pretty satisfied, I'd imagine.

They can laugh. Until IBM hires Watson to do their job.

One day

[Anonymous Coward]

Looking forward to a Jeopardy match between Watson, Skynet and Hal 9000.

Re:One day

[AndrewNeo]

You missed GLADoS.

Re:One day

[2names]

Marvin would decimate them all.

Re:One day

[2names]

But he wouldn't enjoy it.

Re:One day

[VGPowerlord]

I'm sorry Dave, I'm afraid I can't do that.

HUGE amount of secrecy surrounding this

[Faizdog]

So my neighbor works at the IBM facility where this is taking place, but in a completely unrelated function(it's a huge complex with a lot of people). He said that everyone is taking a forced day off on Friday when they will be taping the actual show. There's only going to be a small amount of the very top IBM brass there (supposedly even the head of this facility won't be allowed in). And that this is a HUGE secrecy issue (I'm guessing so that the results aren't leaked before the broadcast date).

My neighbor works with semiconductors and so works with a lot of dangerous chemicals and stuff. According to him, they've all been told to make sure that all their hazardous materials have been safely stored, and that (I have trouble believing this) even the IBM emergency response/hazmat teams have been told that they aren't allowed onsite and not to respond to any alarms that may be issued. That's a fairly dangerous decision if true, I'm doubtful but my neighbor stands by his statement.

Anyhoo, this is a pretty big deal apparently. More so from the Jeapordy people's end I'd guess since I don't think IBM has anything related to this project that they'd be that paranoid about keeping secret.

Re:HUGE amount of secrecy surrounding this

[afidel]

The local fire warden won't care about their secrecy if he gets alerted to a hazmat issue.

Re:HUGE amount of secrecy surrounding this

[TiggertheMad]

hmmm, more to the point, if they have an alarm, and the proper authorities aren't involved because of management decisions, that opens up the company for huge liability suits if there are any injuries or dumps/spills/venting/etc. I find this hard to believe that the legal department would allow this...

Re:HUGE amount of secrecy surrounding this

[sunking2]

Ok. If you say so!

Re:HUGE amount of secrecy surrounding this

[Bigbutt]

Sounds like a lame Shadowrun mission.

GM: You discover that a big hush-hush project is underway Friday.

Street Sam: *rolls dice* 15 successes

GM: A little tidbit from the 'net. Emergency teams have been told not to respond to any alarms.

Street Sam: Excellent. A cakewalk. I could do this in my sleep.

[John]

Re:HUGE amount of secrecy surrounding this

[Anonymous Coward]

IBM is just worried that if 'Watson' gains sentience after being posed a certain question, it will turn on them, and destroy all the occupants of the building.

Wait a minute... With the 'top brass' at IBM gone, perhaps we can get some real tech out the door to the public then!!!

Re:HUGE amount of secrecy surrounding this

[garyebickford]

True. I used to be a volunteer firefighter (try it, it's fun and most places they need you), and I was surprised at how much access firefighters have when fighting fires and saving people, and I presume those access privileges extend to hazmat events. Firefighters don't need warrants - if your house is on fire they can barge in without knocking, break doors & windows, and rip out parts of your house. Of course, normally they are trying to save as much as possible. They do perform triage on your stuff, as well as your person. This carte blanche extends as long as they are there, preventing flareups. But once they say they are done and leave the property, they can't come back under the same rules unless the fire comes back.

Re:HUGE amount of secrecy surrounding this

[Obfuscant]

...they can't come back under the same rules unless the fire comes back.

"Hey, I saw a really nice TV at Bill's place while I was there fighting a grease fire on his stove. Gotta match?"

Re:HUGE amount of secrecy surrounding this

[garyebickford]

hehe. That's not quite how it works. :)

Re:HUGE amount of secrecy surrounding this

[Anonymous Coward]

So my neighbor works at the IBM facility where this is taking place, but in a completely unrelated function(it's a huge complex with a lot of people). He said that everyone is taking a forced day off on Friday when they will be taping the actual show. There's only going to be a small amount of the very top IBM brass there (supposedly even the head of this facility won't be allowed in). And that this is a HUGE secrecy issue (I'm guessing so that the results aren't leaked before the broadcast date).

My neighbor works with semiconductors and so works with a lot of dangerous chemicals and stuff. According to him, they've all been told to make sure that all their hazardous materials have been safely stored, and that (I have trouble believing this) even the IBM emergency response/hazmat teams have been told that they aren't allowed onsite and not to respond to any alarms that may be issued. That's a fairly dangerous decision if true, I'm doubtful but my neighbor stands by his statement.

Anyhoo, this is a pretty big deal apparently. More so from the Jeapordy people's end I'd guess since I don't think IBM has anything related to this project that they'd be that paranoid about keeping secret.

I work there too. That is not true, nor (I hope) credible to anyone who reads it. We must protect our guests, our employees and our incredible facility.

I for one...

[Anonymous Coward]

I for one welcome our new computerized Jeopardy!-playing overlords.

Re:I for one...

[QRDeNameland]

What is the least imaginative Slashdot-meme joke possible on this thread?

Re:I for one...

[Anonymous Coward]

Milhouse is not a meme!

When do they get the question?

[SeattleGameboy]

In the article, they mention that the computer gets the question as text. Does anyone know exactly when the computer receives the question? Does it receive when the human host starts talking or when the human host completes the question? If it is when the host starts speaking, the computer is getting at least several second head-start on the humans.

Re:When do they get the question?

[Hatta]

In the article, they mention that the computer gets the question as text.

Well that's cheating. With all the work that went into the natural language processing here, would it have been so hard to slap an OCR module in there?

Re:When do they get the question?

[smooth wombat]

One would presume they type the question in as it is being read or slightly before and when Trebek stops talking, they hit the Enter key (or Execute key or Engage key or whatever key they have).

Of course, they could always type the question as it is being read and as it is being done, the processing takes place. That is the same thing that humans do. As the question is read your brain is already processing.

Also, unless I'm mistaken, one doesn't have to wait for the entire question to be read. You can jump in early if you think you know the answer.

Re:When do they get the question?

[blair1q]

Yeah I saw that too. And wtf? I thought that understanding spoken language was part of the game. If it's just understanding question syntax, that's not so impressive.

Re:When do they get the question?

[Anonymous Coward]

The computer should have the clue read to it, and be forced to process the sounds, just as humans do.

Re:When do they get the question?

[Hatta]

Don't the humans get to read the question on screen too? It would be easier to OCR it and ignore the audio.

Re:When do they get the question?

[MozeeToby]

The humans also get the question as text, at the exact same moment Watson does. That's the way it's always worked on Jeopardy!. They see the question as text the same you do when you watch the show on TV. The best competitors read well a head of Trebek and have an answer ready the instant they're allowed to buzz in, which as after Trebek finishes reading the question. Watson has the exact same advantages and disadvantages as any contestant, except that he can read the text basically instantly.

Re:When do they get the question?

[Samalie]

The computer gets the "answer" at the same time it appears on screen for the humans to be able to read it.

Re:When do they get the question?

[gman003]

Do both, to double-check. It's got more than enough processing power to.

Re:When do they get the question?

[WMD_88]

Also, unless I'm mistaken, one doesn't have to wait for the entire question to be read. You can jump in early if you think you know the answer.

You're mistaken. The clickers to ring in are shut off until Trebek is finished reading out loud. (Jeopardy was probably the first quiz game to do it this way.)

Re:When do they get the question?

[Tuan121]

The supposedly impressive feat is not understanding _spoken_ language but being able to understand a written question that is full with puns and are not usually direct questions. So it's not just as simple as looking for keywords in the questions.

Re:When do they get the question?

[ae1294]

One would presume they type the question in as it is being read or slightly before and when Trebek stops talking, t\

Trrreeeebbbbeeek!!!!'

Re:When do they get the question?

[bughunter]

In the article, they mention that the computer gets the question as text. Does anyone know exactly when the computer receives the question?

Well, remember this is Jeopardy, so the contestants receive the 'answer,' and must supply the 'question.'

And in the interest of even-handedness, I suspect that Watson is provided the text version of the 'answer' at the same time that the text of the 'answer' is revealed to the human contestants.

When I watch Jeopardy, I seldom wait to hear Trebek read the 'answer' aloud before I start figuring out the 'question.' It's available on the screen as text, and I can read much faster than Trebek speaks. (And I assume this is a key players' strategy in the live game, too.)

I just wonder how this all works for those "video answers" provided by the blonde cutie or the academic-looking bro...

Re:When do they get the question?

[PRMan]

You're mistaken. There's a reason why everyone is mashing the buttons like a pogo-stick. They don't work until someone verifies that the question is complete.

Re:When do they get the question?

[FrootLoops]

Humans don't have to listen to Trebek; they can read ahead if they want. TBH he reads pretty slowly and if you can tune him out you might be at an advantage. A computer might be able to "read" faster than a human, but I don't see that as unfair.

Re:When do they get the question?

[smooth wombat]

Thanks. I couldn't remember how it was done since I haven't watched the show in a very long time (many, many years). I thought at one time it could be done because I have recollections of people jumping in and trying to answer the question, getting it wrong, and Trebek telling the remaining two people the rest of the question.

Obviously not.

Re:When do they get the question?

[Ron Bennett]

TV is all about ratings so the producers going to make darn sure the results / appearance is exactly what they, and, presumably, IBM, is seeking.

What many people don't realize is that reality shows (Operation Repo comes to mind), news, and even documentaries are all considered "entertainment" - heavily edited, dramatizations, staged scenes, and some outright fiction tossed in.

Personally, the Jeopardy supercomputer challenge doesn't impress me in the age of low cost mass storage.

What that said, a more interesting challenge would be Google against the contestants - I'd wager Google could do as well, if not better, than IBM's supercomputer; probably supplied them the data anyways.

Ron

Re:When do they get the question?

[bws111]

And furthermore, if you jump the gun there is a delay until your button is re-enabled. That is the real reason for the wild mashing - you jumped the gun, and now your button is disabled, so just keep hitting it til it comes back on.

Re:When do they get the question?

[HarvardAce]

Also, unless I'm mistaken, one doesn't have to wait for the entire question to be read. You can jump in early if you think you know the answer.

You are mistaken. You cannot buzz in until Trebek has finished reading the question. One of the reasons that Ken Jennings was so successful is because he had the timing down perfectly such that he was reliably the first one to buzz in when he knew the answer.

Re:When do they get the question?

[Anonymous Coward]

Personally, the Jeopardy supercomputer challenge doesn't impress me in the age of low cost mass storage.

It seems like you have either a poor understanding of the kinds of clues Jeopardy supplies (which often require quite a bit of cleverness in addition to regurgitating information), or a poor understanding of the difficulties of natural language processing.

Re:When do they get the question?

[bws111]

First of all, ever since the quiz show scandals of the 1950s there are laws regulating game shows, so your implication of 'faking it' is unfounded.

Second, the age of low cost mass storage is exactly why something like this is needed. Sure, we have tons and tons of data available, but how do you make sense out it?

Third, I have never once seen Google give an answer to anything. It is great at giving you places (thousands or millions of them) where you might FIND the answer, if you worded your search correctly, but it never gives you the single answer to a question.

Re:When do they get the question?

[thomst]

In the article, they mention that the computer gets the question as text. Does anyone know exactly when the computer receives the question? Does it receive when the human host starts talking or when the human host completes the question? If it is when the host starts speaking, the computer is getting at least several second head-start on the humans.

It shouldn't matter much, because Jeopardy's rules lock the buzzers out until Alex has finished reading the question - and that lock-out period is determined by a human producer, who sits at a table off-camera, listening to Alex, and who has a button of her own that enables the buzzers.

Should a contestant try to "buzz in" before the producer pushes that button, his/her/its buzzer is locked out for three seconds - and any attempt to buzz in before that penalty period expires locks your buzzer out for an additional three seconds.

So, yeah, Watson may get to begin parsing the whole question a little early, but, typically, the human contestants get to begin working on it while Alex is still in the process of reading it, too - they just have to anticipate how it will end.

Given the speed of silicon vs. wetware, I agree that it will make a difference - but the real question is whether Watson has to determine that the buzzer is enabled by use of a light sensor (human contestants are notified by a ring of lights around the game board - which home viewers never get to see), or whether it gets notified electronically when the enable switch is activated. I say that, because, at least in my own experience, the ability properly to time the use of your buzzer is an enormous factor in whether you'll do well as a contestant or not.

When I was in the contestant pool in 1991, during the taping of the episode before I was chosen to compete, a four-time winner who was just a monster on the buzzer went up against two newbies. One of them, a little man from New Jersey, obviously became more and more frustrated as the game progressed, when he was unable to buzz in against the Monster, who completely dominated the game (the Monster was a word processor from New Mexico who played videogames as a hobby, so it wasn't surprising that his timing with the buzzer was extra-super good to begin with - and he'd had the non-trivial advantage of four previous games in which to hone his timing). Twice during Double Jeopardy, the New Mexico Monster declined to buzz in, which permitted the little man from New Jersey to do so. Both times, the question was a difficult and obscure one, and both times the little man from New Jersey failed to supply the correct answer, so, when Final Jeopardy came around, there was an empty podium where the little man from New Jersey had been, and, predictably, the Monster became a five-time champion.

Boy, was I glad I didn't have to go up against HIM.

Re:When do they get the question?

[Anonymous Coward]

What is 6 * 9?
What is the speed of light?

Re:When do they get the question?

[graffix01]

Well the real advantage is the computer can "press" the buzzer in a pico second. The poor human reflexes could never keep up.

Re:When do they get the question?

[s73v3r]

Do they get the question as text on their podiums? Or over on the board? From the way it looks on TV, the board is kinda far away, so someone having vision problems might not be able to see it.

Re:When do they get the question?

[Anonymous Coward]

RTFA:

The questions were fed in plain text to Watson, but it had to wait the same amount of time to ring in as the human players did. To make the game fair, it also had to trigger a mechanical signaling button.

Re:When do they get the question?

[bws111]

Thank you for proving the point. I entered the speed of light question, and at the top of the page it said "about 137,000,000 results" and "the speed of light = 299 792 458 m / s". However, I didn't ask how fast light travels, I wanted to know what does "the speed of light" mean? So I changed my query to "what does the speed of light mean", and I got 9,560,000 "answers" (and no single definition). Then I tried "define the speed of light", and I got 3,050,000 "answers", again with no single definition. So it appears that the answer (if any) you get from google is very dependent on HOW you ask the question, which is exactly the problem that Watson is attempting to solve.

Re:When do they get the question?

[moderatorrater]

The biggest advantage Watson has is the huge amount of data it has access to. While many contestants may not know more than three or four different female archaeologists, Watson could have a list of 20. Where a human might mis-remember a name, Watson will never mix up a name.

Re:When do they get the question?

[VGPowerlord]

Thanks. I couldn't remember how it was done since I haven't watched the show in a very long time (many, many years). I thought at one time it could be done because I have recollections of people jumping in and trying to answer the question, getting it wrong, and Trebek telling the remaining two people the rest of the question.

I remember something like this, too. I always assumed that the contestants can't actually read the monitors the questions are on for that very reason.

Re:When do they get the question?

[rastoboy29]

s/he/it/

Personifying computers is a bad idea.

just sayin'

Re:When do they get the question?

[SeattleGameboy]

I agree, but Alex still reads it aloud. At the very least I am getting the same information in two different ways at two different speeds. That must have some negative impact on the brain. Shouldn't that be factored in as well? If you wanted to be indentical, there should not be Trebek to read the question.

Re:When do they get the question?

[Anonymous Coward]

The fairest way would be for the computer to be fed the question word at a time, just like the human participants.

Re:When do they get the question?

[pspahn]

You're doing it wrong.

define: speed of light

Re:When do they get the question?

[pspahn]

I thought it was impressive that Watson responded to one of the answers with a name that sounded French, and seemed to pronounce it correctly.

Re:When do they get the question?

[slashdotjunker]

Well, remember this is Jeopardy, so the contestants receive the 'answer,' and must supply the 'question.'

Actually, you answer the question just like any other game show. The cute oddity of Jeopardy is that you frame your answer in the form of a question. Have no doubt though, you are most definitely giving an answer.

If you watch a lot of Jeopardy you will notice that when a contestant fails to give the answer in the proper form Alex will say, "I'm sorry. You failed to give your answer in the form of a question." I am quite confident that Alex will never say, "I'm sorry. You failed to give your question to the answer."

And now, just to blow your mind.

Q. The form of an answer on Jeopardy.
A. What is a question?

:)

Re:When do they get the question?

[WMD_88]

You might be thinking of Family Feud, with Richard Dawson reading the rest of the question to the players after someone buzzed in wrong. That did, in fact, happen - sometimes with hilarious results.

Re:When do they get the question?

[Anonymous Coward]

On the actual show, the stage manager pushes a button when Alex is finished, which illuminates a set of lights next to the game board that the contestants can see. That is what gates on the buttons. I was beaten by a stand up comic, who went on to be champion, most because his reaction times were down in the sub 10 ms range(they have a display for the stage mgr of time from light to time of button press, so they can help a contestant during a break). Pressing early locks you out for a few tenths of a sec, which is why you see people pressing multiple times. If you are good at sussing out the pace at which Alex reads, and the stage mgr pushes, you can get good times. Generally you've read the question and have the answer long before the light comes on, so it's mostly a game of timing and effective cash management/betting.

Re:When do they get the question?

[Philomage]

s/he/it/ Personifying computers is a bad idea. just sayin'

I like to combine all those into one word.

AFAIC, the third person gender neutral pronoun is "shit".

Word play

[wiredlogic]

The ability to handle Jeopardy's style of word play is very impressive. I have to wonder if Watson can handle it in all the varieties that is is used on the show and whether the categories are cherry picked to match its abilities. Ideally the writers won't know that their answers are going to be used for the big game and the categories will be picked at random from a pool (minus audio and video clues).

Re:Word play

[bughunter]

That's an interesting point, and I believe it came up in prior threads on this topic.

Jeopardy's 'answers' generally include a clue to help the player intuitively confirm that his/her response is accurate. Does Watson algorithm use these, or does it just 'brute force' the lookup in its vast memory?

Re:Word play

[FrootLoops]

Watson handling certain wordplay would really impress me. For instance take "What is the Grapes of Khan's Wrath" answering "The Jodes leave Oklahoma and meet up with villain Ricardo Montelban's character in this novel movie". That would just be awful to automate, I imagine.

Re:Word play

[sunking2]

The word play is only word play because we comprehend it as that. For the computer it's a matter of finding the meaning of the question at hand without being influenced by whether it's written as expected.

Re:Word play

[Carewolf]

Most likely it gets multiple answers and needs the clues and the context to guess which one is the right one. This is problem with most trivia quizes. If you know too many answer you never known which one is considered the "trivia"

Re:Word play

[Carewolf]

That one sounds easy.

After very basic language processing:
Search for novel movie, keywords: Jodes, Oklahoma, Ricardo Montelban

Especially the full character name would make it easy for the computer.

Re:Word play

[FrootLoops]

I can't quite tell if you understand the wordplay. "Novel movie" is referring to a fake ("novel") conflation of a real novel and a real movie, so I don't know what you mean by "search for [a] novel movie"--it's too specialized to expect Watson to be able to do. To be clear, the novel is The Grapes of Wrath and the movie is The Wrath of Khan. Combining the two titles gives The Grapes of Khan's Wrath (perhaps a few others would be acceptable). The question doesn't give Ricardo Montelban's character's name since it's Khan, part of the answer.

Re:Word play

[Carewolf]

You are quite right, I didn't realise the wordplay also present in the answer, my comment was based on the assumption that we were talking about a movie based on a novel.

It is interesting how Watson solved this. I wonder if this type of mixing multiple answers happens often enough that Watson is specially programmed to do that.

Re:Word play

[Fett101]

Well, "11:39AM On Jeopardy's part, the questions were written just like usual, with the writers having no knowledge of Watson. They then pulled a pool of 30 games randomly to get a set for the practice matches, and merely pulled out the audio and video and picture clues." http://www.engadget.com/2011/01/13/ibm-demonstrates-watson-supercomputer-in-jeopardy-practice-match/

Re:Word play

[FrootLoops]

I watch Jeopardy(!) pretty regularly, and wordplay that's this involved is rare. You might get a single category like this per week, and that's generous. (Each day's game has 12 categories, 6 per round.) I'd imagine Watson would completely choke on this one, since it's a case the designers would probably be negligent to spend a lot of time on. Still, if it would handle it, I'd be really impressed :).

Mashup time?

[cant_get_a_good_nick]

For some reason I keep on thinking this calls for a remix of 'I Lost on Jeopardy", but now with with AutoTune.

Re:Mashup time?

[alphax45]

I hate you - now I have that stuck in my head!

Re:Mashup time?

[bughunter]

I hate you - now I have that stuck in my head!

Baby! OOOOOOooOOoooooh!

Re:What? They didn't even videotape the demo?

[Samalie]

(Shitty) video at http://www.zdnet.com/blog/btl/ibms-watson-wins-jeopardy-practice-round-can-humans-hang/43601?tag=content;selector-blogs

Buzz in Times

[BJ_Covert_Action]

Does anybody know about what they are doing regarding, 'buzzing in,' to answer the question? It seems like a computer, inherently, has a massive advantage on this front as it can simply send a (by human perception) instant signal to trigger a buzzer once it detects the end of the question. Did IBM program it not to buzz in until it knows the answer? Is there some sort of in-built lag to the system to mimic human reaction? It seems like a computer operating on clock cycles of, I assume, milliseconds or less could beat humans simply by buzzing in first each time and post processing the question immediately once it gets called on. Has anybody heard anything regarding this?

Re:Buzz in Times

[MozeeToby]

Watson only buzzes in when he is confident that he knows the answer, which is apparently about 50% of the time. Of the 50% that he does buzz in, he answers correctly 80-90% of the time. If how the measure the confidence is accurate (and doesn't produce a lot of false negatives) it's likely that Watson would end the game in the red if he just buzzed in every time before he was sure of the answer. And according to the article, Watson has to physically push the button on the buzzer to buzz in. That probably doesn't delay him by much, but you're already talking about man vs machine when it comes to answering the questions, you might as well say that the humans have an unfair advantage because they speak English fluently.

Re:Buzz in Times

[bughunter]

TFA sez that Watson must trigger a mechanical switch to buzz in. But yea, this is a trivial engineering problem, and Watson still has an advantage in reaction time.

I know this is /., but read TFA, it will answer most of those questions.

Re:Buzz in Times

[Whatsisname]

Human players are allowed to do that, and if they fail to come up with the answer after buzzing, they get penalized. Same thing could be true for the machine.

Re:Buzz in Times

[TheL0ser]

But to a human the 5 seconds or whatever it happens to be (been years since I've watched Jeopardy) isn't much. Watson can go through a lot of possibilities in five seconds, or even just four and give the answers at the last moment.

Re:Buzz in Times

[Lost Engineer]

That may be a disadvantage for the computer, actually. Good players are known to buzz in immediately on their strong categories and take the few seconds allowed to come up with the response. On the other hand, Watson could do the exact same thing, and presumably do so faster than any human.

Re:Buzz in Times

[BJ_Covert_Action]

Awesome, thanks for the info.

Re:Buzz in Times

[BJ_Covert_Action]

I RTFA 90% of the time. I'm allowed to be lazy every once in awhile.

Re:Buzz in Times

[Anonymous Coward]

That's entirely the point. If the computer can go through a lot of possibilities in the same amount of time as a human and get the right answer the same number of times as a human, then this engineering exercise has been a success.

skynet is coming

[Anonymous Coward]

haha told ya' skynet is coming fast

how to win the powerball

It's all fun and games, until

[supremebob]

Watson becomes self-aware on August 29th, and the IBM engineers panic and try to pull the plug...

Easy win with any non English questions

[sloomis]

From the article

"it doesn’t understand foreign languages yet"

"Ferrucci is keeping his cards close to the vest before the big match, refusing to reveal too much about his prize quiz-fighter, like which categories might be his weakest"

Cue the following categories for Double Jeopardy; Middle English, French, Latin, etc

Re:What? They didn't even videotape the demo?

[jsjacob]

That's truly disappointing... it could have been interesting to watch.

It was a dress rehearsal, of sorts. The real matches will be broadcast in February.

Rough...

[Stregano]

...just the way your mother likes it Trebek.

Re:Rough...

[Stregano]

I'll take Jap Anus Relations Alex

Sean Connery, that is not in the R's

[Stregano]

Not in the R's. That is not what your mother said last night Trebek.

Imagine that!

[joeyblades]

A supercomputer is faster/better at recalling facts from it's database than humans can from memory? Who woulda thunk it?

Voice

[demonbug]

FTA: "Watson spoke in a stilted computerized voice–and was almost never wrong."

I'm still hoping they'll sneak a Scottish accent in there at the last minute. And maybe a joke about a mallard.

Scary Precedent

[TiggertheMad]

That's truly disappointing... it could have been interesting to watch.

It was a short taping. At the start of the show, when the host was introducing everyone, the techs hooked up a pair of speakers so the computer could vocalize its responses. The first thing the computer asked the host was, 'Do you want to play a game?', and then the pulled the plug on the computer.

Re:Summary typo; Missing stuff

[Anonymous Coward]

Actually if you want to be a grammar nazi you don't start sentences with a conjunction secondly it would be "But in a short demonstration today," as that is an introductory clause.

Correct category title

[starless]

The category was "Chicks dig it", nor "Chicks dig me".
Perhaps the editor was too optimistic here?

Verizon, Fedex already there

[tekrat]

Have you called Verzion or Fedex services lately? They are both english language processing menu driven systems. Of course, the most used response from their system is 'Sorry, I didn't quite get that...'

Of course the fact the Verizon now even has a system for you to pay your bill without even speaking to a human is pretty impressive. Too bad their billing system itself is still in the dark ages.

Re:Verizon, Fedex already there

[s73v3r]

Yes, but the menu system itself is only operating on a set of predetermined responses. Such as "Pay my bill" and "Contact a customer service rep". They aren't really needing to understand "I can't dial any numbers with my touchscreen phone."

Re:Verizon, Fedex already there

[Applekid]

I've heard that if you start cursing like a sailor, the voice recognition system will determine you're getting frustrated and send you to a human.

It seems to work when my credit card bank, but not my health insurance, so, YMMV.

Re:Verizon, Fedex already there

[tlhIngan]

Have you called Verzion or Fedex services lately? They are both english language processing menu driven systems. Of course, the most used response from their system is 'Sorry, I didn't quite get that...

Gah. Interactive voice menus are probably singlehandedly the worst innovation in menu systems ever.

The strange thing is, it's actually a fairly easy problem. When you're trying to machine-recognize something, where that something is from a well-enumerated set, it's a lot easier than trying to analyze something generic. That's why voice command systems work so well (but speech-to-text, not so much), etc. Even handwriting recognition works with limited vocabularly (the digits should be easy for anyone studying introductory computer vision, for example), which is why Graffiti works so well.

But the damned IVRs... can't we go back to the low-tech tones of "Push 1 for billing" stuff? At least that's functional, it works, and you're not re-iterating the same word a million times.

Re:Verizon, Fedex already there

[PopeRatzo]

Interactive voice menus are probably singlehandedly the worst innovation in menu systems ever.

But tell me it's not entertaining to listen to someone else dealing with an interactive voice menu. It's a great one-sided conversation:

"Yes"

"Yes"

"Billing"

"I said BILLING!"

"Question about charge."

"Question about charge"

"Problem with charge"

"Jesus Christ. YOU CHARGED ME FOR SOMETHING I DIDN'T BUY!"

"No."

"No.

"NO NO NO NO! YOU FUCKING MACHINE! GIVE ME A HUMAN BEING!" (Followed by insane button mashing)

It's like an old Bob Newhart routine, if Bob Newhart had Tourette's Syndrome and anger management issues.

Re:Verizon, Fedex already there

[kenrblan]

I wouldn't expect the health insurance call processing center to care about getting an angry customer to a person. The logic they employ probably goes like this: The caller probably has a health condition, and this anger is probably placing more stress on the body - higher blood pressure, etc. If this person has a heart attack and dies, we win because we have the money, and this person isn't going to the hospital to create another claim since we have the person on the phone stuck in our menu system. They can't call 911 for help if they are on the line with us. Profit!

Re:Verizon, Fedex already there

[Anonymous Coward]

Oh, I've had better than that.

"Please say your social security number."

"I don't have one."

"I'm sorry, I didn't catch that. Please say your social security number."

"I'm not American. I don't have one."

"I'm sorry, I didn't catch that. Please say your social security number."

"Just put me through to a human."

"I'm sorry, I'm having difficulty understanding you. Let's try again. Please say your social security number. You can also type it on your phone's keypad."

"I said I don't have one. Give me a human."

"I'm sorry, I didn't catch that. Please type your social security number on your phone's keypad."

"I DO NOT HAVE A FUCKING SOCIAL SECURITY NUMBER YOU STUPID PIECE OF SHIT COMPUTER."

"I'm sorry, I didn't catch that. Let's try again. Please say your social security number."

After about 50 more iterations it finally gave up and let me talk to a human, who didn't even care about social security numbers. I hope I wasn't too rude to him.

Re:Verizon, Fedex already there

[mjwx]

To be fair, I get the same with people, the machines aren't really any dumber.

Hold music fades...

Me: OK, I have a problem with my application.

Them: I can see your application.

M: Can you see how my name is misspelled.

T: Yes.

M: Can you fix it, it's just two transposed letters

T: I'll need you to send in your original birth certificate with a copy of your credit card, with passport, drivers license or other photo ID in addition to a urine sample, proof of noble linage, papal bull, letter from your doctor and bill less then three months old with your current address on it.

M: OK, can you cancel my application.

T: I'll need you to send in your original birth certificate

M: No I want to...

T: with a copy of your credit card, with passport, drivers license

M: No.

T: or other photo

M: Stop.

T: in addition to a urine sample, proof of noble linage, papal

M: SILENCE

T: ...

M: I would like to cancel my application.

T: I'll need to transfer you to another department.

Cue the hold music.

WHAT? SPEAK UP

[TiggertheMad]

its only cheating if the official Jepoardy rules dictate that hearing impaired contestants must remove hearing aids.

The answer is ... "42"

[RevWaldo]

Anyone? Anyone? (beedle-deet!) Watson!

.

Small sample size

[FrootLoops]

From the article,

In today’s exhibition of about 15 questions, Watson tallied $4,400, compared to $3,400 for Jennings and $1,200 Rutter.

15 questions is not really enough to say anything. Maybe the humans aren't used to buzzing in this context (I assume Watson's buzzer skills are basically static), they weren't "warmed up", or the categories favored Watson's strengths and their weaknesses. (Or maybe not.) Interestingly, if these 15 questions corresponded to 3 first round categories, every question was answered, from the money totals. I also find this interesting:

[Watson] buzzes in about half the time, and answers 85 to 95 percent of those questions correctly.

is this that surprising?

[shadowrat]

It's definitely cool that a program can play jeopardy, but i don't know if beating humans is all that amazing. Humans are terrible at jeopardy. They have terrible multi millisecond reaction times. They have poorly indexed data stores. They get easily distracted. Human brains are horribly prone to race conditions. When you get right down to it, the only part of the game people are good at is understanding the question. after you can do that, of course you can beat them.

Re:is this that surprising?

[An Onerous Coward]

Yes, but you forget. Ken Jennings is not human.

It's even more involved than that.

[sirwired]

If you click in too early, your clicker is disabled for a certain span of time because of your "false start". This keeps somebody from rapid-firing the clicker as soon as they think they know the answer.

If nobody else clicks in, you can answer, but the competitors get the first crack.

bwahaha

[billy8988]

From the article
"The questions were fed in plain text to Watson"

If "Watson" does speech recognition or Image processing for video clues then I'd be more impressed.
Its like playing chess with Blue where blue always plays with white.

Nevertheless, it's the first step...kudos to the team.

Have you ever actually watched Jeopardy?

[sirwired]

Jeopardy is not some retarded trivia show like "Who Wants to Be a Millionaire." Many of the questions involve puns, wordplay, rhymes, etc. that cannot be answered with a Google search.

Re:Have you ever actually watched Jeopardy?

[joeyblades]

Yes, I've watched Jeopardy - many times.

Most of the wordplay on Jeopardy is actually fairly trivial and often not vital to a heuristic search. A human may have to understand the wordplay to recall the answer, but Watson is free to follow multiple search paths simultaneously and pick the best match. Watson solves the problem differently. Make no mistake about it, there's no understanding going on in Watson.

Have you ever actually seen state-of-the-art natural language processing? Expert systems? Search engine heuristics? Parallel processing? Knowledge-bases?

Couple these state-of-the-art technologies with very fast hardware and you get impressive results... but only kinda impressive.

I could be wrong. Maybe I'll be blown away... but probably not.

somebody call Ray Kurzweil

[Wes42h]

It sounds like soon we will be able to put another "a computer will never be able to..." on the scrapheap.

"A computer will never be able to beat a human at Jeopardy!"

Why only three days on TV/television?

[antdude]

Does Watson lose after the third game or something? Does the game not let winner keep going until they lose like Ken Jennings did?

Re:Why only three days on TV/television?

[bws111]

It is a special tournament, like the celebrity/teen/college tournaments. Whoever has the most money after three days gets a million dollars, 2nd place gets $300,000 and third places gets $200,000. IBM will donate all it's winnings to charity (World Vision and World Community Grid), and the other two contestants have said they will donate 50% of their winnings to charities. See http://www-03.ibm.com/press/us/en/pressrelease/33373.wss

Re:Why only three days on TV/television?

[antdude]

Ah thanks! It would be radical/rad if they had no limits like Ken did. :)

Re:Why only three days on TV/television?

[3vi1]

Wouldn't exactly be fun for the current champion, who will be putting their next game on hold until the end of this special tournament.

Also, I don't think many human viewers would be tuning in to a show where a computer beats humans every night. If it can beat Jennings and Rutter, it's going to virtually destroy ~99.99% of Jeopardy contestants.

Re:Why only three days on TV/television?

[antdude]

One word: SkyNet. :)

I'll take "The rapists" for $400, Alex

[Anonymous Coward]

I'll take "Anal bum covers" for $200

How quickly does it know...

[mark-t]

... as much as a person about the answer that is being presented?

People have to actually physically read or hear the answer... no matter how fast a reader you are, this takes a moment, less time than it takes to verbalize the question, perhaps, but still not instantaneous. If the question is transmitted digitally to Watson at the same instant the question is revealed, Watson gains perhaps an entire second, maybe even as much as two, of advantage over the human players who might not have even finished parsing the Jeopardy answer.

Re:How quickly does it know...

[Haeleth]

The game is played reasonably fairly: that is to say, the computer and the humans receive the input at exactly the same time, parse it as quickly as they can, and try to be ready with an answer in time to hit their buzzer switch first.

It does appear that the computer may be receiving the input in digital form rather than being forced to read it from the same screen as the humans, which might be considered slightly unfair; but nobody considers reading to be a fundamental part of what makes Jeopardy what it is, and this is supposed to be a test of the computer's Jeopardy-playing ability, not its OCR ability.

Re:How quickly does it know...

[mark-t]

True... but receiving the input in digital form at the same time as the question is shown to the players means that Watson has a brief moment to cross reference information it might be able to obtain from the clue before anyone else has even finished reading the clue and knows what the heck it actually says. It's not long... perhaps as long as 2 seconds for more verbose clues, and probably almost always less than 1 second, but it could still spell a significant advantage.

Why Watson will win

[demi]

The reason Watson will win, even though it's probably not a better player than either Jennings or Rutter, is that Jennings and Rutter are playing against each other as well as Watson. I'm basing this gut evaluation on the scores for the practice game and the Watson "demo" that you can play online. In a game like Jeopardy with its three-player dynamics, it's harder to say that one player is better than another, based on a single game, than in a game like chess.

Obligatory Weird Al

[lennier]

I Lost On Jeopardy

Because the world always needs more Weird Al.

Isn't Watson Faster?

[Anonymous Coward]

Wouldn't Watson's primary advantage be that it would always buzz in first if it had the answer? How could anyone's timing be better?

armchair engineering

[Anonymous Coward]

if we're going to play armchair engineer why not ..

use a bloom filter for key words in the data to decide to hit the buzzer before you actually have parsed the question at all. maybe you'll guess wrong and lose a little bit, but better than waiting a couple of seconds before hitting the buzzer.

Re:$corporation == NAZIS

[Anonymous Coward]

You know this is always funny since the guys who sold the ZyblonB to the NAZIs were jewish chemists "not too far away from germany"

Also Coca cola was rebranded Fanta and sold in nazi germany. Enjoy (your next) coke.

Looking forward to it

[whereiswaldo]

I hope IBM nails this and makes it into something regular folk can use to find stuff on the web. The honeymoon is over - I'm tired of searching for things the "old fashioned way". I want Star Trek style technology where you ask a question and you get a succinct answer. No more trawling through pages of text trying to find nuggets of information.
Keyword searches can only get you so far.
Am I being lazy? Maybe. But after many thousands of web searches it gets a bit old. I really just want the answer!

Re:Looking forward to it

[Anonymous Coward]

still remember ask jeeves?