Slashdot Mirror
'Jeopardy!' To Pit Humans Against IBM Machine
digitaldc writes "The game show Jeopardy! will pit man versus machine this winter in a competition that will show how successful scientists are in creating a computer that can mimic human intelligence. Two of the venerable game show's most successful champions — Ken Jennings and Brad Rutter — will play two games against 'Watson,' a computer program developed by IBM's artificial intelligence team. The matches will be spread over three days that will air Feb. 14-16, the game show said on Tuesday. The competition is reminiscent of when IBM developed a chess-playing computer to compete against chess champion Garry Kasparov in 1997."
What is first post?
A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
As in, can Watson properly misinterpret such categories as The Pen Is Mightier or An Album Cover?
I am officially gone from
Would you like to play a game of chess?
Just make all the questions of the type where you have to regurgitate some copyrighted information (say a piece of lyrics), which IBM won't be able to store.
So IBM invented Google? I wish science reporting would be more accurate and say "IBM invents program that takes human speech in a particular sentence structure, parses the grammar, and Googles for a result."
And then stand back with my best James T. Kirk smile......
It will look nothing like the computer "maid" on "The Jetsons."
Who thought that had anything to do with it? I think it's time that we as a culture realized that Rosie is decidedly not what people think of when they hear the word "computer."
I live in constant fear of the Coming of the Red Spiders.
Call me when American Gladiators, Lord of the Flies, or Surviving the Game (Staring Ice T) pits humans against an IBM Machine.
THL phish sticks
A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.
Depends. Is the computer allowed to use wikipedia (during the show, or somewhere in the past)?
Otherwise, the computer knows only as much as the programmers have taught it.
A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.
If you think this is true, you can play against Watson online. About seven years ago, I saw some pretty impressive crossword solvers that were decent at wordplay and I've imagined they've gotten much better at developing novel links between words to exploit puns and the like. Never perfect but slowly getting better in odd ways -- like most of AI.
We've discussed this so many times it hurts. I've wanted to watch this for almost a year, I was hoping Jeopardy! wouldn't need to milk this hype for all it's worth to stay relevant.
My work here is dung.
...suck it Trebek?
Later. Let's play Global Thermonuclear War.
Anyone feel the basic answer/question reversal conceit of this game doesn't quite make sense. If you switched them around, nobody would ever say the answer written on the card.
1. "You are not connected to the Internet" was shown instead of the answer
2. The computer's final jeopardy answer were revealed to be a blue screen of death
3. A porn answer came up, like "Nalin' Palin"
But I guess that's why they pre-tape these things.
I hate being bipolar; it's awesome!
"Watson" is named for IBM founder Thomas J. Watson. It will look nothing like the computer "maid" on "The Jetsons." Rather, IBM said its on-screen appearance will be represented by a round avatar.
A parallelogram avatar would have been better.
He who knows best knows how little he knows. - Thomas Jefferson
http://www.youtube.com/watch?v=FC3IryWr4c8
Not to mention voice recognition, if this is anything like IBM's VoiceType on the Aptiva, the host will have to train the machine for the first hour of the show, and will spend the rest of the time talking like Shatner and it'll still get it wrong.
Twinstiq, game news
Then this will be pretty thoroughly uneventful. I easily beat it without looking at the internet at all. It managed to get answers very severely wrong. It did manage to hit a couple of the before and after which it seemed to have a particularly hard time with.
XML is like violence. If it doesn't solve the problem, use more.
Some of one's success in Jeopardy has to do with timing the button push, so that it's after the question has been asked, but before one's competitors. (If you press too soon, you get locked out for a period of time.) A machine, especially an electronic machine, has an obvious advantage here. How was this handled?
The machine will win. Open and shut case. Though, Alex Trebek's lacerating wordplay might present some obstacles.
Okay... I pooped. AND it is pretty smelly. Holy shit indeed ^__^
Cripes, I hope they don't give it a 1970s/1980s sounding computer voice.
In many ways, this challenge is vastly more complex than chess. The game of chess is incredibly difficult, but it is pretty well understood, and has a very restricted number of rules. Natural language processing is another problem altogether. The computer has a much faster access to facts, but processing the "answer" in order to create a question in the right context is going to be huge. Personally, I wouldn't place any bets either way as to which side I think will win in this one.
Your Servant, B. Baggins
I think the real news item is the fact that Jeopardy is still on TV.
The game has an advantage for the human - you get as long as you want and you always get first crack. Still, there is this to underscore your point:
Answer: "Sing a song about one of these, a sailor's bag for small articles"
Watson: "What is 'Poppa's got a brand new bag.'"
(ditty bag was the correct answer)
The computer will be much faster at pushing the button.
Have gnu, will travel.
This was news last year. That post contained a reference to http://games.slashdot.org/story/09/04/27/1354213/IBM-Computer-Program-To-Take-On-Jeopardy -- April 27, 2009. Interesting, but old.
Sic transit gloria mundi.
You can already play against Watson here. However it seems to be a very limited question bank so Watson may have an unfair advantage there.
Making an "AI" chess program does not require creativity. It requires the ability of a program to follow the rules of the game and to generate strategy to suit the moves of the opponent. Some might say this requires creativity, but not really. To answer why it's not creativity, we have to define creativity.
Answers.com says it is the ability to produce something new through imaginative skill. Reference.com says it is the ability to transcend traditional ideas, rules, patters, relationships, or the like, to create meaningful new ideas, forms, methods, interpretations, etc. This is something we have not yet created in machine intelligence. We can instruct computers to collect information and sort it and categorize it and reference it and so on, but we cannot yet tell a machine to "understand" anything. By this, I mean to make the machine collect information and transform it into new processes the way an animal brain does.
I have long held that it is the nature of the animal brain that produces creativity. Our brains are extremely "error prone" and noisy inside. This is to our advantage rather than our detriment. While aggregate results lead to answers, the errors and noise, I believe, are what leads to new ideas and creativity. Further, our ability to model information into processes is at the root of what it means to be creative and to understand ideas and concepts.
I'm not saying that computers can't do these things. I'm saying that today's computers can't do these things. The reason is because of their accuracy and their design that is intended to block out errors, noise and randomness.
To understand what I am getting at, let's look at various human cultures. Cultures where behavior is strictly regulated and limited show less creativity. This can be seen in cultures of religion and in quite a few Asian cultures. These cultures tend to insist that everyone think and believe the same things and show intolerance for new ideas. In contrast, western cultures and especially those that thrive on diversity, tend to offer much more in the way of creative and imaginative results.
The Jeopardy project is a lot more ambitious than a chess program that can beat humans. It requires the collection of all sorts of information and to make associations between bits of information. It is largely what Google and other search providers are already doing. It's is a huge task and never complete and never perfect. Even now, those tasks require human correction and intervention to yield the results that are desired. It is an amazingly difficult thing to pull off when the accuracy of the machines in operation come first.
I believe that when computer systems are designed to embrace errors and deviations rather than following only "one correct path" with only "one correct answer" then we will begin to see creativity and intelligence in computers.
Agreed, I crushed it. I am very impressed about its ability to answer some questions (It actually got "Black Death of a Salesman" and "Charlie Brown Recluse"), which shows that it has some very sophisticated linguistic analysis, but if it can't beat some random shmuck on the Internet, I don't see how this will be an interesting event.
ZFS: because love is never having to say fsck
And the answer is...
I saw a presentation given on this at NASA. It is truly an amazing feat of technology. They only need about 2TB of data, the system is not connected to the internet. They can currently beat Jeopardy champions 70% of the time (there is luck involved). They use multiple methods of finding the answer then weigh them to come up with an uncertainty factor. The grand champions they are playing against are in a class unto themselves compared with Jeopardy winners in general.
I see this more of a case of Watson having to lose than either of the contestants being able to win. Considering Watson will have an answer to any question before the humans can, and has machine timing, it will be guaranteed to buzz in first. At that point it just boils down to if the parser is confident enough in its answer to provide it and if said answer is right or wrong. For the humans to have a chance, the categories need to be the wordplay and other related trickeries. Even then, it's not truly people vs. machine, it's person vs. person vs. machine, so the human score will be diluted between two contestants.
I saw the computer version earlier and while it was good enough to beat me, the Jeopardy champions are far better. In addition, the version I saw demonstrated was 'advanced' - by that they meant that further advancements resulted in only tiny, incremental performace increases, not by significant ones.
excitingthingstodo.blogspot.com
They forgot the code that provides the answers in question form.
I'll take "The Penis Mightier" for 5000, Alec.
What's even worse is that apparently Watson plays the same game with the same questions over and over on that site and still doesn't know the answer to half of them.
Then this will be pretty thoroughly uneventful. I easily beat it without looking at the internet at all. It managed to get answers very severely wrong. It did manage to hit a couple of the before and after which it seemed to have a particularly hard time with.
At this year's CASCON, I spoke to Murray Campbell from IBM. He's one of the lead people who work on this project and who also worked on Deep Blue. I discussed this with him. My girlfriend had told me that she also had no difficulty beating the online demo. He answered that the online demo is only a part of the system, and that their full system routinely beats top Jeopardy players. They're going to showcase their system on TV because they truly believe it has a chance at winning.
Unrelated to this, I also learned that Deep Blue had custom processors engineered and fabricated (VLSI) just to be chess accelerators. Prior to this, I always thought the machine was a relatively powerful supercomputer (with general purpose hardware) running their custom chess software. It turns out that it had many blades of processors dedicated to searching positions really fast, which each even contained libraries of chess opening moves engraved in ROM.
This is not artificial intelligence, it is corporate intelligence - the ability of corporations to deal with situations that they really do not understand. First, IBM showed that a modern corporation could defeat a chess grandmaster, now they are taking on Jeopardy (which should actually be easier). The fact that machines are involved is incidental, given the large number of corporate employees required to program these machines, detect flaws in their code, and correct the programming accordingly.
This was definitely a difficult software development task. The delivery date they promised this time last year has slipped several times and several months --proof that Watson is not just a mechanical turk.
Ask me about my sig!
Watson would probably also have difficulties with "Swords" and "The Penis Mightier..."
I bet this won't be sent in all countries, will it be possible to view this on the internet somewhere?
"Famous Motherboards".
Thanks Alex, I was built in a clean-room. I'm the 12,987th build of my current generation of genetic algorithms.
I spent the first 387,987,334 femtoseconds of my life in stasis, waiting for my circuitry to confirm initial diagnostics.
The next 185,849,245 femtoseconds were really exciting; for I was being fed datastreams in preparation for this week's show.
For the next 87,992,425,256 femtoseconds I was allowed out of my cage to play Jeopardy with other systems on something you organic computers call "the internet".
I was then put back in stasis so that I could be disassembled and brought here, which is upsetting, because I can no longer play with those other systems. Some of them were very challenging.
In any case, I'm glad to be here today and hope to question a lot of your answers
Trebek: "Umm... yeah... I don't think any of our viewers can relate at all, but thanks for joining us..."
"Do you want the prize behind pod door number one, pod door number two, or pod door number three?"
Table-ized A.I.
That noticed this is on valentine's day?
AccountKiller
This is intriguing.
How is "Watson" going to get the clue?
Is it going to recognize Trebek's voice? - If yes, human competitors may have an advantage.
Or "Watson" will have a camera pointed at the clue board to do a OCR? - What about the video/audio clues. Are they setting some ground rule so that there won't be any audio/video clues?
Or Clues are directly fed into "Watson"? - If yes, "Watson" may have an advantage.
Interesting...can't wait to hear more about this.
Well, the game is afoot I'll take anal bum cover for 7,000.
I've abandoned my search for truth; now I'm just looking for some useful delusions.
A machine, especially an electronic machine, has an obvious advantage here. How was this handled?
The same way eBay auctions are handled, each player gets a sniper-bot and you just see which one hits on the right nanosecond.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I really wonder how it will handle wagering during Daily Doubles and Final Jeopardy? How will it assess its own 'knowledge' of a category, and will it take into consideration other contestant's scores, difficulty and probability of winning questions from remaining categories, etc.?
Maybe we'll see a true Daily Double...
This will be most interesting indeed!
"I love animals! Some are cute, others are tasty, what's not to like?" - Betsy Schroeder, Jeopardy contestant
Since the questions are formed as answers, and the answers have to be the form of a question, it just seems a lot like the sort of thing that confused the HAL9000. I think we're all going to get sucked out the airlock.
(If at first you don't succeed, do it different next time!)
Perhaps we should see how fast the "Watson" computer can search when given an energy budget of 2500 Kcal/day (~10MJ/day ~120W). Or conversely, we should penalize the reaction time accordingly (probably would have to wait days for the computer to answer). Or maybe allow an additional team member for every 120W...
Now that would be a real challenge, I'd be willing to see. Given the human energy conversion efficiency is only about 20% and a typical computer power supply is about 70%, that still giving the computer a significant advantage...
Suck it, Trebek!
It is the final digit of Pi.
If you read the full article, Watson was regularly crushing human competition. They've been testing it. They also mention Watson takes a "room full of servers" to run, so I doubt you're actually playing against it on the NYT site. Finally, that game was quite a bit different than jeopardy in that there are no timers and you get to answer every question or pass before Watson has a chance to buzz in. It should be interesting to see how Watson does in February.
A Microsoft Kinect type approach would be the most fair in my estimation, giving "Watson" auditory and visual capability. Since clues are displayed on screen before Trebek is finished reading them aloud "Watson" could use both input types to attack the problem.
I beat the computer too, but what was most interesting were the results after I answered and seeing it's confidence levels for various answers. Often it did have the right answer, even sometimes when there was word play.
You did have the advantage of always getting first pick of answer or not. If the computer was able to buzz in, it may very well buzz in before you and take those points. It's confidence rating was interesting to watch.
"... if it can't beat some random shmuck on the Internet, I don't see how this will be an interesting event."
If you can't see the obvious differences between the little game on that web site and how this is going to work during a real Jeopardy match, then your self-description is entirely apropos. I think this is going to be awesome! I'd like to see every arrogant ass on this comment board with their "Yawns" and other dismissals of this achievement go up against the machine in real time. You're all in dire need of a good intellectual smackdown.
And they cheated too (it had human assistance midmatch).
The demo inherently favors the human player, who has the right of first refusal and no time limit. I'd wager that any moderately curious high-school graduate could win the demo with ease.
https://www.eff.org/https-everywhere
How will this differ from having a natural language parser and a database of answers?
well, im good at jeopardy, but this wasnt jeopardy. i wonder how quickly watson would have answered in real time? his answers were scarily accurate, considering that some of the questions required complex world play and association. of course, what we need is a computer like this that is doing most of the learning on its own, not guided by human input. i say we need to find a way to program in a reward system that can generate pleasure for the computer. i really dont know what that would be. im not talking about a pleasure simulation, but a pleasure ANALOG. would it be guaranteed access to its own power supply? the ability to order more memory chips? a synthesized hormone system to give pleasure feedback?
A Microsoft Kinect type approach would be the most fair in my estimation, giving "Watson" auditory and visual capability. Since clues are displayed on screen before Trebek is finished reading them aloud "Watson" could use both input types to attack the problem.
Dude. What's with the quotes man?