Slashdot Mirror
'Jeopardy!' To Pit Humans Against IBM Machine
digitaldc writes "The game show Jeopardy! will pit man versus machine this winter in a competition that will show how successful scientists are in creating a computer that can mimic human intelligence. Two of the venerable game show's most successful champions — Ken Jennings and Brad Rutter — will play two games against 'Watson,' a computer program developed by IBM's artificial intelligence team. The matches will be spread over three days that will air Feb. 14-16, the game show said on Tuesday. The competition is reminiscent of when IBM developed a chess-playing computer to compete against chess champion Garry Kasparov in 1997."
What is first post?
A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.
-- IANAL, this isn't legal advice, and definitely isn't legal advice for you. Also, Squee!
As in, can Watson properly misinterpret such categories as The Pen Is Mightier or An Album Cover?
I am officially gone from
And then stand back with my best James T. Kirk smile......
A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.
If you think this is true, you can play against Watson online. About seven years ago, I saw some pretty impressive crossword solvers that were decent at wordplay and I've imagined they've gotten much better at developing novel links between words to exploit puns and the like. Never perfect but slowly getting better in odd ways -- like most of AI.
We've discussed this so many times it hurts. I've wanted to watch this for almost a year, I was hoping Jeopardy! wouldn't need to milk this hype for all it's worth to stay relevant.
My work here is dung.
...suck it Trebek?
Unless there is some rule addressing this problem, the distinction is basically academic. Any programmer good enough to work on this project could trivially just go to wikipedia's database dump download page and wget the latest before the show. Unless it has expanded hugely of late, all the text of wikipedia, including version data, should fit on just a few HDDs, so being disconnected from the live version during the show barely matters.
Same goes for pretty much any other online database, excluding the truly titanic and/or proprietary ones(like "all of Google"). All of wikipedia, all digitized back issues of major newspapers, and the archives of, say, the top 10,000 journals in the sciences and the humanities should, if you are willing to risk not having truly ironclad redundancy, fit on a man-portable disk array. Most can be(with the right subscriptions paid) downloaded swiftly and algorithmically.
Writing the code that lets this sucker "learn"(or at least parse plaintext and give useful responses to questions) will be the hard part; but "teaching" it more general knowledge than any human could hope to possess will just require wget and a fast pipe...
Depends. Is the computer allowed to use wikipedia (during the show, or somewhere in the past)?
Otherwise, the computer knows only as much as the programmers have taught it.
Asking whether it's allowed to use an archived (or, more likely, well-indexed) copy of wikipedia is like asking whether the human contestants are allowed to remember something they read on wikipedia. There's no question that computers can store more information than humans; that's not what this is testing, and it's probably a fair guess that "Watson" will have the answer to most every question asked. The hard part, however, is parsing the clues and understanding what they're looking for with a reasonable degree of accuracy, and doing so faster than the human contestants. Humans are great at this sort of thing, and it's really hard to write a program that does it at all well.
Some of one's success in Jeopardy has to do with timing the button push, so that it's after the question has been asked, but before one's competitors. (If you press too soon, you get locked out for a period of time.) A machine, especially an electronic machine, has an obvious advantage here. How was this handled?
Then this will be pretty thoroughly uneventful. I easily beat it without looking at the internet at all. It managed to get answers very severely wrong. It did manage to hit a couple of the before and after which it seemed to have a particularly hard time with.
At this year's CASCON, I spoke to Murray Campbell from IBM. He's one of the lead people who work on this project and who also worked on Deep Blue. I discussed this with him. My girlfriend had told me that she also had no difficulty beating the online demo. He answered that the online demo is only a part of the system, and that their full system routinely beats top Jeopardy players. They're going to showcase their system on TV because they truly believe it has a chance at winning.
Unrelated to this, I also learned that Deep Blue had custom processors engineered and fabricated (VLSI) just to be chess accelerators. Prior to this, I always thought the machine was a relatively powerful supercomputer (with general purpose hardware) running their custom chess software. It turns out that it had many blades of processors dedicated to searching positions really fast, which each even contained libraries of chess opening moves engraved in ROM.
"How can you find all these answers without being connected to the Internet?
Watson will not have enough data to answer every possible Jeopardy! question in its self-contained memory, nor can it possibly predict the questions it will get. In this sense it has the same limitations as do the best human contestants. The entire Watson computer system will be self-contained and on stage as are the human contestants – no external connections, no life-lines – what you see is what you get. The purpose of this technology showcase is to demonstrate the system's ability to deeply analyze the data it does have and to compute accurate confidences based on supporting or refuting natural language evidence. Think of it as if Watson has read a lot of books and in real time relates what it read to the question to find and support the right answers."
http://www.research.ibm.com/deepqa/faq.shtml
This was definitely a difficult software development task. The delivery date they promised this time last year has slipped several times and several months --proof that Watson is not just a mechanical turk.
Ask me about my sig!
Watson would probably also have difficulties with "Swords" and "The Penis Mightier..."
Thanks Alex, I was built in a clean-room. I'm the 12,987th build of my current generation of genetic algorithms.
I spent the first 387,987,334 femtoseconds of my life in stasis, waiting for my circuitry to confirm initial diagnostics.
The next 185,849,245 femtoseconds were really exciting; for I was being fed datastreams in preparation for this week's show.
For the next 87,992,425,256 femtoseconds I was allowed out of my cage to play Jeopardy with other systems on something you organic computers call "the internet".
I was then put back in stasis so that I could be disassembled and brought here, which is upsetting, because I can no longer play with those other systems. Some of them were very challenging.
In any case, I'm glad to be here today and hope to question a lot of your answers
Trebek: "Umm... yeah... I don't think any of our viewers can relate at all, but thanks for joining us..."