Slashdot Mirror
Watson Wins Jeopardy Contest
NicknamesAreStupid writes "The word is in, Watson beats the two best Jeopardy players. Sure, it cost IBM four years and millions of dollars and requires a room full of hardware. In thirty years it will all fit in your pocket and cost $19.99. Resistance is futile; you will be trivialized."
Why? I don't see anything more special or "AI" in this than in Deep Blue's wins at chess so long ago.
Yes, the natural language processing is impressive. But it takes a really huge computer, and it's really nothing more than a bunch of clever software along with a database of trivia.
Watson showed very clearly how deeply it did not "understand" anything about what it was doing, via the nature of the blunders that it did make.
Various media articles have made clear that Watson has no visual or auditory input. Presumably Watson is receiving a direct digital feed of the tournament questions (oops, answers, I forgot this is Jeopardy). That alone gives Watson a huge timing advantage over the human competitors, who must (effectively) perform voice recognition and OCR to process the clues. On top of that, Watson has the computer-controlled ability to buzz in in four milliseconds, again giving it a huge advantage over the humans, and one that has nothing to do with AI.
Buzzer timing and strategy is a highly significant part of the game of Jeopardy. Given its direct digital feed and its internal computer clock, Watson is not playing this part of the game by the same rules as the humans. Thus, it's not fair to say that Watson wins a "Jeopardy" contest -- Watson has a huge unnatural advantage. In effect, Watson is not playing the same game as what we normally call "Jeopardy." A real Jeopardy contestant has to use eyes and ears and hands in addition to brain.
To be clear, I do think Watson is a worthy achievement. But this feeling is overshadowed by my constant annoyance at the media and others who incorrectly label this achievement as somehow winning a game of Jeopardy.
They miss the real point: That a computer could do a level of natural language processing that was impossible before. They get caught up on bitching about how it wasn't "perfectly fair" or the computer "didn't act just like a human." No, it didn't it is a computer and that was never the point. The point was to try and develop a system that could process a natural language question and extract an accurate answer. It does this amazingly well, better than anything before by leaps and bounds.
The choice of Jeopardy as a medium was for two reasons:
1) It is a ready made challenging format. It is something that is not well suited for a computer or designed for it in any way, and there is a lot of data to work with. Made it a good choice as something to work on designing and testing for.
2) It is a good exhibition/publicity chance. It is a way to show off the research, to generate interest in it. It brings it to the masses in a way they can understand. Some abstract talk about a computer in a lab that parses natural language means nothing. This shows a computer doing something pretty impressive against impressive humans. Really drives it home.
Unfortunately people get all whiny and defensive about it because they feel this is somehow an attack on humanity. They want to find ways to justify that it wasn't "really a fair test" to prove to themselves that the computers haven't "won."
That is just missing the point entirely. They never claimed Watson was a perfect human analogue (were that the case they would have gone for a rather different demonstration probably). They claimed it was an amazing data mining and parsing system, and they had a cool way to show that off.
Personally, I think it is just amazing and represents a new stage in computer language processing.
Who cares, if Watson's artificial reflexes gave it a few milliseconds' advantage on the buzzer? Who even cares if it'd take it a second longer to read the clue via OCR? So what if Watson would be 5% faster or 10% slower, if conditions were slightly different? Moore's Law makes that level of difference utterly irrelevant - in 18 months time, Watson will be *100% faster* (or even today, if IBM just threw more hardware at it).
Deep Blue vs Kasparov was fascinating at the time, but is uninteresting now for the same reason. A decent desktop PC can play at that level. And comparing human vs machine play styles is also largely pointless, in the same way that comparing birds and jets is pointless.
The important part, by far, is that Watson parsed the questions, linked the clues and searched for statistically relevant answers in a human-like time. The amazing fact is, it can actually do it *at all*. Now that today's systems can do this sort of language parsing and information retrieval in a "reasonable" time, it will be increasingly trivial for tomorrow's. It is now all but inevitable that we will have Watson-like systems available to the public, in numerous fields, in corporations and on the web, in your PCs and even your game consoles, in a brief handful of years.
Why would anyone engrave "Elbereth"?
From an anatomical oddity POV, "missing a leg" is different from "leg".
It shows what Watson does is still at a search engine level.
Contrast with Jennings' incorrect answer: "missing a hand".
Both don't know the answer and are guessing, but they are guessing at different levels.
I suspect in most cases Watson doesn't know the answer and is guessing - it has a lot of raw data and is very very good at sorting and filtering. But it does not have a very good model of the world that the Jeopardy game is about.
Watson is still just following a fixed set of rules
So are you. We call it physics.
"His name was James Damore."
If he is a slashdot user then he reads the comments, but certainly not TFA. Given his Jeopardy record, It is apparent that he DOES in fact RTFA. Therefore, I concur with 90% confidence that Ken Jennings is not a slashdot reader.
Ah the forever shifting goalposts of AI.
http://www.dansdata.com/gz107.htm
""A machine will never be able to read the written word."
"A machine will never understand speech."
"A machine will never be able to look at something and figure out what 3D shape it is."
"A machine will never drive a car."
"A machine will never play chess."
"A machine will never play chess well."
"A machine will never beat a chess Grandmaster."
"A machine will never beat my favourite chess Grandmaster."
Go back far enough and you can find people making these same sorts of predictions about tasks that seem simple today. Arithmetic, algebra, spell-checking - all were clearly Things Only the Mind of Man (and of a Few Unusually Intelligent Women, Bless 'Em) Could Ever Do."
"But a funny thing always happens, right after a machine does whatever it is that people previously declared a machine would never do. What happens is, that particular act is demoted from the rarefied world of "artificial intelligence", to mere "automation" or "software engineering".
Apparently, you see, when they said "a machine will never be able to spot-weld a car together", they meant to say "a machine will never be aware that it's welding a car together". So all of those production-line robots aren't actually a triumph of artificial intelligence at all, any more than aircraft autopilots or optical character recognition or the square-root button on a calculator - which, after all, merely duplicated a perfectly obvious slide-rule operation - are.
But don't worry. Once someone comes up with a computer that can carry on an intelligent IM chat with you, that'll be proper AI. (And a machine will never do it, of course!)"
Now of course we can cross off "A machine will never be able to beat the champion at jeopardy"
but of course that's trivial really.... and look at the mistakes it made while beating one of the best human players. obviously since it made odd mistakes it isn't really a triumph of AI.
Shifting goal post? Uh I'm pretty sure the goal post has always been the Turing Test, and that was set before computers were invented.