Slashdot Mirror
Watson Wins Jeopardy Contest
NicknamesAreStupid writes "The word is in, Watson beats the two best Jeopardy players. Sure, it cost IBM four years and millions of dollars and requires a room full of hardware. In thirty years it will all fit in your pocket and cost $19.99. Resistance is futile; you will be trivialized."
I think it is safe to say the AI Winter is over.
sig = null;
Does Ken Jennings read Slashdot comments?
can it do my homework for me?
---
I for one welcome our new computer overload.
Someday we'll hit the human carrying capacity. And the band will just play on.
I didn't watch it all, but the thing I noticed was that, when Watson thought it had an answer, most of the time it'd click in first. The other contestants didn't have a chance to attempt to answer.
So Watson wins on reaction time, which isn't a surprise for a computer that knows exactly when it can first ring in. How would it have done with a human's reaction time on clicking, just answering on questions alone?
That said, I don't want to dismiss the natural language recognition capabilities of Watson. They are no small feat, and by all rights, the designers of it should be congratulated on this effort. Nevertheless, with respect to the game of Jeopardy, I remain convinced that Watson's key advantage over the other players was that it is essentially a super-fast speed reader, having a few moments to pontificate the clue before any human could possibly be finished reading it. If the text of the clue had been transmitted to Watson more slowly to approximate the menial task of reading, I think it might have been a better indicator of whether or not Watson was actually out-thinking Brad and Ken. A speed I think would be appropriate to transmit the text of the clue at is about the same as what you'd get with a 14,400 bps modem, which still would amount to insanely fast speed reading, but it's at least within an order of magnitude of what is humanly achievable. Then, the amount of time that Watson has to think about the clue gets a lot closer to how much time the other players get to think about it. As it sits, Watson gets to start trying to parse the entire sentence before any human has even finished reading the first word.
Of course, I don't think that Deep Blue really out-played Kasparov on a level playing field either... I would be far more impressed if they could design a chess-playing computer that only considers a few hundred board combinations and still plays at a grandmaster level, since that is all that even the best human grandmasters do.
File under 'M' for 'Manic ranting'
...it was us that scorched the sky...
I hate being bipolar; it's awesome!
Various media articles have made clear that Watson has no visual or auditory input. Presumably Watson is receiving a direct digital feed of the tournament questions (oops, answers, I forgot this is Jeopardy). That alone gives Watson a huge timing advantage over the human competitors, who must (effectively) perform voice recognition and OCR to process the clues. On top of that, Watson has the computer-controlled ability to buzz in in four milliseconds, again giving it a huge advantage over the humans, and one that has nothing to do with AI.
Buzzer timing and strategy is a highly significant part of the game of Jeopardy. Given its direct digital feed and its internal computer clock, Watson is not playing this part of the game by the same rules as the humans. Thus, it's not fair to say that Watson wins a "Jeopardy" contest -- Watson has a huge unnatural advantage. In effect, Watson is not playing the same game as what we normally call "Jeopardy." A real Jeopardy contestant has to use eyes and ears and hands in addition to brain.
To be clear, I do think Watson is a worthy achievement. But this feeling is overshadowed by my constant annoyance at the media and others who incorrectly label this achievement as somehow winning a game of Jeopardy.
The IBM Research blog has had a few good articles about Watson over the past few days, including one about wagering:
http://ibmresearchnews.blogspot.com/2011/02/watsons-wagering-strategies.html
I didn't think that Final Jeopardy would have been especially easy for Watson. The majority of the clue was indirectly related to the correct response, and the connection hinged on a single word (inspired). I suspect Jennings' behavior was based more on simple arithmetic than on any assumptions about Watson's response.
Uh - Watson obviously, obviously had a speed advantage. On today's episode there were many, many obvious answers (obvious to me - to Ken Jennings or Brad Rutter, blindingly - stupidly blindingly - obvious). Watson got almost all the obvious questions, and many times you could see the little eye roll of frustration from Ken and Brad.
On questions like this, Ken and Brad would have been waiting and trying to time the ring in (they would have known the answer long before the buzzer was active).
They lost almost every time.
So while the computer may not have had an absolute advantage (ie. if Ken could have rung in within milliseconds of the buzzer being active he would have been OK) it's clear it had an effective one in that it's a bloody machine that can ring in very quickly after the buzzer is active. I mean, yeah, it's cool that the computer knows the answer that fast - but we didn't get to see anything of a comparison of who knew a greater percentage of answers.
And, again, this was absolutely, obviously clear to anyone who watched the show. I don't believe anyone could have watched the show and not realize this. Honestly, this is a problem even without the computer. Between high level contestants, buzzer speed is going to determine the winner 9 times out of 10 - the Jeopardy questions just aren't hard enough to distinguish between the best competitors. Oh, and somewhere else in this article someone said "that's part of the game". That's true. But it's a stupid part of the game - and it makes it impossible to compare competitors with different brain technologies in any interesting way.
Let's not stir that bag of worms...
There was a practice round, and it was widely circulated that Watson had won that, though it wasn't true - it dominated the first two rounds but Ken pulled ahead with a huge wager in a double jeopardy and then another in final jeopardy (which Watson got wrong). The results of the actual challenge did not leak.
That is incorrect: Wired.com: And of course, the best Jeopardy players sometimes ring in before they may have come up with the answer, if they have, dare I say it, a gut feeling or sense of intuition that theyâ(TM)ll be able to answer correctly, right? Watson canâ(TM)t do that, can it? Brown: The IBM Research team made a decision that we were not going to ring in unless Watson had already computed an answer with high-enough confidence. There are human players who may have an intuition that they know the answer but donâ(TM)t quite have it on the tip of their tongue, and are willing to ring in because they are confident enough that they will come up with the correct answer in the few seconds they have to actually answer after theyâ(TM)ve won the buzz. That was an implementation decision for Watson that it had to have an answer with a high-enough confidence before it would attempt to ring in. http://www.wired.com/epicenter/2011/02/ibm-watson-speed/
I thought the wrong/skipped answers were much more illuminating than the right answers.
For example, much has been made of Watson's "Toronto" answer to the US Cities question in Game 1. However, it wasn't a terrible answer because one of Toronto's airports is named after a war hero (Billy Bishop, the WWI fighter ace who shot down the Red Baron), and the main airport (Pearson) was named after a politician who was also a WWI veteran. Watson knew that Toronto wasn't in the US, the war was wrong and neither were named after a battle, but Toronto was the least wrong of all its options so that's what it chose. If this question had come up in the regular rounds Watson would have skipped (as happened occasionally). However, it needed to answer so it went with the best available option.
Now, since Watson would certainly have had data on O'Hare, Midway and Chicago in its database, the problem was either in the question parsing or the search heuristics. One suspects that its weakness is the linking together of disparate data, and it's quite likely that humans will retain this edge for some time.
When you have nothing left to burn you must set yourself on fire
Thankyou, thankyou. I'll be here all week, make sure to tip your waitress.
She is a cow, and that is a pastime in my part of the country...
I don't post AC. I like my -1, Flamebaits. Trump/Sheen 2012 on the Batshit Insane ticket!
Even assuming Moore's Law holds, it's roughly $1-2 billion. Now, it looks like Moore said 2 years, but let's be generous and assume 18 months -- 10 years is 120 months, 120/18 is roughly 6.67 iterations of Moore's Law -- let's be generous and round up to 7.
2**7 is 128. So assuming it stays exactly the same size, the very best you can expect is $1 billion / 128 = $7,812,500. Could software save it? Maybe, if you expect software to get 390,625 times faster.
I can't find much on the dimensions, but it's a room-sized cluster right now. What's a 128th of a room? A quick Google suggests the typical college dorm room is 12'x19', so 228 square feet. So, a 128th of the area is still more than a square foot... times however tall it is. Again, you'd have to expect the software to get many times faster.
And predictions are that Moore's Law is going to slow down or outright stop, not speed up. Experience shows that software tends to at best stay about the same speed, if not get slower as people take advantage of higher-level (but often slower) constructs both to manage complexity and out of laziness (the power's available, may as well use it). Sometimes it does improve, but it's certainly not the norm.
Still think it'll take 10 years? Maybe in 5 there'll be a terminal to one that fits in your pocket, but the actual machine? I very much doubt it.
Don't thank God, thank a doctor!
It was probably intentional that they did not give it a realistic, human-sounding voice. Research has shown that people do not want machines to appear too human. They react negatively.
[citation needed]
[Star Trek and Asimov references don't count]
Actually, I think humans DO want robots that appear very human, and have wanted them for hundreds of years. I'd also put it to you that humans do and have, in fact, reacted in certain positive ways towards machines that appear human.
The dame de voyage (French) or dama de viaje (Spanish) was a direct predecessor to today's sex dolls that originated in the seventeenth century. Dames de voyage were makeshift fornicatory dolls made of sewn cloth or old clothes, used by French and Spanish sailors while isolated at sea during long voyages.
-- Ferguson, Anthony. The Sex Doll: A History. McFarland, 2010
One of the earliest recorded appearances of manufactured sex dolls dates to 1908, in Iwan Bloch's The Sexual Life of Our Time. Bloch wrote:
In this connection we may refer to fornicatory acts effected with artificial imitations of the human body, or of individual parts of that body. There exist true Vaucansons in this province of pornographic technology, clever mechanics who, from rubber and other plastic materials, prepare entire male or female bodies, which, as hommes or dames de voyage, subserve fornicatory purposes. More especially are the genital organs represented in a manner true to nature. Even the secretion of Bartholin's glans is imitated, by means of a "pneumatic tube" filled with oil. Similarly, by means of fluid and suitable apparatus, the ejaculation of the semen is imitated. Such artificial human beings are actually offered for sale in the catalogue of certain manufacturers of "Parisian rubber articles.
-- Bloch, Iwan. The Sexual Life of Our Time
So, yeah, it may be a bit taboo to some people, but not admitting to your family that your girlfriend is a Nexus 6 doesn't count as "reacting negatively" to the idea of humanoid machines.
On the average, I'd say people at fascinated by human-like machines (see: Androids/cyborgs in science fiction, or, uh, the 80s/early 90s for fuck's sake, it was full of interest) -- Curiosity is a positive trait in my book, and lusting after our machines is a trait that people from gear-heads to PC Gamers and realdoll owners all share to a degree.
(Keep in mind: As a new technolgy TV was unsettling to some, but like all common place technology it's not a big deal now)
This computer later became Skynet.
Yet ironically, Watson failed to score on any of the "Also On Your Computer Keys" category.
Some poor IBM intern is getting reamed for not giving Watson a keyboard to refer to, on the podium.
Isaac Asimov saw the day coming when man would not be smart enough to design the next version of a computer. The computer would have to design its own successor, each supremely more powerful and yet smaller than the last. Extrapolating this trend over many iterations, the computer becomes uniquely supreme in power and omniscience. But we have a different word for that.
A classic only a few pages long and worth a read, The Last Question, by Isaac Asimov.
They miss the real point: That a computer could do a level of natural language processing that was impossible before. They get caught up on bitching about how it wasn't "perfectly fair" or the computer "didn't act just like a human." No, it didn't it is a computer and that was never the point. The point was to try and develop a system that could process a natural language question and extract an accurate answer. It does this amazingly well, better than anything before by leaps and bounds.
The choice of Jeopardy as a medium was for two reasons:
1) It is a ready made challenging format. It is something that is not well suited for a computer or designed for it in any way, and there is a lot of data to work with. Made it a good choice as something to work on designing and testing for.
2) It is a good exhibition/publicity chance. It is a way to show off the research, to generate interest in it. It brings it to the masses in a way they can understand. Some abstract talk about a computer in a lab that parses natural language means nothing. This shows a computer doing something pretty impressive against impressive humans. Really drives it home.
Unfortunately people get all whiny and defensive about it because they feel this is somehow an attack on humanity. They want to find ways to justify that it wasn't "really a fair test" to prove to themselves that the computers haven't "won."
That is just missing the point entirely. They never claimed Watson was a perfect human analogue (were that the case they would have gone for a rather different demonstration probably). They claimed it was an amazing data mining and parsing system, and they had a cool way to show that off.
Personally, I think it is just amazing and represents a new stage in computer language processing.
Who cares, if Watson's artificial reflexes gave it a few milliseconds' advantage on the buzzer? Who even cares if it'd take it a second longer to read the clue via OCR? So what if Watson would be 5% faster or 10% slower, if conditions were slightly different? Moore's Law makes that level of difference utterly irrelevant - in 18 months time, Watson will be *100% faster* (or even today, if IBM just threw more hardware at it).
Deep Blue vs Kasparov was fascinating at the time, but is uninteresting now for the same reason. A decent desktop PC can play at that level. And comparing human vs machine play styles is also largely pointless, in the same way that comparing birds and jets is pointless.
The important part, by far, is that Watson parsed the questions, linked the clues and searched for statistically relevant answers in a human-like time. The amazing fact is, it can actually do it *at all*. Now that today's systems can do this sort of language parsing and information retrieval in a "reasonable" time, it will be increasingly trivial for tomorrow's. It is now all but inevitable that we will have Watson-like systems available to the public, in numerous fields, in corporations and on the web, in your PCs and even your game consoles, in a brief handful of years.
Why would anyone engrave "Elbereth"?
Then there's the little problem that below 22nm (or 18, or maybe a little lower), all bets are off as far as whether Moore's Law continues to apply. We may very well be approaching the end of the "make computers faster by putting more transistors on a chip" phase of technological evolution.
It's not going to take 30 years for that system to fit in your pocket and cost $20. It's going to take 5 or 10.
Why 5 or 10? Devices as fast and sophisticated as the supercomputers of 2006 or 2001 are not available pocket-sized or for $20 today.
I think a better method than a back-of-the envelope calculation like you've made there is to instead look at the amount of space now occupied by a system as powerful as a room-sized setup in 2001. Unfortunately, I don't have any good room-sized setups from 2001 to use as an example.
Secondly, you are ignoring a few factors. One is that cost isn't an issue. Most of the cost of Watson was likely in software. The software has been made now. Another copy costs absolutely nothing to make. Another is that there are other technologies likely to replace lithography that are likely cheaper and can manage a smaller feature size. I don't think Moore's Law has reached the limit we think it will.
And while you don't ignore the software becoming more efficient, I think that the way you calculate other costs magnifies how much more efficient the software has to become. I would imagine a 10 or even 100-fold increase in the efficiency of the software (though this raises the cost), but not 390000.
Need a Python, C++, Unix, Linux develop
Again, you'd have to expect the software to get many times faster.
I don't think that's an unreasonable assumption though, parsing a natural sentence into computer logic seems like a very hard problem. Languages are full of idiosyncrasies, ambiguities, implied context, fuzzy definitions and subtly changing meaning.
Let's for example take "named after", and that you can properly parse the sentence to find "name" is a transitive verb and "after" is a preposition. That narrows it down to 5 and 12 meanings respectively:
tr.v. named, namÂing, names
1. To give a name to: named the child after both grandparents.
2. To mention, specify, or cite by name: named the primary colors.
3. To call by an epithet: named them all cowards.
4. To nominate for or appoint to a duty, an office, or an honor. See Synonyms at appoint.
5. To specify or fix: We need to name the time for our meeting.
adj. Informal
Well-known by a name: a name performer.
â" prep
1. following in time; in succession to: after dinner ; time after time
2. following; behind: they entered one after another
3. in pursuit or search of: chasing after a thief ; he's only after money
4. concerning: to inquire after his health
5. considering: after what you have done, you shouldn't complain
6. next in excellence or importance to: he ranked Jonson after Shakespeare
7. in imitation of; in the manner of: a statue after classical models
8. in accordance with or in conformity to: a man after her own heart
9. with a name derived from: Mary was named after her grandmother
10. ( US ) past (the hour of): twenty after three
11. after all
a. in spite of everything: it's only a game, after all
b. in spite of expectations, efforts, etc: he won the race after all!
12. after you please go, enter, etc, before me
You can spend lots of power trying to brute force score it into 5*12 possible combinations. Or you could ignore human word boundaries and make "named after" a phrase with a specific meaning. I imagine that by special casing such constructs you could improve performance immensely.
Live today, because you never know what tomorrow brings
The computer received questions through typed entries at the same time as host Alex Trebek read them out loud
I think this qualifies as cheating for an artificial intelligence demo!
No one came back in time during the game to take out Watson. So we can reason that either time travel doesn't exists or that Watson doesn't end up being a problem for humanity's future. Guess the third option is that Watson was successful in eliminating all the humans.