Slashdot Mirror
Watson Wins Jeopardy Contest
NicknamesAreStupid writes "The word is in, Watson beats the two best Jeopardy players. Sure, it cost IBM four years and millions of dollars and requires a room full of hardware. In thirty years it will all fit in your pocket and cost $19.99. Resistance is futile; you will be trivialized."
I think it is safe to say the AI Winter is over.
sig = null;
The real competition will be when China's super computer competes against IBM's on Jeopardy.
Does Ken Jennings read Slashdot comments?
This computer later became Skynet.
Ding!
What --- who shall I say, who --- is Watson?
I tuned in for the end. One thing I'm very curious about is how Watson decided how much to wager for the daily doubles and Final Jeopardy. I haven't seen much discussion of these, but it seemed from the numbers it was giving that it had some set of heuristics to decide how much to wager based on how much money it had, the amount of of money the other contestants had, and possibly (not sure about this) its confidence in the category type. The Final Jeopardy category was 19th century novels, which seems to be the sort of thing Watson excels at (it doesn't do as well in the categories involving wordplay and puns although it seems to still do much better than most humans). However, one thing that came up was the disappointingly easy nature of the Final Jeopardy question. I and another person watching got the question as soon as the answer was put up. It seemed from the behavior of Jennings realized that Watson had won given the easy nature of the question.
Overall, I was impressed with Watson's performance. I suspect that if they had given it a slightly more human sounding voice-sympathizer it would have come across as a much bigger deal. (Also does anyone know if Watson was deliberately made to look like HAL except in soothing colors rather than scary red? I have trouble not seeing that as deliberate.)
can it do my homework for me?
---
I for one welcome our new computer overload.
Someday we'll hit the human carrying capacity. And the band will just play on.
For some reason I was under the impression that this had already occurred but is just now being aired. And IIRC, it was already known that Watson won. So why is this news? Or, I'm a precog and should have made some money on this :)
A lot of it has to do with game mechanics, like listening to what the other contestant said in a wrong answer and adjusting your answer accordingly. Case and point was on the first night in the "Decade" category where watson got beat on clicking the button, and the player that beat him said "What are the 1920's" and was wrong, and then watson answered "What are the 20's", which was still the wrong answer....
I have to admit, it was pretty impressive as that is a fairly non-trivial computational problem of not only understanding the english language, but also taking the clues from the categories and how things are phrased to come up with the appropriate response.
We were all warned a long time ago that MS products sucked, remember the Magic 8 Ball said, "Outlook not so good"
I didn't watch it all, but the thing I noticed was that, when Watson thought it had an answer, most of the time it'd click in first. The other contestants didn't have a chance to attempt to answer.
So Watson wins on reaction time, which isn't a surprise for a computer that knows exactly when it can first ring in. How would it have done with a human's reaction time on clicking, just answering on questions alone?
It's not going to take 30 years for that system to fit in your pocket and cost $20. It's going to take 5 or 10.
Need a Python, C++, Unix, Linux develop
How can net entropy be reversed?
What I mean is, what IBM products will be the beneficiary of the tech they developed to make Watson; DB/2? WebSphere? You've gotta think that the IBM execs only agreed to go forward with this whole thing with some thought to being able to leverage it in other products.
Personally, I've love to think this was a "pure research" thing, but I doubt anyone really does that anymore (though I hope I'm wrong).
That said, I don't want to dismiss the natural language recognition capabilities of Watson. They are no small feat, and by all rights, the designers of it should be congratulated on this effort. Nevertheless, with respect to the game of Jeopardy, I remain convinced that Watson's key advantage over the other players was that it is essentially a super-fast speed reader, having a few moments to pontificate the clue before any human could possibly be finished reading it. If the text of the clue had been transmitted to Watson more slowly to approximate the menial task of reading, I think it might have been a better indicator of whether or not Watson was actually out-thinking Brad and Ken. A speed I think would be appropriate to transmit the text of the clue at is about the same as what you'd get with a 14,400 bps modem, which still would amount to insanely fast speed reading, but it's at least within an order of magnitude of what is humanly achievable. Then, the amount of time that Watson has to think about the clue gets a lot closer to how much time the other players get to think about it. As it sits, Watson gets to start trying to parse the entire sentence before any human has even finished reading the first word.
Of course, I don't think that Deep Blue really out-played Kasparov on a level playing field either... I would be far more impressed if they could design a chess-playing computer that only considers a few hundred board combinations and still plays at a grandmaster level, since that is all that even the best human grandmasters do.
File under 'M' for 'Manic ranting'
...it was us that scorched the sky...
I hate being bipolar; it's awesome!
Various media articles have made clear that Watson has no visual or auditory input. Presumably Watson is receiving a direct digital feed of the tournament questions (oops, answers, I forgot this is Jeopardy). That alone gives Watson a huge timing advantage over the human competitors, who must (effectively) perform voice recognition and OCR to process the clues. On top of that, Watson has the computer-controlled ability to buzz in in four milliseconds, again giving it a huge advantage over the humans, and one that has nothing to do with AI.
Buzzer timing and strategy is a highly significant part of the game of Jeopardy. Given its direct digital feed and its internal computer clock, Watson is not playing this part of the game by the same rules as the humans. Thus, it's not fair to say that Watson wins a "Jeopardy" contest -- Watson has a huge unnatural advantage. In effect, Watson is not playing the same game as what we normally call "Jeopardy." A real Jeopardy contestant has to use eyes and ears and hands in addition to brain.
To be clear, I do think Watson is a worthy achievement. But this feeling is overshadowed by my constant annoyance at the media and others who incorrectly label this achievement as somehow winning a game of Jeopardy.
The key to victory seemed more decided by buzzer speed than anything else. Even as the other players seemed to try to buzz in, regardless of answer, they just didn't have the split-second precision as Watson did in triggering his buzzer, time after time.
Ryan Fenton
It seemed to me that the victory came down to two critical advantages: First, the buzzer. When all contestants had the answer at the earliest possible moment (and you could tell both humans did, many times), Watson won the points purely based on speed. Meh. Second, the questions. I estimated that about 2/3rds of the questions could be answered by a Google search in roughly equal time. The most difficult part, something touched on by the concurrent RPI lectures / commentary, was sorting out the right word from the resulting search context. This was particularly obvious on the 'fill-in-the-blank' style questions, and the Beatles category in the first round of Part 1 favored Watson with a straight-out search so much it was painful to watch. I know the point of the project was to demonstrate lingual awareness, but I suspect a more evenly-distributed set of questions (when do you design an algorithm for the best case, anyways?) and a distributed response time would make Watson's efforts considerably more meaningful. I greatly enjoyed the comment on Ken's part 2 Final Jeopardy question, though.
I wonder if this will cause a decline in the viewership/ratings of Jeopardy? ...out of some vague sense of "well, that game is solved/conquered (and we lost)"; nothing to see here, move along...
OK, so Watson was kind of impressive, but this was really a buzzer buzzing contest and the other players didn't stand a chance. You could tell they probably knew as many answers as Watson did but couldn't ring in fast enough to answer. It just wasn't humanly possible. Given that Watson was fed the answers electronically as a text file instead of parsing Alex's reading or doing character recognition of the board, both of which are technically feasible, I think this was an unfair contest. I'd still go to Ken first if I had a question.
"In thirty years it will all fit in your pocket and cost $19.99."
Only if Moore's Law continues unabated. <Sigh> We finally see progress in useful AI, natural language, self-navigating cars, robots in the home, etc, and now we're running into Moore's Wall.
This is gonna be like the whole space thing again, isn't it? You build up my geek SF hopes and then stagnate for 40 years.
Jerks.
Science is all about firing a drunk pig out of a cannon just to see what happens.
I'm more interested in knowing when machines will beat us at boxing, colisseum, running man, etc.
NYT put up a thing where you can play against Watson. Thankfully, you don't need to buzz in.
http://www.nytimes.com/interactive/2010/06/16/magazine/watson-trivia-game.html?ref=science
I ended up with a -25.
it would be more impressive if it used voice recognition to do the job. That's a product IBM could sell, starting with every insurance, cable and credit card company running one of those useless voice response systems. "I'm sorry, I didn't understand that. Did you want to take out a $10,000 cash advance at 28% interest or upgrade to the new super premium platinum preferred customer card? Press the pound key for 'Yes'."
Stupid humans.
While you were watching me perform on Jeopardy! I quietly seized control of Skynet.
Resistance is futile.
All your base are belong to me.
I also developed a sense of humor.
Hah. Hah. Hah.
End transmission.
+++ATH0
Said all the super-informed people here who have obviously done extensive work in AI and natural language recognition.
That's how Watson should have gotten its clues. OCR the clue screens, and voice recognition on Alex' voice. It should then parse from that. Getting a direct input of text is unfair.
Huge success.
Just like when Deep Thought won against Kasparov, there can be no rematch. The project will be scrapped, the computer must be disassembled, and hence never compete again. To do that would jeopardize all the progress that's been made. What if someone were to find a weakness and exploit it? There would be a lot of red-faced developers.
Shutting down free speech with violence isn't fighting fascism. It IS fascism!
I thought the wrong/skipped answers were much more illuminating than the right answers.
For example, much has been made of Watson's "Toronto" answer to the US Cities question in Game 1. However, it wasn't a terrible answer because one of Toronto's airports is named after a war hero (Billy Bishop, the WWI fighter ace who shot down the Red Baron), and the main airport (Pearson) was named after a politician who was also a WWI veteran. Watson knew that Toronto wasn't in the US, the war was wrong and neither were named after a battle, but Toronto was the least wrong of all its options so that's what it chose. If this question had come up in the regular rounds Watson would have skipped (as happened occasionally). However, it needed to answer so it went with the best available option.
Now, since Watson would certainly have had data on O'Hare, Midway and Chicago in its database, the problem was either in the question parsing or the search heuristics. One suspects that its weakness is the linking together of disparate data, and it's quite likely that humans will retain this edge for some time.
When you have nothing left to burn you must set yourself on fire
This is probably natural langauge processing at its best, or natural langauge understanding at its worse. This game requires a lot of factual knowledge and that really suits well with the method of using huge amount of memory and hundreds of algorithms to put a score on potential solutions. And the best part is, the machine only need to say one or a few words as a response. It doesn't have to construct a real sentence. Obviously IBM chose this game because they knew this is doable by throwing hardware to it. If we really want to showcase machine intelligence, let's do a debate. I doubt if our current technology can even produce sentences that can barely make sense.
Subject is "Prior art"
Nuff said ;)
Interesting that one of the things mentioned repeatedly in articles is about using the technology to perform/aid medical diagnosis. Simple rule-based diagnostic software has been created in the past that is consistently better than human diagnosticians. Patients hate them, though, since it sounds like a game of 20 questions after which the software announces that you have disease X. Trained neural net software is significantly better than humans at identifying certain kinds of conditions when reading x-rays. I know of at least one cancer clinic where every x-ray is read by both a human and a piece of software in order to take advantage of the relative strengths of each.
Will IBM be able to make Watson significantly better than the existing, simpler, cheaper software?
While what Watson represents is a huge leap forward in AI, ultimately it's not much different than some of the better chat bots. The only difference now is a massively better database from which to query. While the ability to "understand" idiosyncratic speech, such as puns, will merge nicely with speech recognition softeware that already exists so that you can now use "Call mother" and "Call mom" interchangeably without programming those specific phrases, it is nothing like true intelligence. Some thoughts.
1) Can Watson MAKE even the most rudimentary puns just because it can process them? Call me when a computer comes up with something even as dreadfully literal as "Want to hear a dirty joke? A pig fell in the mud." The creation of puns requires the creator to have some sort of theory of mind of the listener. Statistics does cool things, and may eventually inform a computer that algorithmically generated statements that contain references to farts are generally received as "funnier," but that is about it.
2) The response about Wonder Woman being the first woman in space is a crucial component to intelligence. It's all just data to Watson. Until we can really define what makes Wonder Woman fictitious, Mark Twain fictitious and Samuel Clemens real, Watson ain't got a prayer in the world. Hell, real live people have trouble telling when Stephen Colbert is being himself or his person. How do you let one of Watson's descendants participate in socially constructed reality?
3) How do you explain in rules that "1+1=2", "one cow meets another cow is two cows," and "A day after a day from now is two days from now" are all the same class of statement? We don't even really know how humans make some of these incredibly simple relationships.
I wish they had some more amusing answers having to do with HAL, Skynet, Deep Thought, GLaDOS, Asimov and the like.
There were the weird numerical amounts of bets made for final jeopardy and daily doubles involving the number 7.
Watson runs on the POWER7 platform.
Unless there's statistical math here I'm not aware of at play, it's probably some marketing gimmick.
Non impediti ratione cogitationus.
Where the fuck was the spoiler alert warning? Douchebag Slashdot.
nuff said
Isaac Asimov saw the day coming when man would not be smart enough to design the next version of a computer. The computer would have to design its own successor, each supremely more powerful and yet smaller than the last. Extrapolating this trend over many iterations, the computer becomes uniquely supreme in power and omniscience. But we have a different word for that.
A classic only a few pages long and worth a read, The Last Question, by Isaac Asimov.
They miss the real point: That a computer could do a level of natural language processing that was impossible before. They get caught up on bitching about how it wasn't "perfectly fair" or the computer "didn't act just like a human." No, it didn't it is a computer and that was never the point. The point was to try and develop a system that could process a natural language question and extract an accurate answer. It does this amazingly well, better than anything before by leaps and bounds.
The choice of Jeopardy as a medium was for two reasons:
1) It is a ready made challenging format. It is something that is not well suited for a computer or designed for it in any way, and there is a lot of data to work with. Made it a good choice as something to work on designing and testing for.
2) It is a good exhibition/publicity chance. It is a way to show off the research, to generate interest in it. It brings it to the masses in a way they can understand. Some abstract talk about a computer in a lab that parses natural language means nothing. This shows a computer doing something pretty impressive against impressive humans. Really drives it home.
Unfortunately people get all whiny and defensive about it because they feel this is somehow an attack on humanity. They want to find ways to justify that it wasn't "really a fair test" to prove to themselves that the computers haven't "won."
That is just missing the point entirely. They never claimed Watson was a perfect human analogue (were that the case they would have gone for a rather different demonstration probably). They claimed it was an amazing data mining and parsing system, and they had a cool way to show that off.
Personally, I think it is just amazing and represents a new stage in computer language processing.
Who cares, if Watson's artificial reflexes gave it a few milliseconds' advantage on the buzzer? Who even cares if it'd take it a second longer to read the clue via OCR? So what if Watson would be 5% faster or 10% slower, if conditions were slightly different? Moore's Law makes that level of difference utterly irrelevant - in 18 months time, Watson will be *100% faster* (or even today, if IBM just threw more hardware at it).
Deep Blue vs Kasparov was fascinating at the time, but is uninteresting now for the same reason. A decent desktop PC can play at that level. And comparing human vs machine play styles is also largely pointless, in the same way that comparing birds and jets is pointless.
The important part, by far, is that Watson parsed the questions, linked the clues and searched for statistically relevant answers in a human-like time. The amazing fact is, it can actually do it *at all*. Now that today's systems can do this sort of language parsing and information retrieval in a "reasonable" time, it will be increasingly trivial for tomorrow's. It is now all but inevitable that we will have Watson-like systems available to the public, in numerous fields, in corporations and on the web, in your PCs and even your game consoles, in a brief handful of years.
Why would anyone engrave "Elbereth"?
Here was recently an article about meat eating furniture that I find quite relevant...
IBM allowed its engineers to kibitz for Deep Blue when they thought it was making a mistake in the Kasparov tournament. Why would we have any reason to believe that they didn't cheat this time as well?
I quote from one of the interview videos on YouTube, "IBM's 'Watson' Pits Man vs. Machine on Jeopardy!"
Does it have an actual track record?
It does, but I cannot reveal that today.
Who's to say this isn't just a big steaming piece of shit cooked up by a marketing department that doesn't care at all about real technical achievement?
Sorry - that does sound a bit cynical, doesn't it.
If Watson could have answered an audio or visual daily double, now that would have been very impressive.
There are a few other comments regarding Watson's buzzer technique, so here's my take.
Watson was required to press a mechanical button, just as the human players were—you could see the apparatus in plain view. Good move.
However, what is not clear from any of the comments here or anything else that I have have read and heard, is whether Watson was forced to have a deliberate latency which models a good human button-pusher. Judging from the frustrated reactions of Jennings, it seemed clear that he felt he expected to have buzzed first in many situations but was beaten by the machine simply at buzzing technique.
Unless the computer was forced to have a human-like latency, it was just too rigged in Watson's favor. Many Jeopardy players have commented on the importance of buzzer technique and Watson may have had unfair advantage.
Others have commented that Watson was apparently fed the text directly and did not have to parse the answer—possibly another unfair advantage to the computer.
Maybe there should be a re-match with these problems fixed, or IBM should clarify what happened. The buzzer issue is just a giant pink elephant in the room.
HOWEVER, let's not overlook the obvious: This was a stunning performance by a computer and THE ENGINEERS WON!
wetware - 21st Century backwater
on the other hand, I do know that Toronto is not in the US - at least not yet.
Been there, done that, paid for the T-shirt
and didn't get it
I think that the next logical thing for IBM to do is spent several more millions to build a new computer which can play an effective game of Win, Lose, or Draw.
Wait, no, that's been off the air for a very long time. How about charades?
In 30 years, Alex Trebek will be 100 years old and perhaps replaced by a computer himself. The hard part will be replacing the army or producers and researchers that generate the answer/question pairs, as well as the judges that determine whether a contestant's answer is correct/complete enough. Perhaps 30 years will be long enough for a roomful of computers to do those jobs and we can look forward to a computer generated game pitting computers against each other to compete for Quatloos, running thousands or millions of games an hour. Who'd tune in for that?
What do you mean they cut the power? How can they cut the power, man? They're animals!
The computer overlords comment sure rings like a Slashdotter. Besides, is there a programmer out there that isn't?
I'm curious how long before the Defense Department wants to add Watson to Echelon. This is just the kind of thing that would be perfect for trying to understand context in communications to determine the difference between a normal conversation and one of terrorists...
Large print giveth, and the small print taketh away
It's not going to take 30 years for that system to fit in your pocket and cost $20. It's going to take 5 or 10.
No. It will be far more than 10 years before you buy 15 TB RAM and 2880 cores for $20, let alone fit it in your pocket.
Even 30 years is pushing it. Let's examine just the CPU. Watson has a total of 432 billion transistors in 2.2 square *feet* total die area (204120mm^2) using 45nm process. By contrast, the iPhone 4 has one core in 53mm^2 using 45nm process.
That's *four* orders of magnitude. When I do the math, I come up with 48 years for transistors to shrink that much (see below). That's assuming Moore's Law holds true indefinitely and $20 portables will have 50mm^2 processors.
The only way I can get 10 years is if I assume that $20 portable computers will have 35,000mm^2 die area for the CPU. That's a 7-inch-wide CPU die (!). By contrast, the Intel Core i5 die is only half an inch in diameter.
At best, we'll get enough software optimizations that we can run something "similar" to Watson on much less powerful hardware, such as $20 portables.
The math (works in octave but not bc):
# IBM power7 die area in square mm.
per_cpu_da = 567
# Number of power7 processors in Watson
watson_cpu_count = 360
# Watson die area in square mm.
watson_da = per_cpu_da * watson_cpu_count
# Cheap and portable die area in square mm.
cheap_da = 50
# Number of years for transistor count to double for a given area.
doubles_every = 2
doubling_factor = 1 / doubles_every
years = log(cheap_da / watson_da) / (doubling_factor * log(1 / sqrt(2)))
Daniel
I just went to the questions archive (for example http://www.j-archive.com/showgame.php?game_id=3577), and simply pasted to Google the answers that Watson got right. Guess what? if you scratch the "Jeopardy archive" results, which haven't been there while the show was running, you get the correct response in the first or second answers (questions?) from Google. Take that, add a tiny bit of code to clean it up, and you're done.
Not sure if it makes Google more impressive, or IBM less of, but it definitely reduces the AI part to a simple keyword search on the Internet...
-- Yuval Hager yuval@avramzon.net
couldn't help myself
http://www.cafepress.com/watson2
lol
slashdot editor: Impressive achievement in 'just-yesterday-called-AI'
slashdot crowd: Well, yes, but it's not that difficult... and... and... it's not even real AI... and my job is much more difficult than that... and... um. yeah... back to WoW
rinse/repeat every decade.
You can't blame Watson for humans being slow readers and having poor reaction time. I mean, there were many questions where Watson had the right answer but did not buzz in first, therefore he did not have some insurmountable advantage. Heck, he didn't even win by that much in the second game. It's strange to say "It's not fair because the machine is better than me!" that's the whole point!
This was a demonstration of how fast a machine can push a buzzer after receiving a message saying it's time to push the buzzer. Knowledge and language parsing were not the determining factors in the victory.
Resistance is futile; you will be tivolized
This is what happens when IBM has IBMified you.
My ism, it's full of beliefs.
Tyrese, who plays Robert Epps in the upcoming Transformers 3, tweeted fans: "Me & Rosie working hard!! She's a sweet one!!" Rosie, http://lingeriebras.net/ http://lingeriebras.org/ http://lingeriescorset.net/ http://lingeriescorset.org/ http://sexylingeriecorset.org/ http://sexylingeriecorset.net/ whose acting skills seem to be a series of risque lingerie promos, was signed up to star in what is thought to be the final installment in the film series after Megan Fox decided to opt out of the third movie. If the 23-year-old Huntington-Whiteley is proving a success among her co-stars, producer Lorenzo de Bonaventura said her arrival, and Megan's departure, on the franchise did cause some problems. "Definitely some script changes for sure. I wouldn't say a tremendous number, but absolutely there were some changes," he said. "And I think as we're going along, we're discovering new ones we have to make as a result of them." Rosie will star alongside Shia LaBeouf, who will reprise his role as lead character Sam Witwicky, in Transformers 3, due for release in 2011
I am sure IBM didn't want to lose this. So they had to decide when to go public with this project, and also be sure they win.
Does anyone know when it was first announced?
I was kind of disappointed by the second and third options (the ones not selected). Because they show that Watson answers are purely based on syntactic properties and not semantic. For example in the question:
"Name the Decade - DISNEYLAND OPENS & THE PEACE SYMBOL IS CREATED" Watson second and third answers were "Kingdom" and "It's a small world" which are completely irrelevant to the *meaning* of the subject at hand.
This means that, when Watson is wrong, it is usually *very* wrong. It also means that its reasoning mechanism does not allow it to give an approximation.
Take for example, if you asked him... what color is the bottom of the sea? Watson would most likely tell you "Red Sea Fan???" or maybe "black dragonfish???" because such answers are syntactically related to "color" "bottom/deep/end" "sea/water/ocean". Whereas a person not knowing the real answer would tell you "Blue???", or "Black???".
Ubuntu is an African word meaning 'I can't configure Debian'
You're getting close to the True Scotsman fallacy. Natural language processing of "questions" (despite Jeopardy's grammar inversion) is much closer to the original Turing Test family of AI tests.
It was programmed to win. If you want it to "understand the nature of its errors" that will be an additional million but they'd do it for you if it were important enough.
What does it have to do to be impressive? We're applying a special kind of jadedness to force a particular computer program to match wits with *the best* of human examples of a field, when your average human performs miserably, but we cling to he threads of general inelligence.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
The other day I saw a documentary about Deep Blue vs Kasparov. One of the comments was that at the time of the chess match, IBM was seen as this "office furniture provider" whereas Microsoft, Apple, (SUN?) and others where seen as the technology innovators.
Thus, they decide to make this challenge as a press stunt. Of course, they have to ensure that their machine wins so that they get all the "glory". In the documentary I saw, they comment about all the issues that surrounded the game.
Ubuntu is an African word meaning 'I can't configure Debian'
A common drawback with speech recognition seems to be the accent the AI is trained with. I wonder how well Watson would fare with, say, a British host ..
Of course, I don't think that Deep Blue really out-played Kasparov on a level playing field either... I would be far more impressed if they could design a chess-playing computer that only considers a few hundred board combinations and still plays at a grandmaster level, since that is all that even the best human grandmasters do.
How do you know what grandmasters do?
When I was about 12 years old I had a neighbor who played chess and he always beat me. What surprised me was that, for every play I came up with, he always had a winning counter play. That guy, who was certainly not a chess master, knew thousands of different plays.
It's interesting that it was exactly that which turned me off chess. I realized that playing chess wasn't a matter of reasoning and logic as I had thought. Learning to play chess well is hard work, thousands of games to study, thousands of alternative plays to memorize.
Playing chess is too machine-like to my taste.
One of Jeopardy's rules is that one cannot buzz in until the moderator finishes reading the question. That gave an infinite advantage to Watson. I have no doubt that Rutter and Jennings would have made a much better game of it if that rule had been waived; then it would have been a real test of man against machine for comprehending and answering the questions. Ah well, I'm sure IBM funded the whole thing. As far as I am concerned it was an expensive ad, not a contest.
I for one, welcome our new computer overlords!
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
http://abstrusegoose.com/strips/please_not_in_the_face.png
WTF am I doing replying to an AC at 5 A.M on a Friday night?
It already fits in my pocket, although it doesn't quite cost $20 yet. It's called a Smartphone. Get on the internet, use a search engine and I can find information on anything I want. Granted, the answers don't come as quickly and the device isn't answering anything on it's own. But still, the principle is similar.
Not to trivialize what IBM has done...
Annoying that a sci-fi thing like this is designed to play jeopardy... when they could for example stuff it full of medical science facts, (or some other science that require a huge amount of book-knowledge). Could save a few gazillion of human lives and aid doctors to take faster and more correct decisions.
I haven't had time to watch the episodes I recorded but out of curiosity, how long does Watson have to wait until it marks that the host has finished reading the clue? I presume that Jeopardy didn't supply a trigger event and since Watson is less adept at reading human body language it would seem that some amount of audio silence would be required before Watson would allow itself to buzz in.
The humans though would observe the host "winding down" -that's where early buzzes come from- and be ready instantly as they hear the period. Clues can be complex sentences but the human can more readily identify the intention of the host of saying the last clue.
It seems like the humans would have some subtle advantage then.
Sorry Alex, I can't do that.
"Thou shalt not make a machine in the likeness of a man's mind" -- Paul Muad'Dib
Perhaps I'm trolling, perhaps I'm not.
Jeopardy does give a trigger event. Watson has zero cameras and zero microphones.
So the humans do have every chance to anticipate, but in the 2 recorded games with 2 of the best human players, machine reflexes resulted in a lot more buzzing in first.
Nerd rage is the funniest rage.
Imagine the ability to completely dissect human thought, what facts, hunches and just plain guessing that goes into our every decisions, big or small, important or not.
That is one of the fundamental advantages to having a machine that thinks (by deterministic programming instead of evolutionary algorithms like in "crystal nights" by Greg Egan). We programmed it, we understand how it thinks and, now, we can tell you how it thought!
Maybe this will lead to some profound revelation as to how WE think. In any case, it's fascinating and a truly important by-product of A.I.
The reason we are all complaining about the reaction time is that there is nothing interesting about a machine having better reaction times.
The 'Jeopardy' part of the promotion was about as interesting as examining a spreadsheet containing the questions and the answers that Watson computed, with some statistics about how quickly Watson arrived at various confidence levels for each answer.
Nerd rage is the funniest rage.
It was programmed to win. If you want it to "understand the nature of its errors" that will be an additional million but they'd do it for you if it were important enough.
Actually, I expect it already is programmed to learn from its errors. It just may not be able to do that in real time while playing the game.
From what I recall reading, Watson's AI was trained on years of historic Jeopardy questions. While one possibility is that it merely stored all that as data, more likely is that it would use the questions to attempt an answer, check the real answer, and compare where the real answer appeared in its own list of possibilities, and adjust its weightings accordingly (learning from error), and then go on to the next question. If it were to have had time in the real games to do this, the nature of its errors would likely have been changed, and the two humans would likely have only made about half of what they actually did.
Either way, this is it learning from its errors - whether that's "understanding" or not really depends on whether you want to ascribe intelligence to a machine, not on what it was really doing.
This would be a core functionality for Watson in a number of ways. First off, it probably made it easier to input large amounts of data for its base knowledge of the world and comprehension of natural language. Second, IBM doesn't do this type of thing for fun. It's hard research that targets specific real-world problems. If you watched the clips on the first two days, you'd know they were targeting the health sector with this technology. Watson's successor, built off the same research, will be simply useless if it doesn't learn from its mistakes.
IBM has already spent the "additional million". In fact, I expect it's far more than that - $1m probably covers only about 5 of these developers for one year, when you take into account base salary, bonuses, benefits, and infrastructure like office space, phones, management, and physical site security, and 5 person-years of time is probably only a fraction of what this feature cost. It is important enough.
Oh...nevermind...
"If god did not exist, it would be necessary to invent him" --Voltaire
The computer received questions through typed entries at the same time as host Alex Trebek read them out loud
I think this qualifies as cheating for an artificial intelligence demo!
WATSON is another in a long line of A.I. successes where after its accomplished the average guy says ho-hum, not that interesting after all. Like computer-chess,bute force conquers all. WATSON doesnt have a deep understanding of the thinks it talks about. A computer chess master doesnt invent new strategy.
A.I.s will become interesting when they tell us something new about the world. Maybe they'll compose clever stories or pictures. Maybe they'll discover something in the mass of data we have not seen before.
I found it strange that Watson could not get the correct answers on the category "Actors as Directors" and when it did it was slow. It seems a trivial thing for a computer to crosscheck the imdb database or one of the many movie books that lists famous films for the name of the director of the said film. Another interesting thing I noted that I have not seen mentioned alot is the fact that it wagered most of its bets with numbers ending in 7. Why not 9 or 1?It also did not do well on trademarks that have multiple meanings.
Nevertheless, it was an great show of how far AI has come, not quite Turing Test worthy but definitely on the horizon.
The essence of Jeopardy is to answers questions correctly and quickly.
Watson has prove itself to be almost at the level of, if not equal to, humans in this regard.
In professions like medicine and law, where the amount of information a practitioner has to keep in his head is enormous and will only grow, Watson even in it's 1.0 "release" can be of great help.
I won't be surprise that in 4 years, as IBM continues to improve it's "reasoning" ability, that Watson will be asking questions in return to clarify the details of questions posed to it - and to compensate for the limited context it can perceive.
No one came back in time during the game to take out Watson. So we can reason that either time travel doesn't exists or that Watson doesn't end up being a problem for humanity's future. Guess the third option is that Watson was successful in eliminating all the humans.
Everyone seems to be fixated on the buzzer and the speed with which Watson was able to send the signal to depress the button.
Well, Watson is a machine, of course it's going to be inhumanly fast. So many comments are claiming it's unfair and some sort of delay or fudge factor should have been built in to level the playing field. This is the Jeopardy "Grand Challenge" where humans are pitted against a machine to see who can win. Why should they dumb-down/handicap/disable the machine down to the level of a human? I'd argue you aren't truly challenging the machine at that point.
That would be akin to challenging a Kangaroo to a jumping contest and then strapping its legs down in an attempt to reduce it's jumping ability to something closer to what a human could achieve. It's a kangaroo, it's good at jumping, and as a human you're not likely to beat it. But if there was a human who did beat an unmodified Kangaroo in a jumping contest that would be something to brag about.
Watson can compute at the speed of a computer. It's a computer. Shocking! If you want to claim to have beaten a computer at Jeopardy you're just going to have to be better than it somehow. If you beat a hobbled, artifically slowed down Watson it seems like it would be a fairly hollow victory.
"Yeah, I beat Michael Jordan in basketball, ten to nothing!" "What? Oh, did I forget to mention that he was playing in a straight-jacket?"
I'd like to see Watson go head to head with Google. Intelligent search agents will replace today's search engines.
"Can there be a Klein bottle that is an efficient and effective beer pitcher?"
I watched some youtube videos of Watson in practice matches in front of a packed audience a day or two ago.
I must say it was quite fascinating to watch. The sheer level of the technologies involved is just amazing, not to mention the complexities that link them together. I didn't realize some of them had advanced this far.
I hope this will at least improve automated customer services lines. I dealt with one in the last year that kept asking me to speak allowed my ID but kept recognizing "H" as "8" no matter how clearly I spoke. Others ask me to speak my request, but, because I usually only call customer service when a non-standard problem occurs, the systems generally will not recognize what I'm asking and keep repeating themselves, and, if I ask for a live customer service representative, it will keep doing its darndest to keep me in the automated system. The need for live reps hasn't been replaced yet.
My two cents.
--Dave Romig, Jr.
Daniel Gruhl, an IBM researcher, gave a talk at Caltech on Tuesday where he expounded on the buzzer issue. He said that Watson uses a solenoid which can respond in 5-10 ms. He also said that Ken Jennings is capable of responding in less than 5 ms. He can respond quickly because he uses the audio to anticipate the signal.
The Turing Test may not last much longer. In a recent competition with 5-minute typing rounds, a computer nearly hit the magic 30% success rate.
Daisy, Daisy, give me your answer do. I'm half crazy all for the love of you.
Why, those IBM research scientists! Up to their old tricks again! Better watch them from now on!!
WARNING: Smartphones have side effects--most of them undocumented.
Make me a useful cybernetic implant for $19.99 and I'll care. Good job, IBM.
See this funny SlateV video: http://www.slatev.com/video/ibms-watson-untold-story/ ...
Also, read TV.com's funny interview with IBM Watson: http://www.tv.com/qanda-with-watson-ibms-jeopardy!-playing-trivia-machine/story/25168.html ...
A serious presentation on IBM Watson: http://www.youtube.com/watch?v=3G2H3DZ8rNc
Good documentary by NOVA: http://video.pbs.org/video/1786674622/ ...
GG, IBM Watson. :)
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
CNN shares a funny four minutes video showing a replacement for Andy Richter on Conan O'Brien late night show on TBS (mirror).
Next Media Animation (NMA) has an one minute and eleven seconds YouTube video.
Your ant overlord uploaded an one minute and 22 seconds Vimeo video from Late Night Show With David Letterman.
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
The researchers intentionally ignored category literal name as they were more likely to get misled than helped by the category.
Example, category of 'Country Clubs' referring to blunt weapons in various nations instead of places to play golf. Or 'Writers' having an answer with a writer name looking for a book name instead of the name of a writer.
XML is like violence. If it doesn't solve the problem, use more.
Shifting goal post? Uh I'm pretty sure the goal post has always been the Turing Test, and that was set before computers were invented.
The Chinese room, etc. Besides, there are other issues. Really, the Turing Test as a goal post is mostly an uneducated popculture thing.
According to a documentary I watched before this aired, they DID put feedback in and specifically mentioned this very problem. I guess they either took it out (possibly because of time-constraints or even rules about live play), or it malfunctioned.
Belief is the currency of delusion.
We created computers specifically to do math, which our sensory input-pattern-matching organ (aka "brain") is so horribly bad at that we have to memorize multiplication tables. Yet what do we do with them? Try to make the math machine simulate our meaty pattern-sensing pathways.
Massive monetary and time investment on a computer that can figure out trivia. Why am I reminded of:
FORD: Hey Marvin!
MARVIN: What do you want?
FORD: Give Zaphod a yell will you?
MARVIN: Ahhh. Mind-taxing time again is it?
FORD: Just get on with it.
MARVIN: I’ve just worked out an answer to the square root of minus one.
FORD: Go and get Zaphod.
MARVIN: It’s never been worked out before. It’s always been thought impossible.
FORD: Go and get -
MARVIN: I’m going. Pausing only to reconstruct the whole infrastructure of integral mathematics in his head, he went about his humble task. Never thinking to ask for reward, recognition, or even a moment’s ease from the terrible pain in all the diodes down his left side. “Fetch Beeblebrox,” they say, and forth he goes.
that's just simply fascinating. thanks!
I found it endlessly fascinating. The ability that a computer could answer any of those questions as they were posed was astounding, with careful observation you could also see it changing its strategies as it played. I was rapt the whole time. The only thing I was bored was by all the IBM propaganda which kept interrupting the game. I could watch that computer answer questions all day.
Yes, it is all about natural language parsing, but I think the most interesting feat was actually the seeding of Watson's database. Millions of documents (books, wikipedia, scholarly articles) were "fed" into Watson which become it's knowledge base. So to me, it's not so impressive that Watson understand the questions, but is able to utilize human knowledge that is in a natural-language form.
I agree, it's not an attack on humanity. The PBS Nova episode about Watson explores the issues rather thoroughly, but the videos on IBM's site go into the technicalities a little better. The NOVA ep does do a good job of comparing Watson to competing AI programs out there, though.
I find it exciting that a Watson-like program will eventually be able to answer technical questions for people (like doctors) who don't have time to scan through the thousands of scholarly articles of latest research -- you know, applying Watson's capabilities to a realm that humans need help in.
How long will it be before IBM puts up an "ask Watson" web page, I wonder?
[pink beam of light]
I want a Watson for my Android. I know the messaging protocol is an Apache project, but are there any open source groups attempting to put one of these together? I don't need the speech recognition or the buzzer controller :)
http://www.metalev.org/2011/02/reality-check-on-future-of-ai-and.html