Slashdot Mirror
Watch IBM's Watson On Jeopardy Tonight
JohnMurtari notes that the media hype machines are massively promoting tonight's battle between Jeopardy champions and a super computer. Yes it's a PR stunt. But I imagine the actual research probably had a lot of interesting problems to address. Anyway, you can learn about IBM Watson if you're interested. I'm sure the most amusing bits will be on YouTube about 30 seconds after air time.
JohnMurtari notes that the media hype machines are massively promoting tonight's battle between Jeopardy champions and a super computer.
I'm so glad we're above that.
Seriously, if this thing doesn't accidentally observe the Higgs Boson while seeking for a question to an answer, I'm going to be disappointed.
My work here is dung.
I mean its going to be one of the first times that a robot with speech recognition will be live and responding against people in real time on broadcast TV. I think you all have been living in your movie plots too much to realize how big of a moment this actually is.
It's a little more interesting than a computer beating a human at chess, which is completely algorithmic. However, at its heart, this is simply an exercise in in data storage, lookup, and statistical probabilities in determining a likely answer. It does not involve any artificial intelligence or machine intelligence at all. From a purely technological standpoint, it's quite impressive what IBM has been able to do. It'll be even more impressive in 10 years when the same type of power is in my phone.
http://video.pbs.org/video/1786674622/ for Americans. :)
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
There was an interesting episode of Nova called "Smartest Machine on Earth" that was pretty interesting. It talked a lot about the challenges they faced, how they addressed them, what adjustments they made along the way, etc. I don't see the episode listed on the schedule for replay any time soon, but you can watch it on the website
http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html
What a wishy-washy summary. It's not like you have anything better to do tonight than watch Jeopardy, is it?
Speaking of which, it seems like *I* was supposed to buy or do something tonight... now what was it...
Adidas To Bring Back Sneakernet
Yes, they are hyping it for more than it's worth, but there's one thing here worth hyping: the fact that IBM still runs a state-of-the-art computer science research program. Just about every other corporation has gutted whatever research institute they had and concentrated their R&D funds on directly marketable products. IBM still runs the Watson research center that develops ideas from basic computer science down to products, even if it takes more than a decade to do it and the results are not certain. That is certainly worth respect.
While obviously it does utilize data storage, lookup, and statistical probabilities to help find and choose an answer, it also significantly relies on machine learning (a large branch of AI) to understand the questions and choose the answers.
Popisms.com - Connecting pop culture
I love the schizophrenic nature of the post. Every other sentence is a reason you shouldn't watch Jeopardy. But the poster clearly wants to watch it himself.
Basically if you like Jeopardy, watch it. It will be good. Ken Jennings will be back. It' not just going to be a computer, it's going to be two really good people playing.
Even if you don't particularly like Jeopardy, you might still like to watch it, if you want to see an unusual spectacle, or are interested in something vaguely related to artificial intelligence. And if you don't want to watch it, then don't, it's just entertainment after all. But don't whine about it, that's even worse.
My guess: the computer wins, not because of its massive database, but because it can push the button really really fast on the questions it does know.
"First they came for the slanderers and i said nothing."
Having seen a video of it in action, I'm very impressed.
http://www.engadget.com/2011/01/13/ibms-watson-supercomputer-destroys-all-humans-in-jeopardy-pract/
They actually showed the beginning of the filming of tonight's show.
The first time the Jeopardy producers saw Watson in action, the performance was erratic. But a fairly simple change made a respectable improvement. That was to use the responses from the other players and the host. This feedback reduces ambiguity for later answers. The improvement was enough to make the producers use Watson then.
True, but it's also an understanding of human language. If you watch the PBS NOVA episode on it, it can be quite hard. Like the category called "Days in months", where you're given two days of a month and have to answer in the month. How does a computer figure that out? (In Watson's case, it didn't until it saw the correct answers and figured out that it needed to be months).
Or a category like "before and after"?
Pure trivia questions - yes it's a simple database lookup (and Watson basically kills at it). But Jeopardy isn't just a nerd trivia game, it's all about subtleties of language - double meanings, puns, wordplay and other elements that make it extremely hard.
It's basically a step towards understanding natural language, with all the issues and subtleties that we put in - emotions, sarcasm, etc.
Or, in Feb 14-16, 2011, Skynet will show off its ability to understand human language.
Except that (1) Watson doesn't do any online access and (2) Watson doesn't do anything that resembles search as we know it today.
Sorry, try again later.
I want my Cowboyneal
>> While obviously it does utilize data storage, lookup, and statistical probabilities to help find
>> and choose an answer, it also significantly relies on machine learning [wikipedia.org] (a large branch of AI)
>> to understand the questions and choose the answers.
If this machine chooses the Answers it will lose at Jeopardy, It needs to determine the Questions . . .
We are Dead Stars looking back Up at the Sky
The ability to follow a rigid set of very specific instructions that someone else has established is not intelligence.
"But this one goes to 11!"
The show that will air tonight has been filmed weeks ago, for no good reason that I can think of. They would definitely stir up more attention and attract more viewers if the show was broadcast live. This is like being told months after the fact that Deep Blue did in fact beat Kasparov, and the moves were 1. Kp3, ... That would have been completely lame.
What IBM hopes for is for Watson to win, but not win by much, so that people aren't put off by its brutality. And this taping of the show weeks ahead of the airing just invites speculation that the game was rigged to produce exactly this result. After investing so many resources in Watson, it's pretty dumb of IBM to not do this last thing right - which would have greatly raised the interest without any additional cost. One imagines that they did this because of their lack of confidence in Watson's performance. And that makes them look far less badass than they otherwise would.
Congratulations. I estimate that 80% of the human race fails your particular Turing analogue. It actually explains a lot, really.
Welcome to the Panopticon. Used to be a prison, now it's your home.
A machine that interprets human language and produces spoken answers is showing some of the most important parts of human intelligence, and it does so artificially. So this is very much artificial intelligence.
You have got to be kidding right? Any reading about Watson will quickly reveal that the subtleties of language (specifically English metaphors, similes, and irony) as well as the ingrained underpinnings of Western culture have been its two biggest obstacles since day one, and that's precisely why IBM chose Jeopardy as their next grand AI challenge.
Having dozens of Chinese colleagues, I can assure you that the hidden meanings and references we bury in the English language are completely lost to them even though they know the English words. Do you really think I will understand their jokes, movies, books, etc, just because I flipped a switch and heard a word-for-word translation from Chinese to English? (Even that situation is absurd, actually, because Chinese-English translators have to see a sentence and translate holistically, where many colloquialisms and phrases lose their meaning in translation)
Here's a quick example:
(Exact translation from Chinese to English) "Watchful caution! Avatar come!"
If you thought that meant a blue creature or a virtual representation of a person was coming for you, you'd be wrong. Chinese gamers call a bombing helicopter/hovercraft an "Avatar," because they first saw one in the movie Avatar. If Watson got that right, he'd have to know a very subtle fact about Chinese culture, and Jeopardy is replete with these cultural landmines.
If IBM can prove a machine understands the deep underpinnings of our language AND culture by correctly answering very apocryphal questions better than a Jeopardy champion, then the company will have effectively demonstrated the world's best language and cultural interpreter to bridge the gap between man and machine.
AccountKiller
I will watch it to see if Alex can chit-chat with it with regular questions, like "Where are you from?", "What's your job?", "Married? Any kids?", "Do you like... human?"
Jeopardy! is the game where an answer is given and the contestant must supply a question...
Once their AI is sufficiently good at it, they're obviously planning to give it the answer 42.
I for one hope that Watson's text-to-speech engine fails miserably and he starts mispronouncing category names like "The Pen Is Mightier".
It'll be even more impressive in 10 years when the same type of power is in my phone.
Your phone will not have terabytes of RAM in ten years. I promise.
The actual phone may not have 10 terabytes of RAM, but I bet it will be able to instantly access computers that do.
In fact, with Google, Bing, and other search engines, one could make a case that your phone already does contain at least that amount of RAM.
We are also seeing this with services like OnLive. Once we get to a point with mobile phones that have very high bandwidth, very low latency and access to dedicated, powerfull remote computers, everything is going to go to dumb terminal renderers.
The only problem I have with the way Watson is being used on Jeopardy is that gets a minor head start on being able to answer the question. Although the text of the question is actually revealed to everybody, including Watson, simultaneously, humans have to take a second (or two) to actually read or hear the clue to even know what it is actually saying, whereas it's my understanding that Watson receives the clue electronically, and thus knows all of the text in the clue in mere milliseconds, before the human contestants have even finished reading/hearing it.
It has been my supposition since I first heard of this that Watson would tend to fare poorer against human competitors when the clues are very terse, because then Watson doesn't get as much extra time over the human competitors to parse the clue and search its database for possible answers.
I look forward to finding out if I was right.
File under 'M' for 'Manic ranting'
(1) It does a lot of search. For example, for answering the movie questions it use the IMDB, (from its own storage and probably pre-digested). The missing "hook it up to the internet" doesn't sound like a big obstacle - unless its algorithms/storage needs aren't linear with its source material.
(2) The fact that it doesn't "resemble search as we know it today" is what would make it useful. It adds a cognitive bit to search that is currently missing. Today, its often hard to find information on the internet if your exact question hasn't been asked before. Its also hard to find answers to questions that are context sensitive. If your search engine could even be a little bit helpful with these it would save a lot of time.
I have no idea what IBM's plans are and I'm sure there are other applications for Watson but they would be nuts to not think about how it could be used for internet search.
There's not consensus on what artificial intelligence is because there's not a consensus on a precise definition of intelligence. One thing that designing computers and software to do such things as play tic-tac-toe and chess has shown us is that the ability to do such complex logical tasks is very different from human intelligence. It also shows how little we understand of the human brain and mind.
"This man was the son of a president, a president himself, and invaded the same country as his father."
I'm feeling lucky...!
http://en.wikipedia.org/wiki/George_W._Bush
"What would be a good definition artificial intelligence that wouldn't be subject to goalpost moving?"
When the AI starts chiming in on where the goalpoast is, and giving us suggestions, I think then we can hang our hats and go home as a dying species.
You have to understand the semantic meaning of an error in computing. An error is something that is generated by an implementation upon the failure of a test at some level of the system -- it indicates the system has entered a state where further inputs will no longer map to the "desired" outputs. The issue is in how we define "desired," and we find that this is always defined semantically by the humans designing the system, a priori. A computer cannot divide a number by zero, or dereference a null pointer, because we say so, because we apply that abstract truth to the system. We do so because hardware and software form an entity that requires internal consistency to respond to inputs, and when that internal consistency is lost the system no longer is useable.
Humans make errors all the time, it's just that we do not generally halt when we make them. We have other ways of reconciling errors, things we call "rationalization" or "denial" or "learning." Human beings have very limited a priori desired outputs and exception states, and none of them apply to symbolic reasoning -- a coma might be an example of an exception state, and it's brought about by "recoverable device failures." The human brain and cognitive system is also much more finely engineered and rigorous than a computer system, inputs and outputs are always "sane," the states of the system, such as they are, are highly distributed in time and between functional units, and on most levels of operation the global system cannot lose internal consistency in a way that jeopardizes operation.
Well, the Nova ScienceNow that directly preceded the Watson episode was all about animal cognition (probably not coincidentally), and they had several rather unsettling demonstrations of a dog that could remember dozens of toys by name, and collect novel toys given nothing but the novel toy's name; a parrot that could count to eight and construct declarative phrases of nouns and modifiers; and dolphins with functional vocabularies that were provably communicating with each other through their squeaks to collaborate on a trick that they invented themselves.
Most creature's brains are capable of abstraction to a degree, but the physical attributes that are associated with humanity, like the opposable thumb, bipedal walking, and particularly a voice, have the effect of creating enormous selection pressures upon the brain. A hand grabbing a pole can kill one animal a year or a hundred, depending on how smart the brain behind it wields it. It may take one individual one lifetime to teach one other individual how to make a tool, or in the same time teach ten-thousand, completely depending on how well they use speech. Because birds and dolphins and dogs can't really manipulate their environment to the degree a creature with a hand can, the selection pressures fall upon other parts of their physiognomy.
Don't blame me, I voted for Baltar.
At IBM "Lotusphere" event, the closing general session was a preview match. It was done as close the the same was as an actual jeopardy show but with contestants picked from a small mini-tournament of attendees, and a comedian as host instead of Alex T.
Having been present at this (and getting my picture taken next to the Watson "icon/screen") and then watching the Nova episode, I can say for sure that the Nova show was a very well done description of what happens; as well as Watson's strengths and weaknesses.
I'm not sure if they'll show it on the live TV show taping, but in the run through we saw, they showed Watson's top 3 picks with a level of confidence on each. It was as interesting to see the second and third choices as it was to see what it actually came up with for an answer.
A couple of things were updated from when they must have taped the Nova show. First, Watson was far more strategic when it came time to place bets than it had been shown on Nova. Second, it was far better at understanding weird language in the categories.
I'm looking forward to the show.
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
I didn't see him put forth any Turing analogue. He only claimed that following a very specific set of instructions established by someone else is not intelligence (i.e., it is *not* a valid Turing test) [and he's right].
Yes.
Watson is given the "answer" (remember it's Jeopardy) as textual input at the same time as it is read to the other contestants. It then figures out all the likely questions along with the likelihood of each of them being right. Only once it's figured out which answer it's actually going to answer will it buzz in.
It won't use the additional time which a human has after they've buzzed to continue to think about the problem - unlike the human competitors.
The thing about Jennings is that in the end he just buzzed for everything because he was so confident he knew he could figure it out during the thinking time. Consequently if Watson is to win it's got to come up with the right answer in the time it takes for the question to be read and the reaction time the other competitors will take to buzz. This'll come out at ~ 3 seconds.
When you think about the problem like that, all of the crap people are spouting about "it gets the question as text rather than _hearing_ it" and "it's got a huge database to search" becomes moot since it's got to so much to do in just 3 seconds!
-- "Hey kids, try this at home!"
So long as computers continue to be built as they are, they will never be anything but numerical manipulation. At their fundamental level, that is all they do. They manipulate numbers in various ways. At the fundamental level there is just binary data, without type or form. In memory the data isn't even characters or pixels or any of that, just a long string of binary digits. It is only given form by the programs that are written to use it.
However that doesn't mean that the behaviour at the higher level will not emerge to be like that of humans to be able to understand, as in correctly parse and work with, human speech and concepts.
After all at a fundamental level humans are just neurons making electrical and chemical signals. When you get down and look at a single neuron and how it functions in the brain it gives no clues that an intelligence might emerge from it.
So in the end, computers will always just manipulate numbers, unless we change how we build processors. That doesn't mean that their numerical calculation may not give rise to something that can accurately be called "intelligence".
On the Nova episode, I believe they said they "texted" the answer to Watson. (I presume they really mean they somehow transfer it, and not actually use SMS. I don't see a transcript of the episode online to confirm.)
However, while you also mentioned reading, I don't think it's as much of an advantage. We can always read the answer much quicker than Alex can speak it.
IIRC, when there was a blind player, the only extra he got was a Braille list of the categories, not each answer in Braille...
It's not like you have a particularly more concrete grasp of birthdate of John of Gaunt than a machine does.
No. Quite the opposite. The reason I know I'm different in the way I grasp the birth date of John of Gaunt is because I can generalize the concept of birth date in ways that a machine cannot. Your example is a perfect one to demonstrate my point. I can look up on Google and find out that the man known as John of Gaunt was born March 6th, 1340. That seems like a reasonably well established factoid. However, is this the birth date of John of Gaunt? I subject to you that it is not. It may well be the birth date of the person who came to be known as John of Gaunt, but I'm reasonably certain that he was not known by that name on that date. It would have been many years later that he would come to be known as John of Gaunt. So I would argue that this is the birth date of John of Gaunt.
Would a machine understand this distinction? Could a machine even parse this distinction?
I can consider the birth date of other things that a machine would not comprehend - the birth date of the universe; or the birth date of my car; or the birth date of manned space flight. There are an infinite number of things I can consider the birth date of. Things that were neither born nor have a specific date to associate.
I can also consider of the birth date of an idea. I can even consider the birth date of the specific idea that I can consider the birth date of an idea.
While it may be possible to train a machine to make similar generalizations, it is not possible to train a machine to generalize any arbitrary concept. That's how I know there's something different going on in my brain.
As an adult, with years of nonstop data input you are able to do this. With literally years of nonstop data input, who knows what machine learning could yield. Remember, children are incapable of abstract thought up until at least age 2, and in some cases, much later. There are well defined tiers of mental development describing what growing humans can and cannot do. Its not that the machine is anything different from you, if anything, its starved of data.
I have a Chinese Room that wants to talk to you.
4096R/EF7BAFA6 79E1 DF98 D09D 898F 9A11 F6F0 DDDC 23FA EF7B AFA6
If you'd been on Jeopardy! and told Alex his question was meaningless because John of Gaunt wasn't named John of Gaunt when he was born, I suspect you'd be the one accused of being the computer. Computers give responses that appear over-generalized all the time -- wether the answer is useful depends on the situation, which is why I sorta carefully said "particularly in the domain of how you know trivia." People often over-generalize or give overly-specific answers, too; they often avoid some of the dumber mistakes Watson does, because they have common sense gleaned from decades of high-bandwidth stimulus, but when it comes to abstract rules and concepts, people often are careless and use lessons learned in situations which aren't germane or appropriate to the situation. Look at how little children draw animals, with human eyes and sexual dimorphism. They take what they know and apply it to new things, and produce inaccurate output. These sorts of inconsistencies contain their own internal truth, like "animals are like people" and "the boy turtle thinks the girl turtle is yucky!" and that's the interesting content, because we cannot understand the mind and these outputs are the only window we have into the mind of others. In a similar way, Watson's associations sometimes give you fascinatingly wrong answers, but they reflect an internal truth, like "People in the news talk way too much about 9/11 and anyone that reads enough news will associate it with disparate concepts." This internal truth isn't interesting to anyone, though, because, unlike children, we can see how Watson works.
I mean generally, when someone says "we know the computer doesn't understand something" it's because we understand the computer. Any creature who's cognitive faculty can be disassembled down to its bolts, by definition, cannot understand concepts itself. The only reason we claim that any particular human being "understands" something is because they produce responses that align with our own model of the concept, and because we cannot see how their brain works. I think if you could actually see how people thought about things, with a Thought X-Ray, you'd be shocked at how differently people modeled concepts, to the point where someone with such an ability would conclude that no one really knows anything, and that all knowledge is emergent from collaboration.
Of course I can't prove that, it's all rather fanciful, but that's just as good as asserting a negative, like:
Don't blame me, I voted for Baltar.
It's striking how many people are willing to die for things they don't understand, let alone converse about them.
I was struck in Wired for War by the stories of EOD units in Iraq who would name their bomb-defusing robots, give them ranks, promotions, and ribbons, and, touchingly, would mourn their robot's destruction. There's one story about an operator who was literally bawling to a support rep at iRobot, asking if they could please somehow repair their bot. They were real creatures to them, and they were completely unintelligent. What really made the robots alive to them is that they were balky, seemed to have a personality in difficult situations (operator's confirmation bias at work), and had saved the operator's lives many, many times. It didn't matter that the robot didn't "understand" why it was being destroyed, the operators were often in a similar situation... what mattered was its (nominal) selflessness and heroism, something the operator's were required to display as well in a war situation.
I mean like, the Chinese Room is interesting, but the dark secret is that, when it comes to the way human beings confer personhood on other things, it makes it so there is no door to the Chinese Room. Only a mail slot, and it's impossible to see what's on the other side. An an unknowable truth is no truth at all.
Don't blame me, I voted for Baltar.
I know that they can't ring in before Alex has finished reading. My point is that I see it as being problematic that Watson receives the text of the clue electronically at the exact same time that the clue is visually presented to the players. It will take a human reader a few tenths of a second (maybe even as much as a second or two) to actually read the clue, whereas Watson has full knowledge of the text of the clue in mere milliseconds, giving it those few tenths of a second advantage to actually contemplate the clue and possibly arrive at an answer before the humans have even finished reading it. I thus expect that Watson fares poorer on clues which take very little time to read, because it does not have as much of a head start on the human players as it does on the more verbose clues.
File under 'M' for 'Manic ranting'
This is probably because 4 species of grasshopper are the only Kosher insects. Watson obviously knew that, which I didn't! (But I'm not jewish.)
Yes, a very good point. AI doesn't need to be smarter than All People, only most people. And I think the brute force simulations of it are getting pretty close. Between AI simulations and robots (and all the acoutrements like sensor tech and such) we're rapidly being obsoleted. The question then is what next? Space?
Cool! Amazing Toys.