Slashdot Mirror
IBM's Question-Answering System "Watson" Revisited
religious freak writes "IBM has created and made the question answering algorithm, Watson, available online. Watson has competed in and won a majority of (mock) matches against humans in Jeopardy. Watson does not connect to the Internet to answer his questions, but rather seeks answers using many different algorithms then employs a ranking algorithm to choose the best answer." We mentioned Watson last year as well.
Part of the deal with Jeopardy! is that they will have as part of the 2010-2011 season be a televised episode in which record-breaking champ Ken Jennings will play against Watson, with a to-be-named-later champion in the third slot. This has been in the works since 2009, but the staff of the show finally thinks the system is ready for it's televised match.
One key factor is how the human behavior will change when prize money is at stake. Jennings has proven in numerous appearances on GSN that he's willing to play in any test of knowledge and the fact that he knew he was Jeopardy's first millionaire in regular season play didn't stop his long Jeopardy! run. He also studied for the show, particularly alcoholic beverages (which he doesn't drink) because he had seen the Potent Potables category on TV.
But, what about that player-to-be-named later? Will they know more than the grad students... and play the game not as if it's for points but real dollars?
and see students from the MIT Robotics Lab test their machine that they say can avoid the Bankrupts and find that Million Dollar wedge on the Wheel of Fortune!
Why can't I ask it my own questions?
Because, like Deep Blue at its time, it requires much more computing power than today's typical web site or PC. Chess has finally been solved to the point that there's now unbeatable AIs available to the average user (assuming it gets to move first) but Jeopardy! hasn't, which is why this is novel. It'll take several more years of computing power increases before we'll be playing this AI on our home video game systems.
If it can properly rate if these people are hot or not.
Attention... all grammer nazi"s! Is they're anything; wrong with: my post,
What is the air speed velocity of an unladen swallow?
__ Someday, but not this morning, I'll finally learn to use the preview button.
Chess has finally been solved to the point that there's now unbeatable AIs available to the average user (assuming it gets to move first)
There are no unbeatable AIs for chess yet, that would imply chess is a solved game (http://en.wikipedia.org/wiki/Solving_chess). It doesn't make much of a difference who moves first, either.
Chess has finally been solved to the point that there's now unbeatable AIs available to the average user (assuming it gets to move first)...
No, checkers has been solved to that point. The solution is available online. Perfect play leads to a draw.
Computer chess is merely at the point that if you haven't been on the cover of Chess Life, you're going to get trounced. Even if you have, you're going to lose more than you win. The current situation is that Deep Rybka 2010 has an ELO rating around 3150. That's running on a 4-core AMD-64 desktop machine. The all-time human record is 2851, which Garry Kasparov had in 1999-2000.
This is a given, since there aren't even any AIs yet, so there could be no unbeatable AIs.
I admit that as a programmer this is a bit of a pet peeve of mine. But people: computers are NOT "intelligent". At all. They do what they do by performing the same specific instructions over and over and over again. That's what computers are good at. Granted, they do more complex things today, but that's just due to clever programming and the ability to do many more instructions many times faster than before (i.e., better hardware and software). But that's just quantity. What it lacks is quality. No matter how clever a combination of processors and program may seem at some specific task, it totally lacks the quality we call "intelligence". To call anything running on a computer today "intelligent" is to undermine the word itself. You might as well call a rock an airplane.
Researchers in this field 20 years ago would have been appalled at what people today refer to as "AI". Of course they would also be appalled at the lack of progress in that same field, but that's another matter.
I am not pointing fingers at the posters here. They are just using "AI" in the way it has become commonly used. But that is an erroneous use and I would be happy to see the practice stop. It gives people the wrong idea.
If we (erroneously) call what occurs today "intelligent", then if something ever really did become intelligent, what would we call it?
To call anything running on a computer today "intelligent" is to undermine the word itself. You might as well call a rock an airplane.
I didn't realize "intelligent" had such a clear definition that you could really say anything meaningful about whether a machine was "intelligent" or not.
If we (erroneously) call what occurs today "intelligent", then if something ever really did become intelligent, what would we call it?
I don't know.. perhaps we'd make a bit of progress and realize that "intelligence" isn't some nice single concept to just nail down like mass that we can all agree on what is is and isn't. We might even come up with 10 very different words to describe something we might now use the word intelligence about, since we might actually have a better grasp on what it actually is. If you ask me, intelligence is more about human ego than any real hard definitions. In many peoples minds computers can never be intelligent because it would bring our self opinions down a notch or two. That's why many people were sooo upset about Kasparov getting schooled by Deep Blue 10+ years ago, and then made up a bunch of excuses why it wasn't fair.
Whether a machine "intelligently" plays Chess, or is "intelligent" is a stupid question. What's more interesting is how we might accomplish the same task in different ways than our brains might do so. 40 years ago nobody ever thought a computer could be programmed to play even a decent game of chess. These days it's surpassed us. I think that says more about what we think is "intelligent" than anything else.
AccountKiller
As humans, we do exactly what physics mandates we do. Unless you're purporting that the human brain uses some sort of hypercomputation or that there's something special (ie outside of our current understanding of physics) about what neurons do, you're not being consistent.
That said, I understand where you're coming from; most AI research is in very narrow domains and has no intention or hopes of solving the problem of achieving human-level intelligence (Watson is an example of narrow AI, as it clearly lacks a genuine understanding of the question or the english language). But the fact remains, that is how the term AI is used.
There's a growing separation between this "Narrow AI" and the kind of AI you seem to be hoping for, Artificial General Intelligence (AGI). There are some AGI projects out there, such as the open source opencog. Since there's no hope of people stopping calling things like computer chess AI, I prefer to use the term AGI whenever referencing "real" AI.
To shorten your post: AIs will exist when a computer can write code for itself to run without interaction from a user. Given a problem, and having no prior knowledge of a solution, nor a way to arrive at the solution, an AI will be capable of creating a set of instructions to solve that problem.
Against a programmer, oh, and an algorithm, both with a PhD....
-Weird Al Parodied
Responsibility is an addiction
Virtue is a temptation
Community is a cartel
It's impossible to fully all possible games of chess. The game tree complexity is about 10^123, whereas the number of atoms in the universe is thought to be somewhere between 10^79 and 10^81. Thus, it's impossible to brute force the game since you can't store all the possible states.
If, however, we ignore this, then the answer to your question would be "it depends on how fast it could calculate the results." Some hypothetical computer with sufficient memory and a sufficiently fast processor would be unbeatable using a brute force algorithm by the definition of brute forcing. However, as already explained the "sufficient memory" part is pretty much impossible
While you're right that it's impossible, the reason you give is wrong: You wouldn't have to store all the possible states at once. After you've determined that you cannot win with a certain move, you don't need to store all those states this move can lead to. And if you determined a winning move, you only have to store the sequence of winning moves. The real problem is time. Even if you could check one move per Planck time (the shortest possible time interval, ca. 5*10^-44s), you'd still need about 5*10^79 seconds, or about 1.5*10^72 years. For comparison, the universe is about 1.5*10^9 years old.
The Tao of math: The numbers you can count are not the real numbers.
Ahem. I did that myself and I should clarify this. "Artificial" can mean "fake", but the original meaning was "workmanship" or "craft". Created by hand.
I don't know.. perhaps we'd make a bit of progress and realize that "intelligence" isn't some nice single concept to just nail down like mass that we can all agree on what is is and isn't. We might even come up with 10 very different words to describe something we might now use the word intelligence about, since we might actually have a better grasp on what it actually is. If you ask me, intelligence is more about human ego than any real hard definitions. In many peoples minds computers can never be intelligent because it would bring our self opinions down a notch or two. That's why many people were sooo upset about Kasparov getting schooled by Deep Blue 10+ years ago, and then made up a bunch of excuses why it wasn't fair.
How convenient! A theory about intelligence which means that we have actually already created AI!
Get back to work!
The real problem is time. Even if you could check one move per Planck time (the shortest possible time interval, ca. 5*10^-44s), you'd still need about 5*10^79 seconds, or about 1.5*10^72 years. For comparison, the universe is about 1.5*10^9 years old.
You're forgetting that speed and parallelism aren't mutually exclusive!
If you could somehow convert all of the matter in the universe into a massively parallel computer running at that speed, with each CPU having a budget of ~40 million particles, then you'd have a computer that could play chess perfectly, providing each move in about 24 hours at first, and then probably speeding up a bit as the game progresses and there's less of the state space to check.
Your homework for tonight is to build two such computers using different methods, and compare the relative merits of each solution. Show your work.
Computers are not intelligent because they are unable to reason. They iterate until they achieve an optimal solution to a specific set of rules.
Could you define "reason"? The AI field has worked for a while (until the 80s) to build machines that reason. There were some successes with expert systems and things like that (which, given sufficient data could "reason" according to standard definition). The problem with them is that they need complete information, so they never really made it out of the lab. Current works have turned instead of "reasoning" systems to Bayesian inference engines which use complicated statistical methods and approximations to find the most likely answer. They build estimates of probability distributions based on training (equivalent to experience) and then can use them to make decisions or predictions. They are much more flexible than the reasoning machines that were build before and handle incomplete data appropriately.
These probably don't "reason" by your definition, but then again, neither does the human brain: the current understanding of cognition suggests that it is also a inference engine, making probabilistic judgments based on experience in spite of limited information.
If these methods count as AI, then AI already exists. It is used in everything from handwriting and speech recognition to ranking players on XBox Live. If you look at the tasks that AI researchers hoped to solve when research in AI began, a large number of them have been solved. So if a computer can solve a problem that was previously considered an AI problem, wouldn't it be "moving the goal posts" to say that we don't have any AI today?
I killed it, too, but mainly because of the rules of the game in which I get first crack at the answer.
Still, I think the point is that it's impressive the number of questions it gets right. I really didn't miss very many. My mental tally had it getting around 70% or so, which is pretty damn good. I got around 80%, but again, I had first crack at the answer, so Watson could have only possibly scored around 20% of the answers at best. If it tallied your score and Watson's score without actually competing head-to-head, that probably would have been a more interesting challenge. Keep in mind that as a computer, if the programmers chose to do so, they could probably have Watson answering everything pretty much simultaneously. Also, Jeopardy! is a lot about timing, hitting that button as soon as the buttons are unlocked, so in that respect, Watson probably really could kick ass in that its reaction time will always be faster than yours.
Just because I won doesn't make the technology not interesting or significant. Frankly, I'm impressed that it's even able to answer 20% of the questions expressed in natural language correctly.
The real issue is that AI research has mainly discovered that Intelligence is not well defined, and everything we thought was difficult (e.g. Playing chess) is relatively easy, and everything we thought was easy (e.g. Understanding Human Language) is horrifically difficult
A human child can walk, see interact and understand it's environment, hold a conversation ... none of these can be done even more than adequately by machines .. (Walking is ahead of the rest so far) but a cheap chess computer can beat most people, including most Grand Masters the majority of the time
Puteulanus fenestra mortis
Back to the drawing board my arse. You may have beaten the web page game easily, but consider the fact that you wouldn't get first shot at every single question in a real match. If you assume 50-50 on the ring-in time when both players think they have the right answer, I think Watson looks pretty good on this game (although they wouldn't have put an example of a particularly bad match on the web).
It also has a probability threshold below which it won't attempt a response. The real machine must be set up so that if it's behind in the score late in the game, it would ring in with its highest probability "guess". That would make things more interesting. Think of "Final Jeopardy" too. If the machine can get >50% of the point leader's score, the match is totally up for grabs.
People that go through the qualifying process to appear on the show are definitely on the upper end of the intellect/knowledge bell curves, so that would be a tough challenge. I totally believe that the machine would crush the "average" person.
Both Watson and Google relied on statistically analyzed word occurrences with a veneer of procedural/rule intelligence. Waston seeks the correct answer, while Google the most popular document containing the answer. Neither incorporate "deep" understanding like CYC's rule ontology. But it may not be that necessary if the statistics are large enough.