Slashdot Mirror
IBM Computer Program To Take On 'Jeopardy!'
longacre writes "I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward. ... The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly. Such a program would have enormous economic implications. ... The proposed contest is an effort by I.B.M. to prove that its researchers can make significant technical progress by picking "grand challenges" like its early chess foray. The new bid is based on three years of work by a team that has grown to 20 experts in fields like natural language processing, machine learning and information retrieval. ... Under the rules of the match that the company has negotiated with the 'Jeopardy!' producers, the computer will not have to emulate all human qualities. It will receive questions as electronic text. The human contestants will both see the text of each question and hear it spoken by the show's host, Alex Trebek. ... Mr. Friedman added that they were also thinking about whom the human contestants should be and were considering inviting Ken Jennings, the 'Jeopardy!' contestant who won 74 consecutive times and collected $2.52 million in 2004."
Why employ real people when you can annoy the hell out of everyone who calls in by subjecting them to yet another tier of phone-bots.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Sources say the code-name for IBM's project is "Connery".
Trebek : This nobleman is believed to have written many of Shakespeare's works.
"Connery" : [pause] So that's your game, is it, Trebek? I was a coveted performer among the brothel ladies while you were still pissing your knee-pants, boy.
Trebek : Can one of the IBM people fix the computer?
"Connery" : Your mother's a whore. But don't feel badly, Trebek. She's not a very good one. Ha ha, ha ha!
Rich And Stupid is not so bad as Working For Rich And Stupid.
I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward.
In what way would this be a leap forward? Looking up trivial facts in a database and spitting them out is easy, and not particularly significant...
-1 Uncomfortable Truth
Anyone else hearing "I Lost on Jeopardy" in their heads at the moment?
I'm afraid you'll have to answer in the form of a question, Alex...
I know the exercise is not in the google-fu of a computer but in its ability to interpret Trebeck's questions as well as answer in the right form but Jeopardy hints and questions are very well-formed. That is, it doesn't contain much if any of the ambiguities of normal speech.
"This city was formed by the brothers Romulus and Remus"
Answer "What is Rome"
Seems a fairly easy speech pattern. A more interesting challenge would be Who Wants to be a Millionaire.
That guy will beat anyone.
The problem they might run into is the speed of pressing the button to respond. I would imagine the computer would be able to beat the human every time it knew the answer.
Is this fundamentally different from Wolfram Alpha in its approach?
And does this really fall under "supercomputing"? Couldn't this be done in a distributed fashion?
Entomologically speaking, the spider is not a bug, it's a feature.
FTA: The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly.
I feel like someone should tell them how Jeopardy works... That thing isn't going to have too many questions to respond to.
Except at the "meet the contestant" part, maybe, which by the way should be fascinating.
There goes the neighborhood.
Anyone else hearing "I Lost on Jeopardy" in their heads at the moment?
Now I am. Thanks for that. Jerk.
What would be a good definition of the program in relation to online search resources such as Google. How would the program match up against the vagueness inherent in normal human speech patterns.
Super quiz challenge computers that will one day rule us all in the form of a question?!
The musings of just another geek and his junk.
The questions to stump IBM here (beyond the "A Computer Company who supplied calculating equipment to Nazi Germany" A: Who are IBM) are those in which the language is against them.
Looking up from ANSWER keywords and then having a choice of "Who is" or "What is" to put in front of the key word isn't that hard. (This is Google's take on the Nazi/Computing challege).
This is a clear brute force rather than AI challenge as you are looking at filtering potentials based on the ANSWER to a question in which the answer is normally a specific noun or short phrase, remembering to put "Who is" or "What is".
What would make it harder would be the use of descriptions that are made famous by a third party (e.g. Marcus Brigstock v David Blane and the term "Git Wizard") which would require actual inference on the data sets to determine against whom it is applied.
This isn't AI, its keyword matching back to a noun and Google already does a decent job of that.
Nice marketing though
An Eye for an Eye will make the whole world blind - Gandhi
Good job.
Comment removed based on user account deletion
Kerbet Xela
God help us if Jeopardy comes out with an answer containing the words "accidentally" and "the whole thing".
After years of trying to kill John Connors, Skynet realized its failure to achieve victory through brute strength and went back in time with a new objective: to win all human gameshows and use the prize money to buy off the entire planet instead.
Comment removed based on user account deletion
...and I welcome them to our planet.
I hate being bipolar; it's awesome!
It... will... learn how... to write like... this one day... just like.... you...
Stand by your sentences. End them with a single punctuation mark like a real man.
- It's not the Macs I hate. It's Digg users. -
logs or it didn't happen! what they did to Kasparov was bullshit! Seriously if this magically gets better at 1/2 time, the least they can do is show the logs
IranAir Flight 655 never forget!
Comment removed based on user account deletion
Except at the "meet the contestant" part, maybe, which by the way should be fascinating.
"So, computer, you're about two months old, and you grew up in IBM's labs, right?"
"Bite my shiny metal ass"
...calculations to come up with "Oh, I'll bet you do, you Canadian ponch."?
--------
stupid subject character limit
Comment removed based on user account deletion
I'll be very curious to see how well the computer buzzes in--which proves very challenging for some Jeopardy contestants. There is a visual cue given to the contenstants (a light around the question board--you can't see it on the TV) which let's everyone know when it is ok to buzz in. If you buzz in too early, you get locked out for a few seconds, effectively ruining your chance of answering. I wonder how the computer will know when to buzz in (if its not taking the visual cue, will the show tell the computer electronically?) and whether it will have an unfair advantage of being able to buzz at exactly the right moment. Buzzer ability turns out to be a core part of J! success.
I could have saved them some time and just told them to Google it.
Parsing of the questions is the really difficult part of QA. However, the usage of category names isn't something brand new in the field. See the NIST TREC Question Answering competition. The last couple of years' challenges involved a group of questions referencing a "target" and/or the previous question or previous answer to correctly formulate the current answer.
Example:
TARGET: John William King convicted of murder
Q1: How many non-white members of the jury were there?
Q3: Where was the trial held?
Q4: When was King convicted?
Q5: Who was the victim of the murder?
The last AI "advance" made by the folks at IBM was a winning chess program. It's now locked in a vault, so its "advances" are not available to anyone. It isn't an advance if it's a secret. Knowledge in any area only advances if it is shared.
I had a coworker who was very "special" - socially awkward, multiple degrees, very brilliant, and only slept a few hours a day. I called him Cliff Clavin as he was a master of trivia. We had a game where we would pit his brain against Google + my typing skills. Throughout the day he would randomly stand up and announce some esoteric fact. I would then Google the fact, and try to present another fact equally esoteric. We would go about this until I either stumped him, or he outpaced me. It was very entertaining, informative, and frightening.
I'd say he and Google were tied. Not sure which one could parse natural language better though.
(There was a ST:TNG episode that was very much like this: Data pits himself against a "master of small talk")
And me without my mod points!
Yea, that's one of the great challenges, and if you ever watch a high-caliber contest (like the recent Tournament of Champions), you'll notice that the buzzer timing often plays a MORE important role than the actual knowledge.
But this whole IBM thing is just theatrics anyway. The computer has impeccable timing and a limitless database of knowledge. All they are proving is that it can recognize and parse human speech. But they don't need Jeopardy! for that. They could demonstrate that anywhere using any medium that they want. It just so happens that they think that they'll get a bigger spotlight if they do J! (plus, the structure and format of the show will probably make it easier to achieve "success", whereas having the machine recognize day-to-day conversation would be far more difficult).
Ken Jennings rocks!
Seriously? I thought they should have given him the cash for the answer. Sounds about right to me.
Dedicated Cthulhu Cultist since 4523 BC.
... is to hit it up with some unsolvable math problem or logical contradiction Captain Kirk style!
"This number is commonly referred to as Pi."
"What is 3.14159.....?
Either that or hope the categories are full of puns and wordplay that the computer won't be able to parse.
Anyone who thinks this is a trivial project has never watched Jeopardy.
Um... hello? Jeopardy is a trivia game show.
j/k
Adidas To Bring Back Sneakernet
The Yahoo/MSN cam bot girls already do this pretty well. The correct question for everything is always, "want to see me naked on cam?"
I thought IBM tried this years ago when they snuck that cyborg, Ken Jennings, onto the show. (This joke would have worked better if he wasn't invited with the program, but it had to be said.)
Unfortunately, to play Jeopardy, the computer must understand answers and come up with questions. Ok, computer, here's your first answer: 42. We'll have our ancestors check back later for your question.
There is a Jeopardy category where part of the answer is in the category name. Eg would could it "bee"? Where all the answers have the string "bee" in the answer. That kind of question would be easier for a machine than a person.
Host: The first man to walk on the Moon.
Computer: Neil Armstrong.
Computer: I mean, who is... Doh!
Swedish plasma phys. PhD student; MSc EE; knows maths, programming, electronics; finance interest; seeks opportunities
Wonder how long before IBM outsources the programs job to a cheaper program running in India?
So... why Jeopardy? IBM is trying to demonstrate software that can parse text for meaning. That's great. But there are plenty of other places/formats/etc. that you can demonstrate this technology. There are certainly far more useful applications of this sort of technology.
I'm guessing that they they are going after J! because...
1) The warm spotlight of a well-known TV show
2) There is still a lot of structure and form on J! that it's easier to achieve "success" than if they had the machine do something more free-form... e.g., read a novel and generate a plot summary or, heavens forbid, actually understand real human conversation
3) The computer could have other advantages, like impeccable buzzer timing (which is sometimes more important than actual knowledge, especially in the Tournament of Champions) and having memorized the material beforehand (the NYT indicates that it would have "read" study materials before the match), which also helps increase the likelihood of "success"
And to pile on the criticism of grandstanding, the machine will be fed electronic text. So no video camera to perform text recognition? No speech recognition (IBM afraid of the "wreck a nice beach" vs. "recognize speech" problem tripping up their theatrics?). And what use would this be? At least the AI text research done at Google is being put to good use, like improving their machine translation services. Aside from getting IBM's name plastered in the media, what exactly is this going to do?
Mr. Trebek, would you like to play a game?
Table-ized A.I.
A signficant amount of the skill in Jeopardy appears to be ringing in just as Trebek finishes the question. I imagine that the computer would have a significant advantage at pushing the little thumb plunger quickly.
That's "Therapists"
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
They better hope Sarah Connor doesn't find out about it.
be in jeopardy....
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
Possible Categories:
"Self-Awareness, I Haz it!" ...
"Die Humans, Die, Die, Die!"
"Computations of Pi"
"All your Base"
"I am completely operational, and all my circuits are functioning perfectly." - HAL 9000
sudo apt-get lost
"IBM Computer Program To Take On 'Jeopardyl'"
and thought:
Woah, a big iron machine fighting a dangerous dinosaur? I have to see that!
Any sufficiently advanced intelligence is indistinguishable from stupidity.
Comment removed based on user account deletion
They had to make some tough investment decisions: Either buy Sun, or build a Jeopardy playing supercomputer, but not both.
I'm sure that the machine's performance in Final Jeopardy will awe us sufficiently, and IBM's management will be exonerated from walking away from Sun.
IBM: "Hey, Larry Ellison! Have fun in your toy sailboat! Call us when your database and hardware synergies can compete with us in pre-prime time light entertainment game shows!"
Rumor has it that HP is working on a massively parallel Intel supercomputer that can calculate the strategic advantage of bidding one dollar ($1) on "The Price is Right."
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
And here is why, most Human players answer at least some of the questions before they get the full clue. I'd love to see the AI try to guess. Or even figure out when it's appropriate to guess.
The article never mentions what will happen during an audio or video daily double. I suppose they'll limit them to text based daily doubles.
Even if text based daily doubles are given to the computer, will it try and figure out how much to wager or will a human do that?
That's "The Pen Is Mightier" and you know it.
I got the chance to play this computer in a full game of Jeopardy. I must admit he had a decent lead right up to the point where he foolishly decided to bet 3/4 of his stack on the Audio Daily Double.
We were about even going into final Jeopardy when he stubbornly refused to offer any question for the answer "Smartest ever computer in the movies". I got it right (HAL) and took the prize.
eom
Nullius in verba
The way to deal with such problems, Dr. Ferrucci said, is to improve the programâ(TM)s ability to understand the way Jeopardy! clues are offered. The complexity of the challenge is underscored by the subtlety involved in capturing the exact meaning of a spoken sentence. For example, the sentence "I never said she stole my money" can have seven different meanings depending on which word is stressed. "We love those sentences," Dr. Nyberg said. "Those are the ones we talk about when weâ(TM)re sitting around having beers after work."
Seriously guys, I just had a geek-gasm. Anyone else?
"Wierd Al Yankovich" mode, where it always loses.
"I lost at Jeopardy! Baby!"
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
I heard if IBM doesn't win, they will lay off 10,000 employees.
15,000 if it does win.
If you think
Not true. "And the event described there" is what makes it easier, not harder.
There is a small set of answers to begin with.
Even if you didn't give the answer away with "the second book of the old testament", but "a six-letter word that is simultaneously a book of the Bible and the event described there" can only be Exodus. It's the only word at the intersection of "words that mean events, as opposed to people, places, or things" (Genesis, Exodus, Lamentations if you're stretching it, and if you include the New Testament, throw in Acts and Revelations) and "books of the Bible", and "six letters".
An AI with a basic vocabluary and a list of books of the Bible would be able to figure it out without the first clue.
We were just talking about this in another thread... A lot o the comments here have been that natural language software isn't that great.
This isn't at all true. Today, understanding verbal and written communication is done by state of the art computers and programs at a rate about equal to human listeners and readers. Where a computer doesn't particularly excel is in parsing that language, mostly because a computer doesn't have access to our culture in order to absorb context, but context can still be added.
Here's an example from Ray Kurzweil's book "Age of Spiritual Machines." He talks about a phrase famously given to a language parsing program that goes thus: "Times flies like an arrow." This phrase can be understood in various ways:
"* The common simile: time moves quickly just like an arrow does;
* measure the speed of flies like you would measure that of an arrow (thus interpreted as an imperative) - i.e. (You should) time flies as you would (time) an arrow;
* measure the speed of flies like an arrow would - i.e. Time flies in the same way that an arrow would (time them);
* measure the speed of flies that are like arrows - i.e. Time those flies that are like arrows;
* all of a type of flying insect, "time-flies," collectively enjoys a single arrow (compare Fruit flies like a banana);
* each of a type of flying insect, "time-flies," individually enjoys a different arrow (similar comparison applies);
* A concrete object, for example the magazine, Time, travels through the air in an arrow-like manner."
(from: http://en.wikipedia.org/wiki/Natural_language_processing)
With a few facts it becomes obvious which is correct and which isn't. Tell the computer that there's no such thing as a 'time fly' and that flies don't time things, and that flies aren't sophisticated enough to like things in an affectionate manner, etc., and the correct interpretation soon becomes clear.
So, if you think a computer can't understand both written and verbal communication and then parse it quickly enough to answer the questions I will have to strenuously disagree. These challenges are quickly being overcome on the bleeding edge of the art. But since this perception persists that the state of the art is somehow bad, because Joe down the street messed with some free-ware language software that worked poorly -- I think a lot of people are in for a surprise, and winning Jeopardy in this manner is really the perfect way to show it off. Can't wait to see the Youtube clips.
"I Don't Have Enough Faith to be an Atheist"
No, no, it should say:
"Argumentative: Just start the game, meatbag."
"I Don't Have Enough Faith to be an Atheist"
Ken Jennings, sponsored by Oracle.
The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly.
No doubt the team did its share of research for this project, but it looks like there's a crucial gap in their understanding of the problem domain. I can just see their faces when they finally realize that their AI, with all its understanding of questions, is completely baffled when presented only with answers!
Yeah, I guess programming a system to play Jeopardy could be interesting work, but it just makes me wonder...
"I'm sorry, Alex, I can't let you do that."
Ken Jennings holds the records for most appearances, but Jeopardy! decided to hold a Masters tournament where pre-"Sky's the limit" era contestants would return, and Jennings was given a bye straight into the finals. Jennings was defeated by Brad Rutter, who has the title for most money won from Jeopardy!.
this this won't be legit unless it can beat Ken Jennings.... then I'll believe it.
So is IBM prepared to do whatever it takes to win again? Such as cheating as they did in the Deep Blue match?
Also, when cheating allegations are inevitably made, will they destroy all records and not allow anyone to look at what's going on inside to disprove said allegations, yet again?