Slashdot Mirror
Post-Googleism At IBM With Piquant
kamesh writes "James Fallows of the New York Times reports an interesting search technology that IBM is developing. IBM demonstrated a system called Piquant, which analyzed the semantic structure of a passage and therefore exposed 'knowledge' that wasn't explicitly there. After scanning a news article about Canadian politics, the system responded correctly to the question, 'Who is Canada's prime minister?' even though those exact words didn't appear in the article. What do you think?"
They don't come out and say it, but it sounds like it's just a big ol' LSI System. It works really well for some types of searching, but I'm not sure if such a thing would out perform google for a general purpose search engine.
"Latent semantic indexing adds an important step to the document indexing process. In addition to recording which keywords a document contains, the method examines the document collection as a whole, to see which other documents contain some of those same words. LSI considers documents that have many words in common to be semantically close, and ones with few words in common to be semantically distant. This simple method correlates surprisingly well with how a human being, looking at content, might classify a document collection. Although the LSI algorithm doesn't understand anything about what the words mean, the patterns it notices can make it seem astonishingly intelligent."
That's pretty impressive. It takes quite a clever AI to read between lines and connect concepts, but I have to wonder how much of its 'understanding' was hard-coded rather than purely abstract. Would it be trivial to just stick in another language database and have it read translations of the article the same way?
Nevertheless it makes me feel like all the programming and design I've ever done is pathetic and I will never amount to anything. That's how it is in the software industry - always someone out there who makes you look bad.
Sam ty sig.
Reg-free link
Till you realise the computer answered 'some asshole' which could be any prime minister in the world really.
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
What if 2 sites said the Prime Minister of Canada was Santa? explicity said it, would that overwrite the linked information? How would the system know what is right? You can't always just pick the majority answer, so you need to set up little areas of trust "I trust www.thisplace.com and everything it says" and that site in turn will say "I trust www.overhere.com" but who allocates the trust, couldn't those people be biased?
The semantic web will have a fantastic impact on the world, but the trust issue is something that needs to be addressed, and I don't see how it can ever, globally be done.
More likely we would have systems like this for individual sites, or intranets, trusted circles that would be unlikely to contradict themselves.
hopefully one day, if we truely get a global semantic web, we can see if the answer really is 42 :]
Using google means that this would have to contend with a lot of noise - looking for one nugget of information on the internet will tend to yield a low signal-to-noise ratio. I wonder what would happen if instead, you were to run it using Wikipedia as a back end (full discosure - I'm a wikipedia admin). There'd be less information, but I suspect the quality of the results would be better.
To make laws that man cannot, and will not obey, serves to bring all law into contempt.
--E.C. Stanton
One example is meaningless. To get a realistic idea of how useful this system is, we'd like to see what it says if you ask several dozen questions. For all we know this was the one question out of 100 that it answered correctly.
We must integrate ourselves with computers to a point at which the living being and computer cannot be separated anymore. The perfect union of the biological component (wetware) and computer (hardware) will mark the end of the human race - and the birth of something new and wonderful.
Obviously this will face strong, religious and quasi-religious (ethics) resistance from the old guard but it will pass with the fools themselves.
I for one congratulate Canadian Prime Minister Tim Horton for running a great campaign and his wife Wendy for her fantastic chain of restaurants!
Feed it the news about Iraq. Then ask it what the war was about.
Good bye, new system, too dangerous for "national security".
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
Disclaimer: I haven't read the article; however, I was somewhat involved in research in this field in late 2003 and early 2004.
What the summary of the article claims IBM is developing-- a technology for getting the semantics behind an arbitrary sentence on the web-- is the Holy Grail of the discipline of Natural Language Processing (NLP) and very, very, very, _very_ far away at this point. Many people believe that we cannot ever get there (that's the point of a Holy Grail, after all), but I don't want to be quite as pessimistic (or realistic?) at this point.
The problem here is that English (or any other natural language, for that matter) isn't SML, or Haskell, or some other language with a well-defined denotational semantics. Natural language suffers from at least three problems that make it very tough to gather anything useful from a given piece of text:
(1) Grammar. Natural language isn't typechecked, and frequently uses incomplete sentences, which makes it hard to develop grammars (context-free, context-free probabilistic, lambek-style/proofnet-style or whatever else people have come up with) for it.
(2) Anaphora resolution. "I saw a dog on the street this morning. It was barking". So who's barking, street or dog? Gramatically, both would be possible; only with prior knowledge we can see that we're talking about the dog here.
(3) Polysemy. What does "play" mean, taken by itself? It can be used for different meanings in "to play a game", "a play of words", "a terrific shakespearian play" etc.; you might want to have a look at wordnet one of these days to get a feeling for this. Not knowing which meaning an arbitrary occurence of "play" refers to means that you have to try lots of options when parsing, LSIing or whatever else you do (though most people simply ignore this problem in research today-- it's too hard to disambiguate words in practice).
That's not all, of course-- try thinking of the need to deal with irony/sarcasm, metaphors, foreign words, the credibility of whichever sources you're using etc., and you'll get a pretty good feeling for why this is beyond merely being "hard". Of course, for very small problem domains (a "command language for naval vessels" was investigated in one paper I read a while ago-- those DARPA people definitely have too much money on their hands, but I digress), this can be solved, but general-purpose open-domain NLP is what you need to do a web search.
It might happen in my lifetime, but I won't hold my breath for it.
-- Christoph
Is it just me who would, if designing an AI, would have have a trivial off switch. Probably a few backups, like wire cutters next to the main power cable, a jug of water near the PSU, things like that.
It is just me, isn't it...
Can you imagine when a system of this kind is capable of reading google's online library? If knowledge is power, we are looking towards creating a very powerful entity.
The genius being google's success was paying *less* attention to the content of a page when categorizing it, and relying on links *to* the page instead. Why? Because of spammers.
"Genius" would imply some sort of brand new insight, but citation analysis has had a long tradition before Google appeared on the scene as a search engine. Google's biggest achievement is probably in implementing citation analysis on a very large scale, but they didn't break completely new ground in how people search.
And, in the long run, semantics-based analysis, like IBM's Piquant, is probably going to be the better technology: citation analysis for determining relevance to a query is really just a limited substitute for understanding of the content.
User: Who is Canada's Prime Minister?
PIQUANT: WHY DO YOU ASK, USER?
Because I want to know.
WHAT IS THE REAL REASON, USER?
I want to test you.
WHAT WOULD IT MEAN TO YOU IF YOU GOT TO TEST ME?
It would mean I can determine whether or not you know who Canada's Prime Minister is.
WHAT MAKES YOU THINK I COULD TELL YOU THAT?
I programmed you to figure it out.
LET'S TALK ABOUT YOU -- NOT ME.
I want to know the name of Canada's Prime Minister.
WHY DO YOU WANT TO KNOW THE NAME OF CANADA'S PRIME MINISTER?
Are you going to tell me his name or not?
WHY ARE YOU INTERESTED IN WHETHER OR NOT I AM GOING TO TELL YOU HIS NAME OR NOT?
Stop avoiding the question! Just tell me his name!
NAMES DON'T INTEREST ME.
Fuck you!
TELL ME MORE ABOUT YOUR OWN PROBLEM, USER.
Shit!
GO ON.
You suck.
YOU'RE NOT REALLY STILL TALKING ABOUT ME, ARE YOU?
... is whot bwings os tugevza tsuzay.
As it happens, The Economist recently ran an article addressing some of these issues. The article also provides context and perspective that should be of interest to those participating in this discussion. For convenience, the full text is reproduced below; it is also accessible online (may require paid subscription).
----
Computing
From factoids to facts
Aug 26th 2004 | REDMOND, WASHINGTON
From The Economist print edition
At last, a way of getting answers from the web
WHAT is the next stage in the evolution of internet search engines? AltaVista demonstrated that indexing the entire world wide web was feasible. Google's success stems from its uncanny ability to sort useful web pages from dross. But the real prize will surely go to whoever can use the web to deliver a straight answer to a straight question. And Eric Brill, a researcher at Microsoft, intends that his firm will be the first to do that.
Dr Brill's initial crack at the problem is a system called "Ask MSR" (MSR stands for Microsoft Research). This program uses information on web pages to respond to questions to which the answer is a single word or phrase--such as "When was Marilyn Monroe born?" Ask MSR starts by manipulating the question in various ways: by identifying the verb, for example, and then changing its tense or moving it into different positions in the sentence ("Marilyn was Monroe born", "Marilyn Monroe was born" and so on). The resulting phrases are then fed into a search engine, and documents containing matching strings of words are retrieved. It sounds a promiscuous strategy, but gibberish phrases produce few matches, so, as Dr Brill puts it, "being wrong is very cheap."
Once accumulated, the pile of documents is scanned for possible answers, and these are ranked by frequency. In practice, the correct answer appears in one of the first three places around 75% of the time. That might not sound very good, but human intelligence provides a second filter, since wrong answers are often obvious. If you ask how many times Bjorn Borg won Wimbledon, for example, "1980" is not a plausible answer, but "5" is. If in doubt, clicking on an answer produces a list of links to pages which provide support for that answer.
Ask MSR is still a prototype, although Microsoft is trying to improve it and it may be launched commercially under the name AnswerBot. Dr Brill, meanwhile, has moved to a more difficult task. One of his most recent papers, written jointly with Radu Soricut of the University of Southern California, is entitled "Beyond the Factoid". It describes his efforts to build a system capable of providing 50-word answers to questions such as "What are the rules for qualifying for the Academy Awards?" This is harder than finding a single-word answer, but Dr Brill thinks it should be possible using something called a "noisy channel" model.
Such models are already employed in spell-checking and speech-recognition systems. They work by modelling the transformation between what a user means (in spell-checking, the word he intended to type) and what he does (the garbled word actually typed). Just as a telephone line distorts the voice of the person at the other end of the line, this process can be thought of as being a noisy channel that transforms the user's intention into something rather different.
By analysing many pairs of correct and mis-spelled words using statistical techniques, it is possible to predict how such transformations work in general cases. A system can then be designed to work the process backwards. Given a mis-spelled word, it can guess what that word is most likely to be a mis-spelling of.
Dr Brill's question-answering system does something similar. Many question-and-answer pairs exist on the web, in the form of "frequently asked questions" (FAQ) pages. Dr Brill trained his system using a million such pairs, to create a model that, given
The extreme centre is the paper's historical position. --Geoffrey Crowther
Phenominal technology, IBM built the desktop search that everybody is pushing now, way back when. Cutting edge search and indexing capabilities, fully extendable, you could write your own plugins to deal with your data (use JPEG meta tags to label pictures from your digicam? Write a little plug in so you can search through your photos) and it had semantic and linguisitic searching.
For a long time SM/2 was kind of the poster child for IBM's inability to take remarkably cool technology to the consumer. Everyone that used it thought it was cool, nobody ever knew about it. They had trouble getting the word out within the company about it. Last I heard anything about it, they were turing the technology into some kind of intranet spider. It was the shit, it might have even had primitive cross referencing, like you could search for president and it would find references to Clinton because a third article may have referred to him as the president. They seemed to have some foresight into this area, web searching has to cut out some much bullshit, you wouldn't want to contaminate your semantic searches with all of it, keeping it in intranet space might be a good idea. Local search is hot right now too though so maybe it'll come back.
NLP and semantic extraction and conceputal indexing is nothing new; admittedly, practical implmentations have been few and far between.
;)
However, as I'm often fond of pointing out, the problem is not getting the 80 - 90% accuracy in translation and interpretation that I'm sure these systems can attain.
The challenge quickly becomes how to deal with idioms and idiosyncratic constructions. Is this system even ready to deal with sentences like "The criminal was shot dead by police"? If it is, great. How about "The trolley rumbled through town"? Or the idiomatic "time flies"?
This is what, so far as I know, the field of computational linguistics is now facing in textual interpretation and translation. Coming up with a system to effectively identify what appear to be three-argument verbs ("Mary hammered the metal flat") or constructions or idioms above may well be something that traditional systematic recursive grammars aren't yet up to handling.
Somehow these situations have to be identified, and separated in the parsing process so that they don't get processed like standard grammatical expressions.
Hopefully these problems are how I'll make my living
I've worked for a company making a system that could easily answer a question like that. It really isn't hard to do. If you want to know how much of this is "black magic"/AI and how much is statistics, compare the results of the following two queries:
If the system really understand the semantics of the indexed documents, the two result sets should be very different, and both should have a fair number of relevant documents.
If the system is just based on clever use of statistis, the two result sets will include a lot of the same documents, and the result set for the second query will probably have very few relevant documents.
Acts@core.mailboks.com Acrux@core.mailboks.com Adam@core.mailboks.com Adar@core.mailboks.com Ada@core.mailboks.com