Slashdot Mirror
"Understanding" Search Engine Enters Public Beta
religious freak sends word of the public beta of Powerset, a closely watched San Francisco startup that promises an "understanding engine" to revolutionize Web search. An article in SearchEngineLand points out that Powerset is reaching higher than for mere "natural language." Techcrunch has more details and analysis. For the beta, Powerset makes available all of Wikipedia to search — not all the Web. It's said that their understanding engine required a month to grok Wikipedia's 2.5M articles. The Web is currently at least 8,000 times as large.
But come on, that's a simple question. Let's talk stuff I get into arguments over with my coworkers: Who played the villain in the first Die Hard? Which at least put Alan Rickman at #8. But let's try mutating that to make it harder but still understood by you and I: Who played the bad guy in the first Die Hard? Which resulted in very little but drivel with no mention of the great Alan Rickman whatsoever
So maybe it can't understand 'bad guy.' Well onto another question: Who was the organist for The Beatles on Abbey Road? Which resulted in at least the first 20 having no mention of the great & oft forgotten Billy Preston.
So you want to know what the kicker is? I put those same inputs into Google and found the name in the first or second result. Granted PowerSet doesn't do the whole web, I'm pretty sure that if it did, it wouldn't have the pretty results that it gave when I did what one of the articles told me to--ask it when earthquakes hit Tokyo. Just imagine the dates it would come up with if it hit a site with an html table of any seismic activity whatsoever in Tokyo!
I think it's a novel idea to mine Wikipedia for a search engine so long as it isn't just plain old token matching like PowerSet seems to be up to. Be inventive, try a natural language parser written in Prolog that digests all of Wikipedia into a huge network/ontology of concepts
I find them talking about this in the articles: Powerset is different. It says that its technology reads and comprehends each word on a page. It looks at each sentence. It understand the words in each sentence and how they related to each other. It works out what that sentence really means, all the facts that are being presented. This means it knows what any page is really about. Yet, I'm not impressed. You can try to personify your software and convince me that Baby Alive really defecates like a human being all over so it feels like I have a real baby. But I know it's just software. You don't have to dumb it down if you're going to blog about it. What is this? A pattern matching implementation? A depth first search tree parsing implementation? An ontology builder? Could you at least drop one of the buzzwords of the natural language parsing field for me here?
So does this story actually have more than a startup looking for a sugar daddy to buy it out?
My work here is dung.
True Knowledge actually interprets your question using Natural Language Processing, and then looks through a massive database of user-contributed facts, combining them using sophisticated inference rules, to give you the answer you need. Even the inference rules are user-editable.
We need a +1 Punny.
I asked 'Where do babies come from' and it just gave me back a bunch of articles with that string somewhere in their text.
Pathetic, and you'd hope it's got a long way to go really because at the moment it does NOTHING of merit that I can see.
They're very different. It's not expected that this natural language parsing will replace SQL (anytime in the foreseeable future).
Every so often, I find myself wanting to use them natural language in google. Like today I wanted to find out about the symptoms of a codeine histamine reaction. Sure, I could search for 'codiene', read about it and follow links (on no doubt, wikipedia) until I find what I want - but being able to search with "What are the symptoms of codiene histamine reactions?" is quite powerful.
Although, to be honest I'd prefer to be able to search google with regex and hashes (like search for all pages/images that have a certain MD5 hash).
1 + 1 = 2 is a special notation/langauge that is both more consise and easier than writing "add one and one to make two". So is music score, which is far easier than reading make a high note for a bit then wait a bit and make a low note". Same with C, C++, SQL or Python: the hard bit in programming is algorithm design, not understanding the actual language itself.
Is Natural language really a barrier to entry in using Google? I doubt it. My untechy wife and her friends find everything they need. Plugging natural language into Google gives reasonable results moset of the time.
Engineering is the art of compromise.
So I tried to search for the person who quoted, "What doesn't kill you only makes you stronger.". The search text was "Who said, "What doesn't kill you makes you stronger?"
Google returned the closest match, who was Frederich Nietzsche, with several websites pointing to him. However, Powerset returned only instances of people who randomly said that quote. Google returned what I was looking for, while Powerset returned instances of the phrase (including one reference to Nietzsche).
I can't really say which one is better. Google has the entire web to its advantage, while Powerset is just growing. It seems that the search engine has a lot of potential to grow, which is great as Google and company could use another competitor in the mix.
I've been trying various queries, and Google is doing better than Powerset even when I type in some actual question, like "How many Japanese died in WWII?".
Question: "What is the planet closest to the sun?". First answer from Powerset: "Pluto".
I think I see how this works. It takes the question and breaks it at noise words, ("closed class words" in linguistic terminology) constructing a query with both words and phrases. So "What is the planet closest to the sun" becomes "planet closest" sun. In fact, if you rewrite a natural language question in that form and use Google, it does better on question-answering than Powerset does.
Remember Ask Jeeves? It worked like that? No technical breakthrough here, move along.
I believe you've stumbled upon this startup's business plan.
-David
I totally agree! What is the benefit of asking a computer questions using natural language? It is just going to be making an educated guess as to what you really mean. I am thinking of the stupid little dog in MS Office or the computer on the ship the Golden Heart in Hitchhikers Guide. "Perhaps you would like some tea." "Share and enjoy!" Those aren't the type of conversations we want to have with computers. That's what people are for. But really I don't think natural language works with people. I think we should get rid of it. How many times do you hear "what do you mean?" or "oh, I thought you meant..." Natural language sucks. And I have seen some very passionate poetry written in XML and Java.
Actually, am I the only one who thinks that google's results are worse now than they were years ago? It's still the best general search engine out there, but it often gives me results I don't want now, forcing me to put plusses in front of every word or quoted phrase just to make it actually search for what I asked for.