Slashdot Mirror
The Future of Google Search and Natural Language Queries
eldavojohn writes "You might know the name Peter Norvig from the classic big green book, 'AI: A Modern Approach.' He's been working for Google since 2001 as Director of Search Quality. An interview with Norvig at MIT's Technology Review has a few interesting insights into the 'search mindset' at the company. It's kind of surprising that he claims they have no intent to allow natural questions. Instead he posits, 'We think what's important about natural language is the mapping of words onto the concepts that users are looking for. But we don't think it's a big advance to be able to type something as a question as opposed to keywords ... understanding how words go together is important ... That's a natural-language aspect that we're focusing on. Most of what we do is at the word and phrase level; we're not concentrating on the sentence.'"
"I'm sorry Dave, I'm afraid I can't search that."
I tend to agree with Norvig's focus on keywords and less emphasis on natural language. Trying to even define a natural language on top of a query engine introduces a layer of complexity probably unnecessary. Natural Language even introduces a level of noise to interfere with accurately (as possible) defining what the user is asking for.
Google has done a good job, and they get better each iteration figuring out what the user is looking for. I find their suggestion an effective way to not only constrain a query, it actually provides a way to spell check in a pre-emptive way. If you've not used this, install the Firefox Google toolbar, or use the experimental Google "Suggest". Often Google will provide suggestions in the drop down menu that refine your search in ways you hadn't considered that drive to a more direct and accurate representation of your intended query. Of course if their suggestions don't satisfy, you get to continue typing your keywords to your heart's desire.
(I have to offer an example of suggestion's effectiveness. I often Google to get to the Chicago Tribune (I don't visit there often enough to have created a bookmark, plus it's easy to do this in anyone's browser). Simply typing the first four letters, "chic", I see the first suggestion is "Chicago Tribune". A simple TAB and RETURN, I'm on the Google page with the first link or so my link to the Tribune (with the added bonus of Google's breakout of sublinks).) Your mileage may vary (Google's ranking system may vary the order and options that appear in the drop-down over time), but I find it an amazingly effective research tool (suggestion, not the Trib).
Natural language is mostly trying to guess intent with structure and key words (as opposed to keywords), but at the end of the day, if you filter out the natural language, and focus on keywords you're going to end up in close to the same place.
The problem with natural language searches is that natural language itself is a moving target. Sure, ten years ago "How do you change the air filter in a Toyota Camry?" would have been a legitimate question to ask a search engine online, but these days it would probably be asked like "lol how do u chng filtr in my pos car? kthxbye :)". I don't know how Google is supposed to keep up with that.
Isaac Asimov's fictional Multivac was a huge computer with some near-universal knowledge database that answered natural-language questions, giving Asimov all sorts of opportunities to present philosophical conundrums as entertaining short stories.
In the 1960s and thereabouts, when I used to hack around on minicomputers, but personal computers weren't well known to the general public, I always found it difficult to explain what computers did. One of their commonest questions was "Well, how does it work, do you type in questions and does it answer them?" Programming in assembly language didn't really fit that description.
Many technological fantasies seem to remain surprisingly distance. I tried ViaVoice and gave up: it's not a "voice typewriter." Roomba is not a general-purpose housekeeping humanoid-form robot, and neither are the machines that weld automobile chassis.
However, it seems to me that Google is within striking distance of Asimov's "Multivac" fantasy.
Incidentally, if you type in queries as complete sentences Google seems to do any worse than if you don't. Sort of the converse of adventure games, where one begins by typing "Walk over to the table on the left and pick up the silver key with your left hand" and quickly learns to use telegraphic style: "Go table. Take key."
"How to Do Nothing," kids activities, back in print!
A phrase is part of a sentence. WP
No, researchers in this field generally aren't kooks. Mainstream researchers realize that conlangs are not appropriate objects of study.
When even Lojban supporters admit that they have not succeeded in carrying on conversations for much longer than a few minutes in the language, then it doesn't look too likely that the project will take off.
Furthermore, wasn't the point of Lojban initially to test the Sapir-Whorf hypothesis by teaching a child only Lojban? I'd hope that any parents that embarked upon such a stunt would be prosecuted for child abuse, because you would have to isolate a child from all human society to ensure he doesn't learn the local language.
text-to-speech or speech-to-text is also useless (unless your blind/ deaf/ driving a car)
the idea of interacting with a computer like a human is an artificial hangover from being introduced to the computer the first time. after using it for awhile, you realize that ineracting with a computer, in small limited ways, like searching information, is easier NOT using natural language
for the very simple reason that it takes more thought, and more typing to interact naturally. it is easier to train a human to interact with a computer than it is to train a computer to interact with a human. and for the human, it is more rewarding, because the human realizes he doesn't need to exert so much effort
"what is the capital of france?"
versus
"france capital"
if you were to shout "france capital" at someone, it would be rude and confusing. but for a computer, it's actually superior
it is the conservation of communication effort at work here that wins out over natural language in computer interaction
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
Typing "What is the capital of France?" won't get you better results than typing "capital of France." ... Most of what we do is at the word and phrase level; we're not concentrating on the sentence. We think it's important to get the right results rather than change the interface.
This misses situations like searching for "That sf-short-story were the crew of the visiting spaceship is given a dog as a present" in which googling failed, at least for me, or, more technically, when you have absolutely no idea about what the relevant terms within the outcome might be. In short, if you have a real question.
CC.
TaijiQuan (Huang, 5 loosenings)
These days, hardly any user enters queries in the form of natural language questions, judging from log files. That was different a couple of years ago.
Just like "Click here to do X" isn't used as much on Web pages anymore. People now tend to know that they can click on underlined text to find out more.
Everything. All languages are natural. In fact, the spoken word is as good a subject to study evolution and 'survival of the fittest' (to a degree) as any biological organism. The way that different languages and dialects have collided over the years and weeded out words, phrases and structures that work or don't work is one of the most complex and interesting topics around. Despite its quirks the English language is as natural as any creole or foreign language out there, simply evolved differently.
art is science made clear. -cocteau
do people really type questions into search boxes? that always stumped me about the ask jeeves thing....who the crap really ASKED anything. I thought you just googled what you wanted to know about (or nowadays, hit the wikipedia page for it for starters).
Maybe I'm just not up on my search engine technology (or, rather, I don't know anything about it). I just don't know anybody who'd think to put a regular question into google.
"That's easy! The capital of France is 'F'."
----------
Any problem can be made unsolvable if there are enough meetings made to discuss it.
If you have the opportunity to look at query logs, you see how dumb most search engine queries are.
First, a big fraction of queries are simply navigational. Many are just URLs. The major search providers recognize these in the front end machines and send back canned answers, without even passing them to the real search engine. If you type "myspace" into Google, very little work is expended returning the canned reply.
After that, most queries are one word. Phrase queries are less common.
Few people seem to have noticed, but Google started returning results based on synonyms and homonyms a few weeks ago. There have been some significant algorithm changes recently.
Less than 1% of queries use any operators, like '"" or '-'.
The real problem with natural language queries, though, is that "Ask Jeeves" was a flop. Remember Ask Jeeves? That was a system designed to process queries written as sentences. But it wasn't used that way, and didn't succeed commercially.
I think Norvig's lying. Google may not be pursuing linguistic structure above the phrase level in searches, but I'd bet a donut they're working their asses off trying to analyze crawled docs linguistically. To get relevance, they need to extract what a document is about. That implies sentence-level syntax analysis, which is input to sentence-level semantics, which is input to paragraph-level semantics, which is input to "pragmatic" analysis. I think what he's not saying is that the place the linguistic research dollars are going is elsewhere than parsing "Where is Paris?"
How much natural language do you really need for a search? Not much.
I think that actually misses the point. If you've worked as an engineer or a consultant - or even if you've just helped people search for stuff on Google - you probably have realized that THEY DON'T KNOW WHAT TO ASK FOR. A really good consultant/engineer is someone who has the ability to figure out what a person wants based on what they say.
Even if you mastered natural language (and I'm not saying that's a surmountable task) I think people would be shocked to see that Google searches would still be frustrating.
I'm not just saying "blame the user", I'm saying that language itself is not even the last obstacle to overcome. You're going to need to figure out an program that not only understands natural language, but also context, culture, etc.
Getting an AI of near-human intelligence is not enough, because to be really good at getting people the answers to questions they can't ask you have to be of above-average capability.
The Southern Baptist Convention has creationism. On Slashdot, we have porn.
All you have to do is look at Yahoo answers' average question clarity to get a sense of why whole-sentence AI may not be the best strategy for a search engine.
stuff |
Bah, the engine just has to ask refinement questions. Of course, this could be interesting:
.... And so on and so forth...
User: Who is the winningest coach in football?
Search Engine: Did you mean, What coach has the most wins in football?
User: Yes
Search Engine: Did you mean American football?
User: Yes
Search Engine: NFL NCAA CFL...?
User: Umn, all of the above
Search Engine: Are you sure?
User: What?
Search Engine: Are you sure you want to compare all years, after all, NFL rules significantly changed in 2001, and leagues are not comparable...
User: Yes.. Yes, please compare them all....
Search Engine: You know winningest isn't a word right?
Natural language processing is useful when it is well-done. Getting it well-done is the tough part. Don't let Google reps trick you into thinking otherwise just because their R&D in the field isn't where they'd probably like it to be.
Here are some situations where it's useful:
1) interpreting a question rather than just treating it as a "bag of words." For instance, one can type "how tall is Mt. Everest" in the search bar and Google, rather than searching for documents that contain those 5 (or so) tokens will interpret that as a query asking for height and also search for documents that contain "Mt.", "Everest", and "height". Take that a step further and it might look for strings that represent height such as a number followed by "ft" or "meters" or "m".
2) Condensing query chains. Suppose you want to know what sport our 4th president enjoyed playing most. You can ask "what sport did the fourth president of the US like playing?" and the system will give you an answer by first interpreting "fourth president of the US" as Madison, and then searching for what sports Madison enjoyed playing. If not for such interpretation you would either have to run 2 queries (first to find out who the 4th president was, then what sports he liked), or hope that there is a document out there that Google's indexed that contains the words in that initial query.
3) Speech recognition! If you want to run a Q/A session with a computer system that has a speech recognition front end, it is more natural (easier and faster) to ask it "how tall is mt. everest?" than to say "mount everest height" or whatever you would end up typing into Google today. People like to speak using *natural language,* after all. They would gladly do it with computers if the SR systems in them were good enough (some are).
4) More precise query results. What's better, getting back a document that is likely to contain the answer to your query, or getting back the sentence that contains it? Or better yet, getting back the answer and nothing else? The more robust an NLP system the more complicated queries it can interpret and the more elegant its result can be.
On that note, Google actually *does perform* NLP on queries despite what from the summary (I didn't RTFA) looks like claims to the contrary. If you ask Google "how tall is Mt. Everest?" it actually DOES interpret that particular sentence and gives you the answer -- 29000ft or thereabouts. And you only get such an elegant result if you type "how tall is Mt. Everest" (without quotes) or "Mt. Everest how tall". Other queries of this nature will not give you quite as precise a response.
I like basketball!!1!
User: NAKED WOMEN!
Search Engine: Would you prefer woUser: NOW!!!
Search Engine: *sigh* As you wish...
> wii
Your query does not include a verb.
> find wii
Whose "wii" do you want me to find?
> find wii review
Unable to find any reviews authored by "wii".
> find review about wii
No reviews found concerning the common noun "wii".
> find review about Wii
Here is the most recent review about the proper noun "Wii": [url to a page full of keywords related to Wii]
> find review about Wii order by relevence
"relevence" is not an English word. Did you mean "relevance"?
> find review about Wii order by relevance
Here is the most relevant review about Wii: [url to a 2 year old pre-review of the Wii before it was launched]
> find review about Wii order by relevance then date
Here is the most recent and most relevant review about Wii: [url to a fanboy site]
> find all reviews about Wii order by relevance then date
Working...
> abort
Abort what?
> abort search
I am currently performing 1,231,415 searches. Which search do you want me to abort?
> abort last search
You do not have permission to abort others' searches.
> abort my last search
Last search aborted.
> find several reviews about Wii order by relevance then date
"Several" is not a quantifiable adjective. Do you mean "seven"?
> find seven reviews about Wii order by relevance then date
Here are your results. For better search results please capitalize the first word of sentences, and end sentences with proper punctuation.
Dan East
Better known as 318230.