Slashdot Mirror
Looking for Answers in the Age of Search
prostoalex writes "James Fallows, in a New York Times article, notices that search engines are getting pretty good at providing information for simple keyword-based queries. However, when it comes to the actual information, such as finding the necessary data and statistics, they're not doing a great job. The article talks about the NSA- and CIA-sponsored Aquaint project that aims to deliver answers to questions that might be expressed with a variety of keywords, and need to be 'understood' by the search engine before providing the answer."
No matter which search engine I use, still none of them can help me search for my missing car keys or missing left socks.
It must be Windows. It needs half a gig of RAM and a hardware-accelerated graphics card just to run Solitaire.
Ho yes, there's a good idea. Give Google the ability to understand.
Deleted
One thing missed in so many search engines now is finding information on a particular company quickly and efficiently. Often when I type a particular company into google or another search, I get a bunch of other hits before the actual company itself. Now, bigger companies come as the first hit often (i.e., apple, dell, canon, etc.) Try finding for that lesser-known company though and you'll encounter a lot of crap first. The company listing should always come first.
sig here
Google will come up with something to combat this, they always win.
Maybe there is already something like this that google offers?
Socks. Buy only one colour of sock. Black, white, whatever as long as they are all the same, I mean who really cares about socks anyway as long as they're comfortable.
Car keys is also solved. Get a beeper keyring (£3) for your car keys. They beep and flash when you whistle.
Deleted
...is here:
h tml
http://www.ic-arda.org/InfoExploit/aquaint/index.
...like the Semantic Web?
No, I don't know why it's being relaunched. My guess is that it's probably one of the answers that we are looking for in the age of search that didn't quite cut it. But isn't that what all these different meta-searches are talking about? The ability to get semantic meaning imbued into the web?
Mit der Dummheit kämpfen Götter selbst vergebens.
Google really lacks at filtering out noise. I was looking for Gran Turismo tuning stuff yesterday. Gran Turismo tuning -"release date" -cheats -faq, &c &c &c. The list of restrictions to filter out noise kept getting bigger and bigger, but it was still just the big agencies that were getting hits, nothing about the game itself.
Clusty on the other hand is no sucker for a press release. I find its much smarter at locating actual content.
Myren
Just try to google a website for these guys!
I know where I'd like to see this first: A digital librarian for Wikipedia. An agent that would recommened articles based on your preferences and maybe store the articles in some language neutral format where articles could be expressed into a target language or parsed from a language into neutral format. Too bad nobodies publicly demonstrated anything close to the level of machine intelligence that would be required to do it.
Shh.
Anyone else getting this message?
Slashdot requires you to wait 2 minutes between each successful posting of a comment to allow everyone a fair chance at posting a comment.
It's been 15 minutes since you last successfully posted a comment
What gives?
...she's the only one who does...
I for one would welcome our new search engine overloards, except google already runs my life, so it'd be a bit redundant.
I continue to welcome the mighty all-knowing google as the ruler of our lives.
"That's so plausible, I can't believe it!" - Leela
Than finding 50+ people asking the same question you are, and not a single answer. (or even only one person asking the question, but because the mailing list or newsgroup was being archived on more than one website, you find the same question over and over again).
It's even more annoying when you had the same question a couple of months before, and had found the answer, but can't remember what the answer was, where your found the answer, or what search terms you had used. (and it's even worse if that site has gone down in its rankings, and something else with people asking the question, but no answer, now ranked higher).
Build it, and they will come^Hplain.
for an 'intelligent' FAQ.
It uses more of a human based system, it 'learns' as folks type in different questions (and versions of the same question)and tell it whether the answers it gives are helpful. As uses 'teach' it, it gets better at providing relevent results to natural language queries.
Worth a look:
http://mindmeld.sourceforge.net/mmsf/index.php
What a surprise. http://www.ai.sri.com/search/
1. Google needs to find and kill keyword-filled spam/malware/whatever pages. Not just remove them from their search list, but murder the people who started them.
2. Google needs to filter responses based on ad content. If there are a ton of ads, chances are, the site is bupkiss and its priority should be massively downgraded.
3. Google needs to filter based on ownership by holding companies. These cybersquatters should be downgraded in response priority and their pets should be sterlized or neutered to control the pet population.
4. Google needs to get back in the kitchen and make me a pie.
===
Fix those things, then perhaps we should worry about statistical analysis... (but hey, thats just me)
IMHO, if you want accurate stats and information, go to a library...
MoM++ - A Classic Expanded - [Master of Magic 1.5]
http://mompp.sourceforge.net/
http://mindset.research.yahoo.com/
seems to be a good crap filter
Can you say Dogpile?
Google is the number one search agent for me as more often than not with a short list of carefully chosen starting terms, and a little refinement from sleuthing, I can find what I need pretty quickly.
Do the search engines have to be so smart they find what we meant to find or even what we think we meant to find as opposed to what we literally asked for? They're tools, like library cards, not servants there to do our work for us and stop us from thinking about the search process. Are we complaining because this all isn't as brainless as AOL?
If my grammar and spelling are off, I am [distracted/tired/careless] (take your pick)
I think that there will always be a need for knowledge specialists (professional researchers), whose job it is to develop useful results from requests for information, using whatever tools are at hand.
Tools like Google and MSN Search are not the only thing you need to find information. There are still places for other information, and 'because Google said so' is not a valid reason for accepting information as relevant, or factual.
Although these tools will continue to improve, the application of wisdom will still require human input to make the results useful.
InfoSec that matters, when it counts.
is the ability to filter out certain types of sites. Like the sites that are webinterfaces to Usenet or sites that sell stuff.
When I now look for a digital camera, I get hunderds of sites trying to sell me one, then a lot of sites that talk about it till I get to the makers homepage. an example The page I am looking for is this one
Vivisimo makes it a bit easier, but not completely.
A9 also failed to produce the correct page.
Don't fight for your country, if your country does not fight for you.
I for one welcome our new CIA/NSA designed search engine overlords (because they will get me otherwise).
Just think; In Soviet Russia the government searches you!
Get your Unix fortune now!
The stats aren't online, and I can't be arsed to go to the library or ring up someone for some help. Google suxx0rs!!!1!
Classical Liberalism: All your base are belong to you.
I think that something like ORA:CLE solution it's a good one, better than a search engine. And Google Answers it's a good approximation.
My city: Barcelona.
works surprisingly well. even better is their answer bar which is a simple download. i can alt click on any word in any program and i usually get the information that i am looking for. google usually gives me too much crap.
Data isn't information, information isn't knowledge, knowledge isn't wisdom.
If you run Linux, you have a decent tool-kit on hand to enhance search engine performance. Use lynx from the command line, with either the -source or -dump option, and pipe it through sed and such to filter it however you like. A recursive check of each link from the main page should get you most of the results, but you'd have to alter the formulae depending on which engine you use.
You could even put a Python script somewhere in the pipeline, which could sort the resulting links and keywords into a dictionary data structure, useful to save as a pickled object which can be recalled at leisure. Heck, once you have a source file in text form on your desktop, you have all those text tools to fiddle with. I'm sure others can come up with 100 more ideas.
In point of fact, the only thing requiring me to use search engines at all is the question, given that it would be simple enough to have a bot crawl the web for me while I sleep, of where would I *put* the data? But for small, specific applications, this manner could even work, and it could generate a list of links as bookmarks for you to try in the morning.
On the whole, I prefer that search engines *not* try to read my mind, because too often in the tech age, reading my mind changes to "making my mind up for me". I favor broad results which I can narrow in batch scripts, vs pre-narrowed results that reflect some corporate IDIOT's idea of what I'm supposed to find, but which will inevitably make what I want unobtainable.
yes but unlike skynet. Googles mantra is "Do no Evil"
Automation - The Car Company Tycoon Game
Understanding is seldom achieved by anyone. How do we model it ? At the very minimum "understanding" implies a consistant context. This means that the engine needs be tailored culturaly and demographicaly to be of use. The CIA/FBI/paranoid right wing fanatic context is probably fairly straight forward to model, one dimensional definitions with a few weighted variants. But what about real people asking questions not related to the investigative context ?
what the Symantec Web was all about?
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
If you have to ask, you won't get away with it.
The insanely talented and successful people I know with extensive mods would never think to ask. They simply wouldn't take a job that required them to change their appearance.
If that's not where you are in your career, well, suck it up.
Everyone else in this thread babbling about how "unprofessional" and "childish" mods are, well, they can suck it up too. There are people out there who are good enough to come to work every day with eye patch, a jester's cap, and a codpiece. It has nothing to do with the Internet bubble. They're always going to get away with it. It's a good bet that people who complain about them won't ever get away with it. Too bad.
yeah! i was using google to find a fish medicine for asthma - hyderabad website. but google did not return correctly the result. but yahoo did. google's page rank might move the real info pages to the back and may not be visible to the user. in that way, they are lacking.
balachandar muruganantham
Google is great and I use it frequently.
But lately often when I am curious about a subject, I look it up on Wikipedia. [ www.wikipedia.org ] the free online encyclopedia.
Works great, it's quick, fast and very informative.
your fish have asthma? Man, that sucks. I kid, i kid.
"Our interests are to see if we can't scale it up to something more exciting," he said.
I'll never understand how people can post to a completely different thread. Or is it a slashcode hiccup?
"Our interests are to see if we can't scale it up to something more exciting," he said.
bet you forgot about that bane of mindless link propogators!
The Aquaint program has been running for a few years now and before that there has been a question answering track held at the Text Retrievel Conferences where systems have to return exact answers to questions. In more recent years questions have also been list or definition based.
For those who are interested you can get info about the competing systems (academic papers some of which will contain links to demos of the systems) at the TREC website: http://trec.nist.gov/
The first evaluation to include question answering was TREC 8 so have a look there first as there is a nice overview paper discussing the task.
The linked article states,
"Recently, for example, I was trying to track the changes in California's spending on its schools. In the 1960's, when I was in public school there, the legend was that only Connecticut spent more per student than California did. Now, the legend is that only the likes of Louisiana and Mississippi spend less. Was either belief true? When I finally called an education expert on a Monday morning, she gave me the answer off the top of her head. (Answer: right in spirit, exaggerated in detail.) But that was only after I'd wasted what seemed like hours over the weekend with normal search tools. If it sounds easy, try using keyword searches to find consistent state-by-state data covering the last 40 years."
I know from my own similar searches that finding answers on a Web search engine is not the easiest type of search. One problem is that a Web search engine can only find information that has been prepared and posted (usually on the Web) by someone. I have come to the conclusion that most of the specific information that people want (at least, that I and James Fallows want) is not posted on the Web.
It is possible that no one has ever compiled a list that compares school funding across states for the last 40 years and posted that list to the Web. But, sometimes searching for that information could take a person close to finding the answer. By using Mr. Fallows' hypothetical search question (minor modifications to make it more general), I quickly arrived at "The National Center for Educational Statistics," which devotes itself to answering questions like that asked by Mr. Fallows. My link wasn't direct; I first was taken to an old paper, titled, "Education policy a complete failure: Report: Pumping more federal money into system won't help." http://christianparty.net/juliefoster.htm The article also mentions the National Taxpayers Union, which might be another source of the desired information.
Taking stuff apart since 1969 (TM)
Google vs Yahoo ! - otherwise called as "Engines of Discrimination"
Quidquid latine dictum sit, altum videtur
Should Your signature not be "Gegen die Dummheit..." instead of "Mit der Dummheit..."? Probably both is correct, but "Mit der Dummheit..." could lead to funny missunderstandings...
Frankly I think that searching and the results from searching, are limited by the searcher. If someone wants to search for something complex then they should learn to use the rules of the search engine. Using quotes properly (terms inside of quotes, i.e., "prime numbers"), or putting period marks at the end of words, they're simnple rules and can completely change the way you search. But it's not in the search engine's interests to help, as the more searching gets done, the more advertising can be delivered.
Honi soit qui mal y pense.
I would have to disagree with the poster - significant researching efforts have been put into question answering and factual data retrieval from the web. Visit the University of Massachusetts Center for Intelligent Information Retrieval website. For a more specific project, check out QuASM
Normally, I'm all for artificial intelligence, but think what kind of warped intelligence we'd get with Google. It'd have to conclude that to be human is to desire warez, "lesbians + dogs" and sex tapes of disgustingly skinny stupid rich girls. I wouldn't raise a kid on the 200 most recent Google searches, is all I'm saying.
Erik
YOU ARE SAYING IMPUDENCE TO ME! THAT IS IMPUDENCE!
http://www.google.com/search?hl=en&q=answer+to+lif e+the+universe+and+everything