Slashdot Mirror
Looking for Answers in the Age of Search
prostoalex writes "James Fallows, in a New York Times article, notices that search engines are getting pretty good at providing information for simple keyword-based queries. However, when it comes to the actual information, such as finding the necessary data and statistics, they're not doing a great job. The article talks about the NSA- and CIA-sponsored Aquaint project that aims to deliver answers to questions that might be expressed with a variety of keywords, and need to be 'understood' by the search engine before providing the answer."
...like the Semantic Web?
No, I don't know why it's being relaunched. My guess is that it's probably one of the answers that we are looking for in the age of search that didn't quite cut it. But isn't that what all these different meta-searches are talking about? The ability to get semantic meaning imbued into the web?
Mit der Dummheit kämpfen Götter selbst vergebens.
I know where I'd like to see this first: A digital librarian for Wikipedia. An agent that would recommened articles based on your preferences and maybe store the articles in some language neutral format where articles could be expressed into a target language or parsed from a language into neutral format. Too bad nobodies publicly demonstrated anything close to the level of machine intelligence that would be required to do it.
Shh.
I certainly know the problem you've describing.
Solution for me was to download the Firexfox "CustomizeGoogle" extension.
Once installed, the last tab allows you to enter regular expressions of sites to completely remove from displayed search results.
A little bit of config later and its goodbye "about.com", "go.com", "experts-exchange.com" and all the other similar "nothing to see here (unless you give us money)" sites.
http://mindset.research.yahoo.com/
seems to be a good crap filter
is the ability to filter out certain types of sites. Like the sites that are webinterfaces to Usenet or sites that sell stuff.
When I now look for a digital camera, I get hunderds of sites trying to sell me one, then a lot of sites that talk about it till I get to the makers homepage. an example The page I am looking for is this one
Vivisimo makes it a bit easier, but not completely.
A9 also failed to produce the correct page.
Don't fight for your country, if your country does not fight for you.
If you run Linux, you have a decent tool-kit on hand to enhance search engine performance. Use lynx from the command line, with either the -source or -dump option, and pipe it through sed and such to filter it however you like. A recursive check of each link from the main page should get you most of the results, but you'd have to alter the formulae depending on which engine you use.
You could even put a Python script somewhere in the pipeline, which could sort the resulting links and keywords into a dictionary data structure, useful to save as a pickled object which can be recalled at leisure. Heck, once you have a source file in text form on your desktop, you have all those text tools to fiddle with. I'm sure others can come up with 100 more ideas.
In point of fact, the only thing requiring me to use search engines at all is the question, given that it would be simple enough to have a bot crawl the web for me while I sleep, of where would I *put* the data? But for small, specific applications, this manner could even work, and it could generate a list of links as bookmarks for you to try in the morning.
On the whole, I prefer that search engines *not* try to read my mind, because too often in the tech age, reading my mind changes to "making my mind up for me". I favor broad results which I can narrow in batch scripts, vs pre-narrowed results that reflect some corporate IDIOT's idea of what I'm supposed to find, but which will inevitably make what I want unobtainable.