Better Search Engines

prostoalex writes "Scientific American is seeking better Web searches. They report on all sorts of innovations happening outside the Google-Yahoo-MSN zone that the press is usually reporting on, including GPS-enhanced searches from University of Maryland, Shape Retrieval and Analysis from Princeton, musical search engine from New Zealand Digital Library Project, and some of the projects that A9 and Ask.com have been working on."

Comments

What we need is whitelisting

[Jailbrekr]

If we can whitelist sites, and reduce the total number of advertisments cluttering the search, the existing search algorithms would work quite nicely.

It is a pipe dream, I know. :(

Re:What we need is whitelisting

[Anonymous Coward]

lol, what? Seriously? Whitelist the web? Huh? Come again? What?

Re:What we need is whitelisting

[vbdrummer0]

If we can whitelist sites, and reduce the total number of advertisments cluttering the search, the existing search algorithms would work quite nicely.

I agree, but why not just eliminate all ads from search results? As far as I'm concerned, they can put real ads all over the result page as long as the results themselves are legit.

Re:What we need is whitelisting

[prostoalex]

Theoretically Google should be able to put that into their toolbar. Also you could probably use an extension in Firefox that would modify your query to exclude certain sites. After the list gets long, however, it wouldn't be too effective.

Re:What we need is whitelisting

[eln]

Kind of like a directory of sites like, say, Yahoo was in the 90's.

The problem with whitelisting is that a spider-driven site like Google will always end up having a greater quantity of relevant results (as well as a greater quantity of non-relevant results, of course). History to this point has shown that people prefer to deal with a lot of bad results mixed in with a lot of good results rather than having to rely on a small set of "good" results from a directory-driven search engine.

Re:What we need is whitelisting

[iminplaya]

This is kinda close.

Re:What we need is whitelisting

[Antonymous Flower]

I think advanced searches that allow or disallow sites with advertising on them could be an interesting option. If not wholy useful from the beginning (I'm not saying it wouldn't be, though,) it could have a positive effect on the way webmasters design sites in the future.

Re:What we need is whitelisting

[JPriest]

This is sort of the aim of the google pagerank feature currently in the google toolbar. Some people think the feature is intrusive though.

Re:What we need is whitelisting

[DeathFlame]

Make it similiar to adblock where you can use notation to exclude a wide number of things with only a few listings.

Re:What we need is whitelisting

[Elwood P Dowd]

He's talking about search result spam, not ads on search engines (which you can ignore).

Re:What we need is whitelisting

[zonker]

this brings up another issue that i was thinking about. i hope all of these companies are also thinking of ways they can make money from their search engines while they are dreaming up their new technology or it will be for naught...

Re:What we need is whitelisting

[Anonymous Coward]

Ads exist just because search engines are managed by some companies that need money.

Imagine now that the search engine is totally distributed...

Re:What we need is whitelisting

[Anonymous Coward]

*.go.com

Re:What we need is whitelisting

[Anonymous Coward]

...or managed by someone who is independently wealthy.

Re:What we need is whitelisting

[Lord_Dweomer]

I can't imagine the people who pay for those placements would want search engine spam appearing on the tops of their pages that they've paid to get people directed to.

Re:What we need is whitelisting

[Mornelithe]

Companies like the University of Maryland and Princeton?

Re:What we need is whitelisting

[zonker]

those two aren't the only ones working on these types of technologies. btw, yahoo and google were first projects that started in stanford...

and if you want to get into semantics, yes, colleges and universities are companies. they just don't like to admit it.

Re:What we need is whitelisting

[Anonymous Coward]

I'd have to disagree, the company I work for makes a nice profit selling custom build "whitelist" search engines to major corporates. Typically only searching over 50-100 sites in depth, or a few thousand for breaking news.

Re:What we need is whitelisting

[Anonymous Coward]

Try: The Google Directory http://www.google.com/dirhp.

The data is from the Open Directory Project http://dmoz.org/ an almost entirely volunteer-run project http://dmoz.org/about.html. I suggest using the Google version because, for most people, its search facility is better than the ODP search, due to the fact that it works like most Google users would expect a search to work.

The actual directory is variable in quality - some of it is very, very good indeed. However, it suffers from the normal problem that many volunteer-run projects have: parts it are neglected, and rather out of date. Always worth a look though.

Re:What we need is whitelisting

[Anonymous Coward]

> Kind of like a directory of sites like, say, Yahoo was in the 90's.

DMOZ

Music Search

[fembots]

a user can record a query by playing notes on the system's virtual keyboard. Or he or she can hum the song into a computer microphone.

I tried that, but I was so out-of-tune the search engine returned all songs from Britney Spears.

Re:Music Search

[k4_pacific]

I tried it too, but all the results were blocked for DMCA violations.

Re:Music Search

[YU Nicks NE Way]

Yeah, but think of what it would do for the RIAA. They'd have soooo many opportunities to send out threatening letters!

Re:Music Search

[pronobozo]

I lipsync'd and came up with Ashley Simpson. incredible!!

More like...

[Anonymous Coward]

ashley simpson am i rite? lawl

What I want

[Anonymous Coward]

Is just some better work done on recognizing essentially similar documents. Like, if I perform a search, and 40% of the returns are the same wikipedia article copied to different sites, it would be nice if the search engine could only show me one (wikipedia). Or, like, if I'm searching for some kind of error I got while using Linux. Most of the returns I get will be various old Linux mailing lists, but only some of them will be relevant to my problem. There must be some way the search engine could logically organize them for me so that I could more clearly identify that block of returns that is most applicable to my problem of the moment.

Re:What I want

[JPriest]

What you can do in that case is take a random quote from the article that is distinct and use a -"exact unique text here"

Re:What I want

[me at werk]

CopyScape can do the recognizing of copied stuff, but it's purpose is only finding website plagarism. This, however, would definately find all the wikipedia forks unless it's a really old copy and the page has had a major rewrite.

If google could integrate copyscape into their search, you would be happy.

Re:What I want

[Anonymous Coward]

What I want is a button that lets me resrtict my search for a thing to either a review of the thing, a forum/blog discussing the thing, places to buy the thing, or specs/datasheets on the thing. So many times I type in a product name only to get two dozen "find prices/read reviews on X" -- none of which actually have reviews ("be the first to review X!") or even more than a couple of not-so-great prices. A filter could be done by creating a statistical fingerprint of the page.

I also want to be able to sort my search based on the amount of grammatically correct (or mostly correct) text on the page. Something would have to keep it from indexing hidden (white on white) words or keywords designed to grab top spots on search engines. There would have to be some more complex grammar checker (and place checker to make sure it isn't a bunch of tiny text at the bottom of the page) to accomplish this.

Re:What I want

[susano_otter]

And if it was an old fork, and a major rewrite, I'd appreciate the opportunity to compare the two versions. Also, if the algorithm didn't eliminate one of the two results in this scenario, then by definition they're two separate results, each containing unique information that the other does not have. Giving you one copy of the old Wiki and one copy of the new Wiki seems like exactly what such an algorithm should do, in this scenario.

Re:What I want

[JesusQuintana]

"Like, if I perform a search, and 40% of the returns are the same wikipedia article copied to different sites, it would be nice if the search engine could only show me one (wikipedia)."

Like, I agree. I have done some searches and simply find the same text on page after page. It would be nice if the search engine could provide some sort of heirarchy. It could say here is the authoritative source and here are all the sources that qoute it.

I did say it would be nice, but it really isn't necessary, or it would seem very feasible. What if the authoritative source changed, but the subordinate sources are not updated? It would seem that this would apply to your Wikipedia example. And how could an algorithm determine the parent source?

Yes, it would be nice. But better search is also a detriment. If we are an information economy, as the talking heads on television keep telling us, isn't part of our value as information workers our ability to deal with this information. In the old card catalog days, wasn't a researcher's job to take all of the information that is available and gather and interpret that information in a meaningful way? Isn't that still true of today's information worker? If your CEO can type something into Google and get the answer he wants, do they need you to find the answers for him? If software becomes too intelligent, then doesn't the human mind become obsolete. Frankly, I've always been proud of my ability to parse and understand information and then recapitulate it in a meaningful way.

"Or, like, if I'm searching for some kind of error I got while using Linux. Most of the returns I get will be various old Linux mailing lists, but only some of them will be relevant to my problem."

Well, you've probably got several problems here:

1. Lack of relevant material due to smaller installed user base. (It will always be easier to find the answer to the Windows problem because more people are having it and writing about it.)
2. Lack of need to publish material. Linux users are generally not computer illiterate so there isn't much need to write thousands of Linux hand holding articles.
3. Lack of people searching for and linking to material. This makes Google's relvance ranking somewhat useless.

Of course, all of this could be attributed to poor search criteria. As they say, "garbage in, garbage out." And that takes me back to my previous point. If your CEO can fix his computer with Google, why do they need you?

In the information economy, knowledge is power and money. And if knowledge is easily obtained, then the laws of supply and demand dictate that the value of knowledge decreases. As a freelance video producer, I am watching the devaluation of my services occur as practically every Joe Schmoe can edit video on their home computer. The latest technological innovations have devalued the video/film production industry. The only people still making lots of money are the stars, because the work proactively to protect their value. I don't know why technology/information workers aren't interested in the same. If I buy a faster computer that enables me to produce videos in half the time, then I have also cut my billable hours in half as well.

So, I am all in favor of poor search technology and lazy people. Simply, if I have the patience and the ability to skillfully use technology in ways that others cannot, then I am a valuable commodity. But if anyone can do it, then I'm just like an 18 year old who wants to work for MTV. That's why MTV staffs with interns, because there is an endless supply. If the supply is endless, then you have no value. You're a dime a dozen. You might as well be Chinese.

Re:What I want

[monsterhead78]

What the f*** is this supposed to mean? You have problem with the Chinese?

Re:What I want

[JesusQuintana]

Not at all. I was simply pointing out, although not very clearly, that the Chinese labor market is very cheap and many trans-national corporations are moving manufacturing opertations there. Why? Because there are a lot of Chinese people. And if you combine them with the Indian labor market, people in the Asian labor market are a dime a dozen. Big corporate America can literally work Chinese employess to death. Because there will be more to replace them. Labor in China is cheap for many reasons, but certainly one of them is supply and demand. There is a ready supply of Chinese workers. And because the demand for work exceeds the number of jobs, it is an employer's market. Large corporations can drive down even further the cost of labor. Certainly, global economics are much more complicated than what I am saying here and the Chinese economy is still evolving (arguably the U.S. economy is still evolving, or rather devolving), but I was simply saying that the Western world (not just the U.S.) is losing vast segments of its manufacturing economy because a seemingly endless supply of low-cost Chinese workers. Companies don't place much value on Chinese workers. They just see them as warm bodies. It could be argued that productivity increases for the information worker mean the devaluation of the worker, particularly if less and less skill are required. If google can do your job, then the information worker becomes a warm body that big business no longer needs to pay.

And I don't think that the Chinese deserve their standard of living or lack of value Transnational corporations place on them, but I always try to take care of myself first. So first and foremost, I worry most about my value as an employee.

And I guess I didn't actually say any of this, I merely alluded to it. And please forgive me for any U.S.-centric views. I am merely a product of my horrible environment.

Re:What I want

[adamruck]

1) Try using line breaks.

2) Stop trying to defend what is really racism (there are Chinese people who have lived there whole lives in America and I am sure would not like to be paid less just becuase of their race) with economics.

Re:What I want

[JesusQuintana]

1. Don't go around calling people racists when they are simply trying to have an intellectual discussion pointing out technology's undeniable influence on world economics.
2. When I spoke of the Chinese I was referring to people who live in China, not people of Chinese ethnicity. So I am not a racist. I was not referring to Chinese-Americans. Again, it's not race, it's geography. And if you read carefully enough, you'd realize I was against large, faceless trans-national corporations underpaying anyone.

And I do not think that people of different races should recieve different pay scales in the same labor market. But it is undeniable fact that people in China make less money than those in the U.S. (again I'm talking about geography and not ethnicity) And by people, I mean people of all colors. If I were allowed to work in China (I can't because I am a U.S. citizen) I'd enjoy a lower standard of living than working in the U.S. even if I were of Chinese ethnicity.

I may be a U.S. citizen but how do you know that I am not myself Chinese-American?

Re:What I want

[acebone]

1. Try reading what you comment

2. Try reading what you comment

Re:What I want

[Anonymous Coward]

but it's purpose

"its".

Re:What I want

[HugeFatty]

I agree that the things you have listed are problems, and that they'd sure be nice to solve. I just wanted to address one of them for now, as I have been trying to deal with it myself.

The hidden text problem that you mention is a surprisingly hard problem to deal with, as there are so many ways to do it.

You have:

• The <font> tag
• CSS (several ways, such as the :hidden property, changing the colors, using the z order, etc.), both internal and externally linked (for which the search engine must download that file while spidering)
• DHTML positioning over other elements
• A background image the same color as the text
• Javascript to generate any of the above
• Use of nearly identical colors for all of the above (such as #FFFFFF for the background and #FFFFFE for the foreground). In fact, there could be dozens of colors that are all slightly different enough that a human wouldn't be able to detect it without looking very closely, or at all.

I'm sure there are more that I'm missing, but I think you (meaning everyone...I'm not just picking on the parent here...) get the idea. You pretty much have to render the page like a browser to take care of all of those, which really sucks for us search engine developers trying to fight it, and us users that have to deal with that crap.

Re:What I want

[me at werk]

You seem to have missed the history page. You can see past versions of the page, and usually comments give hints as to when it had a major rewrite.

Re:What I want

[susano_otter]

I couldn't care less about the Wikipedia "history page", in this context. We're discussing what a good search algorithm should do, in the case that it gets two different Wiki pages, with different content, on the same topic.

My conclusion is that the search algorithm should return both pages in its search results, not only making me aware of the change over time, but giving me a quick and easy way to compare and contrast the two versions.

Obviously, if I was restricting my search to the canonical Wiki of the day, I'd simply lookup the topic in the Wiki itself, and make use of the "history page" you speak of to learn about changes.

Ask.com!

[joshsnow]

and some of the projects that A9 and Ask.com have been working on

I want a search engine with a Genie-Jeeves. Imagine: I snap my fingers, smoke streams from my monitor, materialising into Jeeves, complete with tray, glass and a bottle of that beer I couldn't quite bring to mind when I clicked the search button...

Re:Ask.com!

[Anonymous Coward]

I used to work for Ask Jeeves. About 5 years ago they considered launching an adult-themed (aka porn) search site. They came up with the concept of a blonde in a skimpy French maids outfit and went so far as to register the domain askmimi.com. Unfortunately they higher-ups eventually decided that they didn't want to potentially damage the Jeeves character by having it associated with a porn search site so they canned the whole thing. Personally I thought Mimi would have done quite well...

Re:Ask.com!

[Anonymous Coward]

I found your story a bit surprising, but I asked Jeeves himself, and he came up with this article, among many others. Funny.

Re:Ask.com!

[Webmoth]

I snap my fingers, smoke streams from my monitor, materialising into Jeeves...

I once snapped my fingers and smoke streamed from my monitor. Unfortunately, it didn't materialize into Jeeves, and my monitor never worked after that.

It's not cuz you were out of tune

[Anonymous Coward]

It's because all pop music sounds exactly the same these days. From the perspective of a search engine, it *IS* all just songs from Britney Spears.

Like Yahoo, Only Cheaper

[mstyne]

From The Daily WTF:

I want a website directory, like a yellow pages, or Yahoo. I want any web user to be able to add a link, under the relevant categories available, like...finance,real estate,travel,games etc. I would like the links to be approved before they appear. I want the search results displayed in the following fashion: A URL text, or URL image, with a little description underneath. I want the following tools - top 50 searches, most popular links, a search facility. A space across the top of the page to insert my own logo.

metadata

[subrama6]

as we get into video search and the like, aren't searches dependent on the quality of the metadata associated with the item? i just tried video.google.com, and was impressed that typing in "bauer" got me stills from recent episodes of 24. but surely that's based solely on the fact that "bauer" was a tag for the still. at that point, why is new search technology impressive? it's the metadata that makes it possible. am i missing something?

Re:metadata

[Anonymous Coward]

Yes. You're missing the point that no one wants to metatag their data in any meaningful way. They just want the search engine to 'know' what it is.

Of course, when you get into it, your definition of meaningful and my definition of meaningful and a company's definition of meaningful are entirely different.

In the end, we simply need full blown AI to solve this problem.

Re:metadata

[Unordained]

Most people are too lazy to enter good metadata in webpages, even assuming a good, standard way of doing so. However I see some future in sites like GlobalSpec where your search is narrowed down until you know exactly what type of product you're looking for, and then extra metadata for that class of products is exposed in the search (usually in the form of sets of checkboxes and ranges of metric-aware values) -- allowing you to search by thread size on bolts, but not on other equipment where it's not relevant. I think some sort of community-driven version of this could be nice; let users create new types of attributes (or whole modules) for new types of 'items' and search using better metadata. Eh. It's an idea.

Clusty = Innovative

[int2str]

Asides from the horrible name, clusty (a clustering search engine) is very innovative and easy to use. I hope more search engines will adapt similar technology soon.

Link to clusty.com search engine

Re:Clusty = Innovative

[myukew]

It can't be no good, looking for martin ist jesus (martin-ist-jesus.de.vu see sig) found my own site as first hit! Come on, my site sucks so much it mustn't be nr 1!

Re:Clusty = Innovative

[DeathFlame]

That does a very good job of organising links. I really like the cluster by topics feature.

Re:Clusty = Innovative

[dankelley]

+ Aside from the fact that sponsored links take up 1/2 the search-result window sized for a typical laptop.

Re:Clusty = Innovative

[lucabrasi999]

Agreed. I've been using Vivisimo/Clusty for over a year now (I prefer to use Vivisimo, since the "clusty" name sucks). I think it is a great search engine.

Re:Clusty = Innovative

[lucabrasi999]

Use vivisimo instead of clusty. It is the same search engine/company, just different names. If you search use Vivisimo, the sponsored links aren't quite as obnoxious. Unfortunately, the firefox extension uses Clusty, not Vivisimo.

As for the names, both of the suck big-time. "Vivisimo" and "Clusty". Geez. I remember a few years ago, Price Waterhouse Coopers Consulting decided to change their name to "Monday". I wonder if the folks at Vivisimo hired anyone from PWCC, because their names suck almost as much.

Re:Clusty = Innovative

[Anonymous Coward]

wow, you're right! your site does suck! ;p

Re:Clusty = Innovative

[dapyx]

Vivisimo has not Wikipedia search, like Clusty.

Re:Clusty = Innovative

[myukew]

see what i mean... better use google

Re:Clusty = Innovative

[myukew]

he uses pascal you insensitive clod!

GPS-enabled search

[jxyama]

GPS-enabled search would be excellent, as more and more people probably will adopt accessing the web on their cell phones. (already happening in japan, afaik.)

Musical search already exists...

[Humorously_Inept]

It has been available as a service on mobile phones for something on the order of two years. The same thing, called TuneTracker, is available in Canada now under the MuchMusic brand. Put your phone up to the mystery tune and you'll get the song title and artist's name back in an SMS message.

I'd like to see a search engine that can intelligently filter results for the word "review." When I search for a product review, I do not want some hole-in-the-net online store's product page with a link to 0 customer-submitted reviews.

Re:Musical search already exists...

[Anonymous Coward]

the meldex project has been around for about 6 or 7 years, I think. Also, it can match user input to the song in the database.

Some of those other services (like the one you do) actually work completely differently - they create a fingerprint of an audio signal, and require the exact same recording to get the same fingerprint - it won't work for a cover version or re-recording of the song. And that sort of service isn't really that useful - most of the time that you have access to the original recording, you normally have access to the song metadata as well :)

Um, dude.

[Anonymous Coward]

You just described both dmoz and del.icio.us. And since both offer web services interfaces, you could easily create your own frontend to either.

Re:Um, dude.

[Anonymous Coward]

and both of which suck horribly - especially both of the interfaces

they suck

suck sucksuck!

Re:Um, dude.

[Anonymous Coward]

Here's instructions on how to draw information from del.icio.us

Here's perl

Write your own interface, bitch.

Re:Um, dude.

[orangesquid]

Speaking of interfaces, I have oodles of data ganked off of freenet. Who wants to make an interface to searching this?

I'd host it, but, (a) my server would come after my neck, (b) my ISP would come after my neck, and (c) the FBI could come after my neck.

Maybe I should design a special javascript tool that does lots of cross-referencing, or something, and put many overlapping data sets up on a freesite? Drawbacks to that are the sheer amount of data and the lack of persistence for uncommon search terms...

Yeah, but when will...

[Anonymous Coward]

www.findmysocks.com by up and running?

Re:Yeah, but when will...

[darkpixel2k]

You posted the link to slashdot. No. No it won't.

Re:Yeah, but when will...

[Lord_Dweomer]

As soon as RFID tags become entirely mainstream and dirt cheap. You joke, but I'm being serious.

All together now...

[Tackhead]

> GPS-enhanced searches from University of Maryland, Shape Retrieval and Analysis from Princeton, musical search engine from New Zealand Digital Library Project, and some of the projects that A9 and Ask.com

Jeeves, what sort of music would you recommend to my friends if I told you I was listening to a sculptor singing the plaintext of the curvy-shaped thingy that talks about 38 57' 6.5" N 77 8' 44" W to the tune of The Hymn of the Soviet Union?

Re:All together now...

[Anonymous Coward]

The day Jeeves could come back with:

"Something spooky"

Is the day I start wondering if he's intelligent and enough of a smart ass to pass the Turing test.

(Check the lat/long)

Re:All together now...

[Anonymous Coward]

Why don't you try it:

http://web.ask.com/redir?u=http%3a%2f%2ftm.wc.as k. com%2fr%3ft%3dan%26s%3dj4%26uid%3d0F17C32A9809D6F1 4%26sid%3d1685142A9809D6F14%26qid%3dFE74F009D25A96 48BE5BE7550CD19518%26io%3d0%26sv%3dz6f5372cf%26o%3 d0%26ask%3dwhat%2bsort%2bof%2bmusic%2bwould%2byou% 2brecommend%2bto%2bmy%2bfriends%2bif%2bI%2btold%2b you%2bI%2bwas%2blistening%2bto%2ba%2bsculptor%2bsi nging%2bthe%2bplaintext%2bof%2bthe%2bcurvy-shaped% 2bthingy%2bthat%2btalks%2babout%2b38%2b57'%2b6.5%2 522%2bN%2b77%2b8'%2b44%2522%2bW%2bto%2bthe%2btune% 2bof%2bThe%2bHymn%2bof%2bthe%2bSoviet%2bUnion%253f %26uip%3d432a69d5%26en%3dte%26eo%3d-100%26pt%3dMus ic%2bof%2bSan%2bFrancisco%2b-%2bAmzp%2540SiteRank% 26ac%3d12%26qs%3d0%26pg%3d1%26ep%3d1%26te_par%3d15 2%26te_id%3d%26u%3dhttp%3a%2f%2famzp.siterank.org% 2fus%2fcat%2fmusic%2f1100107808%2f3%2f&bpg=http%3a %2f%2fweb.ask.com%2fweb%3fq%3dwhat%2bsort%2bof%2bm usic%2bwould%2byou%2brecommend%2bto%2bmy%2bfriends %2bif%2bI%2btold%2byou%2bI%2bwas%2blistening%2bto% 2ba%2bsculptor%2bsinging%2bthe%2bplaintext%2bof%2b the%2bcurvy-shaped%2bthingy%2bthat%2btalks%2babout %2b38%2b57'%2b6.5%2522%2bN%2b77%2b8'%2b44%2522%2bW %2bto%2bthe%2btune%2bof%2bThe%2bHymn%2bof%2bthe%2b Soviet%2bUnion%253f%26o%3d0%26page%3d1&q=what+sort +of+music+would+you+recommend+to+my+friends+if+I+t old+you+I+was+listening+to+a+sculptor+singing+the+ plaintext+of+the+curvy-shaped+thingy+that+talks+ab out+38+57'+6.5%22+N+77+8'+44%22+W+to+the+tune+of+T he+Hymn+of+the+Soviet+Union%3f&s=j4&bu=http%3a%2f% 2famzp.siterank.org%2fus%2fcat%2fmusic%2f110010780 8%2f3%2f&qte=0&o=0&abs=It's+the+one+I+recommend+to +friends+who+haven't+listened+to+AMC+before.+...+a lbum+to+listen+to+-+very+good+music.++Early+60s+ro ck%2c+with+a+sort+of...&tit=Music+of+San+Francisco +-+Amzp%40SiteRank&bin=&cat=wp&purl=http%3a%2f%2ft m.wc.ask.com%2fi%2fb.html%3ft%3dan%26s%3dj4%26uid% 3d0F17C32A9809D6F14%26sid%3d1685142A9809D6F14%26qi d%3dFE74F009D25A9648BE5BE7550CD19518%26io%3d%26sv% 3dz6f5372cf%26o%3d0%26ask%3dwhat%2bsort%2bof%2bmus ic%2bwould%2byou%2brecommend%2bto%2bmy%2bfriends%2 bif%2bI%2btold%2byou%2bI%2bwas%2blistening%2bto%2b a%2bsculptor%2bsinging%2bthe%2bplaintext%2bof%2bth e%2bcurvy-shaped%2bthingy%2bthat%2btalks%2babout%2 b38%2b57%2527%2b6.5%2522%2bN%2b77%2b8%2527%2b44%25 22%2bW%2bto%2bthe%2btune%2bof%2bThe%2bHymn%2bof%2b the%2bSoviet%2bUnion%253f%26uip%3d432a69d5%26en%3d bm%26eo%3d-100%26pt%3d%26ac%3d24%26qs%3d0%26pg%3d1 %26u%3dhttp%3a%2f%2fmyjeeves.ask.com%2faction%2fsn ip&Complete=1

it's spooky, jeeves does have an answer

Re:All together now...

[Anonymous Coward]

Obviously everyone's forgotten the Kryptos trifecta (Wired, fark, /. bringing up the rear) stories this week. Or NSA's modding this guy down because w^Hthey already know the answer.

The BBC Search Engine

[Anonymous Coward]

Personally I use the BBC Search engine. Not only does it seem to provide relivant results, it also has recomended links (info here http://www.bbc.co.uk/search/recommended.shtml ) which are editorially selected.

The site seems to return far less porn probably due to the fact they "use a combination of technology and regular human checks to detect and block offensive websites. We aim to be the safest search engine in the UK"

Also slashdot is the first return for "IT News" under the web tag.

http://www.bbc.co.uk/cgi-bin/search/results.pl?g o. x=&tab=www&go.y=&go=go&q=IT%20news

Re:The BBC Search Engine

[Anonymous Coward]

A search engine with the BBC bias is useless. Just try any (even slightly) politically incorrect term in it and your jaw will drop when you see the results.

I'm not kidding, go ahead, type something that the minitruth doesn't like... such as a politically incorrect person or party, and see how far they go in spinning what you see.

Re:The BBC Search Engine

[queef_latina]

Of course, in 2005 'political correctness' is flag waving, and yellow ribbons on cars, and 'red and blue should "work together."'

So you're saying that you can't find information in the BBC engine about how poorly the iraq war is going, or anything unfavorable regarding the bush administration?

In that case you aren't using it correctly; I found all these things and more :)

Re:The BBC Search Engine

[acebone]

You found that ? Well - why not link to it ?

Slashdot needs a 'smug' or even a 'too smug' moderator option

Need better division of info being searched.

[Anonymous Coward]

I need a search engine that lets me search for information vs products vs forum posts vs whatever else is on the internet.

I get frustrated when I'm trying to research a new technology and most of the search results are for commerce sites.

Re:Need better division of info being searched.

[timeflux]

more than that! we need a search engine for the search engines. sure better tech brings better opts.

Re:Need better division of info being searched.

[Anonymous Coward]

Queryserver has been clustering results from various search engines for years. It takes a little longer to search, but I find the results are typically better than Google for specialized topics.

Pinpointing the Nearest Restaurant

[Inhibit]

From the article "letting your wireless PDA, for instance, pinpoint the nearest restaurant"

Even if there isn't some kind of Windows software to do this with GPS, I can do it right now by punching in a few digits on mapquest.

Sure, it's restricted to there sponsors or somesuch. But I don't see this as being any different.

Re:Pinpointing the Nearest Restaurant

[Anonymous Coward]

Well I can do it here in Tokyo, no problem, where is nearest restaurant? Where is nearest bar? When does the subway go there, do I turn right here to find the substation?

It answers all this, in two clicks. (But you need to learn to read kanji first of course)

Better Search techniques

[rueger]

Nice article which summarized many of the problems with contemporary search engines.

My experience is that a few years ago you could type say "baked gorgonzola" into Google and be sure to get a useful result pretty near the top. These days though what you want is likely to be on page three or four, after a dozen links to price comparison sites.

There really is no such thing as a quick Google search any more. It almost invariably involves multiple formulations of your query, and probably trolling through at least two or three pages of results.

Whether that's because of Google, or the sheer volume of content on the web, or sites that capitalize on Goggle's weaknesses is something I don't know.

Re:Better Search techniques

[jumpingfred]

Strangely the first few sites seem to be recipies for baked gorgonzola. What were you looking for instead?

Re:Better Search techniques

[Anonymous Coward]

Erm, the first result is the recipe for that. Isn't that a "useful result"?

Re:Better Search techniques

[KingBahamut]

It would seem to me, if capable, for an engine as large as google.com , to include a localization element to its searches. That way someone like me, searching for linux developers in Atlanta, GA to commune with, wont get results for developers in East Asia. =)

Re:Better Search techniques

[rueger]

Ok Bad example. Try searching Google for information on say a Sony STR-DE945 reciever and see how far you need to look to find anything beyond retail. Like maybe for a page from the Sony website?

Or try to find a User Maunal for the same item: sony STR-DE945 receiver manual.

Re:Better Search techniques

[gkuz]

I've found it helps to spell correctly.

Re:Better Search techniques

[Anonymous Coward]

How about this?

Re:Better Search techniques

[susano_otter]

Why would you Google for the user manual, instead of just going straight to Sony's website?

Google's rankings are based in part on what other people care about. The results you're seeing are because people are more interested in finding and using websites where they can buy the product, rather than the manufacturer's official brochure page for the product. And since that page is trivial to find, if you really do need it, it would end up being noise on most Google searches for the product.

When I need a manual for a server in my datacenter, I don't go to Google. I go straight to the vendor's website. Works every time.

Re:Better Search techniques

[KingBahamut]

That seems likely, but more likely to just work it into the overall engine. Why bother with trying to make it seperate, as most of us admin type know, end users are 1d10t3n.

Re:Better Search techniques

[Anonymous Coward]

I don't know. I just tried "Intel DK440LX", an old P2 dual cpu motherboard. Intel's tech support site was at the top of the list. The places still selling them (!) were further down. I see your point -- I too have had similar search results to what you describe, for other things -- but it obviously depends upon what is being searched.

Re:Better Search techniques

[HyperCash]

Because Sony's website is an incredibly difficult to navigate almost impossible to use steaming pile of shit?

--HC

Re:Better Search techniques

[Anonymous Coward]

I think you've still got a bad example. From all appearances, that product is no longer part of Sony's current product line. A search at Sony's site returns nothing. And the only page I was able to find about that product on Sony's site involved submitting a form to a CGI program. Even then, the page was an all but useless FAQ page with only 3 entries.

Blame Sony for their lack of an indexable page rather than Google.

Re:Better Search techniques

[ashot]

I'm not sure how much truth there is to it, but my understanding is that grandparent is saying that those results are there because the sites have found ways of optimizing for the search (link farms, etc)

Re:Better Search techniques

[susano_otter]

Maybe that's why nobody Google cares about links to the damn thing, and why Google itself has a hard time indexing it in any useful way:

Maybe the problem has nothing at all to do with Google's alleged limitations, and everything to do with asshats building websites that are unsearchable for both the best search engines currently available and for the human mind itself.

ashlee simpson

[Anonymous Coward]

i tried searching for one ashlee simpson's songs, but it didnt pick up on my lip syncing

Vivisimo

[Dan667]

Interesting, the first thing I thought is I had seen this with Vivisimo, but I guess no one could spell that so the changed the name?

http://vivisimo.com/

But I agree, it is a great search engine and has gotten better as I have used it.

Sure, sure...

[susano_otter]

And the moment any one of these other technologies becomes at all useful, except in certain limited applications, the technology will be acquired by one of the search engines that everybody actually cares about (coughGooglecough), and the functionality will be added to their Internet search solution.

"Yes but"

[Anonymous Coward]

I don't want to have to do that myself, that's the exact sort of thing that would be relatively easy to integrate into the interface of a search engine (like a little X next to the link, and you click it and it chooses a phrase from the link and auto-adds it as a -"quote" modifier to the search) but a bit more annoying for me to do myself. Anyway, if I did it myself I might not choose a particularly good quote to add into the -"" there. But you know who would be very much able to choose a good quote to add there? Google. I mean, that's basically what they do, they categorize pages based on sequences of characters used (words). All they'd have to do is start categorizing pages based on sequences of words as well.

I want...

[Lil-Bondy]

I want a big house, I want lots of money, I would like for everyone to bow to me as their supreme ruler, I want the girls to like me, I want super powers... You See, We Never Get What We Want...

Synonyms

[e4ward]

Odd, just before reading this story I wanted to google for:

searchterm1 searchterm2 bogus

and how I would have liked the search engine to actually search for:

searchterm1 searchterm2 (bogus OR fake OR spurious OR wrong OR specious OR ...etc)

by being able to specify a qualifier on bogus eg, bogus:synonyms

Re:Synonyms

[mE123]

searchterm1 searchterm2 ~bogus

~ is the google synonym operator. To play with it try ~word -word, so you only get the synonyms.

Ah, google is there anything it can't do?

Re:Synonyms

[Anonymous Coward]

Ah, google is there anything it can't do?

screen out the garbage sites that tell me they will show me prices and reviews, but have no actual content.

Shape Retrieval and Analysis? Hmmmm...

[R2.0]

Does that mean I will be able to search for porn with 38DD's?

Did I say that out loud?

It's available!

[ByteMangler_242]

You can do this in google: searchterm1 searchterm2 ~bogus The tilde will look for synonyms. You can see which ones hit back by reading the bold results which are neither searchterm1 or searchterm2. I use ~howto and ~cheats often.

This is the bomb shit

[Anonymous Coward]

liquidinformation.org

http://wired.com/news/culture/0,1284,66382,00.ht ml ?tw=wn_tophead_2

better to search information, not pages

[AnonymousCactus]

Enhancements to normal search engines are great and will always be important, but better is to go beyond that to searching, indexing and retrieving actual information. Services like AskJeeves and company originally promised true question answering and other, more experimental, projects like UW's Know-It-All promise to operate over information, not webpages.

Perhaps these are just very generalized search engine enhancement...but I think it's a new way of thinking that will become very important over the next decade as facilitating technologies mature.

Google is failing me

[zymano]

I have been using it less and less.

Trying others. Google is way too spammed with commercial sites in their finds.

Their technology is very low tech to me.

Re:Google is failing me

[Anonymous Coward]

yeah, i need tons of blinking stuff by the link that i really want to use, make the initial page like 4 meg with a bunch of useless graphics and make the start button look like a puffy ballon, that would be kewl.

An interesting color image search

[jbum]

I've been playing with Flickr, since it was mentioned on Slashdot a few days ago.

Flickr has a really nice API for retrieving images. I used Perl and ImageMagick to build a database that provides this amusing tool for searching images by color:

http://www.krazydad.com/squaredcircle/colorfinde r. php

And a related project:

http://www.krazydad.com/squaredcircle/

- Jim

Tools for scientific searches?

[Pi_0's don't shower]

Anyone here who's a scientist ever try to use "google scholar"? Unfortunately, it's not very good. What I'd like to see (as an Astrophysicist) is some way to do a search that combined results from difficult-to-navigate scientific sites, such as NASA's ADS abstract service, the Spires HEP database, and the arXiv.org preprint database. Finding what you need on these individual sites is often a pain, and to be able to search a compilation of them would sure be nice for me...

Re:Tools for scientific searches?

[dones]

Part of your problem has been solved: NASA's ADS can search arXiv.org at the same time as it searches its own database. Near the top of the page you can pick these databases to query:
Astronomy/Planetary
Instrumentation
Phys ics/Geophysics
ArXiv Preprints

Collect them all!

Quick search

[IanBevan]

[shameless plug] We make File Journal, which has a lightening fast search and displays the results in an Outlook 2003 style interface. This means you can group results by folder, date, file type etc. It can find (and restore) delete or renamed files too. Link in my sig if you want to take a look...

Oh boy! Shape-searches!

[gwoodrow]

...now I can just do a search using the stuff from Commujism to find more real pr0n!

Google

[Anonymous Coward]

What could possibly be better than Google?

Re:Google

[pubjames]

This is a test

how about google blocking a domain for good

[Sark666]

When I get pages and pages of crap that we all know are ads, I wish I could just check a box, block this domain from future searches.

Click on enough of them and a user might just see search results similar to circa 96

What I want is ....

[chris_sawtell]

... some method of telling the search engine that the link is dead.
All it needs is a tick-box beside each link.

If we all co-operated about it, the quality of searches would be improved considerably for all of us.

Google: Please patent the idea on my behalf, I think & hope it's sufficiently trivial, yet innovative & revolutionary, to impress the USPO.

PS: I mean co-operation, not the tick-box.

Re:What I want is ....

[kertong]

I like that idea, but seeing as how much Google's ad system is being spammed these days, the system probably wouldn't last long, with competitors blocking out each other's pages with bots and if it comes down to it, trained monkeys clicking away at a terminal.

it was the best of times, it was the blorst of times? you stupid monkey!

Re:What I want is ....

[chris_sawtell]

Fairly obviously Google would have to check the link really is a 404 before removing it from the index.

Please consider it Google. I would reduce the amount of storage you use.

Easy (relatively) improvement...

[Nobody You Know]

The number one search engine feature that would make my life infinitely easier would be precise proximity operators in search engine syntax.

(For those who don't have a clue what I'm talking about, LEXIS-NEXIS, among others, allows you to run searches like foo w/5 bar (the word "foo" within 5 words of the word "bar"), or even foo pre/5 bar (the word "foo", followed, within five words, by the word "bar". Good proximity engines allow you to search not only within x words, but also to order terms, to specify root words within terms, etc.)

It would be great to have people reviewing and whitelisting page results, but that takes human interaction. Implementing precise proximity operators, though, can give you nearly the same benefits without any of the human cost.

Many people here have suggested eliminating ad text from search results, but if history is any indication, any algorithmic system that we can come up with to do so will be circumvented pretty quickly. The one way to fix this is to allow me to say that I want the word "modperl" within 10 words of "solaris", rather that just specify any page that contains both terms. That will get rid of 95+% of ads right away.

Surely, with all the bright people at Google, this is something that they can figure out pretty easily.

Re:Easy (relatively) improvement...

[joker784]

You mean like this: Google API Proximity Search ?!

Re:Easy (relatively) improvement...

[divbyzero]

Try "a within 5 b" or "a beforewithin 5 b" using an AltaVista advanced/boolean search.

Google is not a good choice of search engine if you want to specify your query using precision operators. Maybe someday they'll decide that the small number of users who want and know how to use such a feature are worth the large amount of effort to develop and support it, but it's unlikely.

(And yes, I'm both a former AltaVista developer and a Lexis/Nexis user.)

New Zealand in Canada?

[refactored]

What I want to know is why, although I am sitting on South Island New Zealand, when I type in www.nzdl.org do I end up in Canada? http://www.sadl.uleth.ca/nz/cgi-bin/library

Re:New Zealand in Canada?

[kinko]

because most of our visitors are international, they get much better response times from our Canadian mirror. (And also then we don't pay as many exorbitant data charges).

Our server here assumes that clients that don't resolve to *.nz are from overseas, and redirects them to the mirror. (Yes, I know this isn't perfect. It's not my fault.)

People First, Technology Second

[bildungsroman_yorick]

People are the problem. What's needed is for people to get off their butt and learn to exploit the technology to it's full capability. These people could learn how to use Google more efficiently if they read something like Johnny Long's Google Hacking. Link: http://www.securitymanagement.com/library/Google_H acker0704.pdf

However this will never happen because the average joe is inherently lazy so we'll have to spoonfeed all the techno-numpty's with technological updates until they stop complaining.

Re:People First, Technology Second

[1tsm3]

Dunno when you geeks will try to look at computers as a tool for the common man. What you are suggesting is similar to a doctor saying "don't manufacture medicines, because if the people do exercise and keep their body healthy, they wouldn't need most medicines". Well that just doesn't happen. You probably are a couch potato yourself.

Btw, I consider myself to be a geek, but I don't look at computers with a narrow mind. I don't use text consoles when I can get more done using GUI based tools. Make the computer work for you, not the other way around.

Re:People First, Technology Second

[bildungsroman_yorick]

We are talking about search technology not medicinal technology. Therein lies your fallacy.

Search technology is supposed to help find valid information from the query one enters. The better the query entered the more efficient the search.

I read an interesting essay once from New Scientist that stated that in an information age our questions are more important than our answers. Our society has become a culture of answers. Look at pop-culture for an example. We have an abundance of quiz shows, facts on demand etc. Our society demands good answers to our questions. Not matter how stupid the question. This demanding of 'better search engines' isn't the problem it's people not asking good questions of the search engines. If you want a good answer ask a good question.

A simple example of this would be to start using advanced operators in google like site: or filetype:. I have a sneaking suspicion that these 'tests' these companies are using are nothing more than single words entered into the search engines. Talk about an ambigious query.

WOW!!! And it's even useless!

[mattspammail]

I think I can speak for everyone else in the entire world when I say that *nothing* sounds as worthless as the forementioned items.

Maybe I'll write an article about how the Oldsmobile is a fantastic find.

They already do that

[harmonica]

Google already identifies similar or exactly identical results. Sometimes it returns a message saying that it has suppressed similar results like the one it is displaying.

So you might say that they have to improve their similarity detection algorithm, but I'm quite certain that they are working on that already.

A related problem is to find parts of a page that are "just" menu structure, like links on the left or on the right that are less important than the actual content. That information could then be used to influence ranking. Just like the importance of text between h1 tags is higher than normal text.

Another cool concept in search engine.

[geek2be]

This one connects you with people searcing for similar keywords. I guess the idea is to have another set of helping eyes.
site: http://www.chatnsearch.com/

do you mean searches like this?

[Anonymous Coward]

query: murder some text like this for keeping it simple

results by weight:

1. ...the text about dogs is simple for... yet to some
2. ...keeping it simple stupid... an OCR scans text into an electronic...

...
...
...

(around hit number 381) ...in order to murder some text like this for keeping it simple we were forced to...