Slashdot Mirror
Objectively Comparing Competing Search Engines?
aendeuryu asks: "My default search engine of choice is, like most of you I assume, Google. That said, some complaints about Google over the years do seem to have some merit -- basically, that sometimes the indices aren't always updated, that it's too easy to manipulate via googlebombing or legislation, and that maybe too many of its featured services never get out of beta stage. Maybe the fact that Google has gone so long without significant competition is enough to make one at least begin to ask questions about it possibly becoming stagnant. Personally, I'm so used to doing things the Google way (and achieving acceptable results quickly) that I'm not really interested in switching -- case in point, all the above links referenced were quickly found via Google. However, what am I missing out on by not giving (for example) Yahoo search a shot? Or, more to the point, how would one go about trying to effectively and objectively compare competing search engines? In what areas have people found Google to have become obsolete for their purposes? Have less ignorant people than myself figured out ways to test a competing search engine's efficacy for themselves?"
I open my browser, and see the Google page up and running. I started with Yahoo, I tried meta search engines, altavista, a9, and many others, but I never change my home page to be the other ones. I know Google, I know how to use the results and to view pages all in HTML and to get the cache and to search sites that link to me, or search a specific site. It's easy in the other sites, but I already figured Google out. Google works for me, when I find the wrong thing, I just add "-wrongword" to the end and I find what I need. I see all the blogs and misindexed pages, but I've never really suffered from Google Bombing or any of the other problems that are mentioned.
Make your computer faster: rm -rf
When I am not getting satisfactory results using Google(about 30% of the time), I try Yahoo, and I usually find what I am looking for. If this keeps up, I might start my searches using Yahoo.
University of Washington
Student
Try a metasearch and let the server figure it out.
Teoma has this great feature called Related search which is very useful. Basically if you look for a particular topic, the search engine identifies all related topics and offers you a one click access to all of them. Makes the search equally usable for both a rookie and a domain expert using the same search term.
One thing I like about askjeeves and a9.com is the way the present the search results. I think the next step is to improve on the presentation of the results (data) to make it more usable/accessable. Hit up askjeeves and run a search. The preview feature is pretty nice. And check out a9.com searches with their Site Info mouse-over.
I know that's not true, but generally if what I'm looking for isn't in the first two or three results pages of Google, then I give up.
This has only happened to me a few times (not finding what I want with Google), however it does bring up an interesting point. I trust Google results so much, is it possible that all the search results can be misleading or wrong information?
I got this from a friend who works at yahoo...
m l
http://www.langreiter.com/exec/yahoo-vs-google.ht
Sorry if it gets slashdotted.
You don't have to bother evaluating better web based technologies. When they are worth using others will tell you about them. It's the nature of the web.
For example, a professor of the university department in which I worked came back from Digital Research Labs, enthusing about a great new search algorithm the designers of Digital's Computer Aided Design software had come up with. A short time later Altavista was 'it'.
The same happened a few years later. The buzz from collegues and those on the web was about a new search engine called Google.
The short answer is, "Don't go looking for the 'next search engine'. It will find you."
I was looking through my website's logs and noticed a ton of MSN bot hits. Then I noticed one coming from their search page. The search term was "UTC+flash" and my site was listed third in the search results.
My site has nothing to do with UTC or Flash. Turns out, it indexed my lame little archive page that displays article dates in UTC format. One of the article titles was something like "Flash Storm," so it indexed the "UTC" portion of the previous article's date and the word "Flash" that began the next article's headline below it.
It was cool that I got a free hit for it, but my site was hardly a relevant search result for that query.
Since I tend to have to do some SEO for sites, I tend to keep an eye on how search results are returned... one thing I've noticed is that Yahoo seems far more easily manipulated by URLs - ie, it seems to weight something like, "www.goats.com/goats" high for the term "goats" even when the site has little or nothing to do with goats.
Also, Yahoo and MSN both seem extremely poor about figuring out the "right" url to link to. It's almost as if they index the first thing on any domain they come across, instead of trying to figure out where on the site most people link to, so you'll often find yourself deep-linked into a site where you'd prefer to be looking at a higher-level page to start. Google deeplinks too, but it seems to be only when it's really more relevant to the content.
I don't use a9 much, but it seems like google with a different skin. I swear sometimes they're snarfing google's results and storing them. Not that this is all bad, since Google's results tend to be some of the best, but it's still eerie.
Try this: yahoo! vs. google
Google DOES NOT spider dynamicly created webpages. If you have, say for example, forums... it will spider only the first page. Yahoo, however, will spider the dynamic content [though with a limit to assure it doesn't get caught in a bot trap].
The Peanut Gallery, Ubergeek, Biblically Sober
NCAAbbs.com: Thousands of fans, Hundreds of teams, Just one place
I'm so glad someone raised this. I was thinking just yesterday that the internet was *seeming* to have become smaller. The linking pagerank system google uses is strange IMHO because not all pages are massively linked.. or have reason to be linked.. it turns the net into some kind of boys club...
With that said, I have found that my more obscure and better quality sites have been found on the last pages of google, with the first few pages being generally filled with amazon and other *for sale* sites...
There are sites i've seen that have been around for years and don't even get a mention on google. Word of mouth was the only way I found out about them. I also remember how much the internet opened up when I first used the "stumble to" firefox extension. Who knew these sites even existed?!
I'd say the google solution would be so somehow incorporate a similar "word of mouth" type ranking system as "stumble to" (or slashdot for that matter), so individual users can rank results "useful/not useful" to modify page ranks... Also their "similar page" section would also benefit from a "useful/not useful" to help google learn similarities...
my 2c
'plex
Rich Gentlemen Hide - The Existential Comic
No single search engine had won out so yo uhad a bank of search engines that you always scrolled through. What one engine didn't have another would.
Well a hell of a lot of those "old" search engines are still around! And they have become better over time. Google at one time was so much nicer than the others that people sort of got "lazy" and stopped browsing qround the engines. But everyone else didn't just curl up and die.
So just start engine hopping again. Try Google first if you must, but then try Yahoo, search.msn, alltheweb or search.com or other meta search engines that search all the real search engines for you.
Multiple sources of info have always been and always will be better than one giant conclomerate of info such as Google is becoming.
Contrary to popular belief, coding is not all free blow-jobs and beer. Those things cost MONEY!
This is an interesting demonstration of just how much subjectivity effects perceptions about search engines.
Alltheweb.com produces the same results as Yahoo search (basically ever since Yahoo merged with Overture). Yet you describe them as being distinct and with different qualities. You even will search on one after searching with the other.
sigs are a waste of space
is the insane amount of blogspam that it picks up.
searching for a product name or model number, especially for anything new will bring up thousands of metoo blogs that copy & paste the same fucking press release over and over providing absolutely nothing new to the subject.
fuck that
While I'm sure you had good intentions when posting this I decided to do my own little research (just to see what the other search engines returned) and I got rather different information that you did.
~ NS-1~linkin_id-3058014 & PAGE=PRODUCT&PROD_ID=962864&fp=F
l amingos_are_pink.htm
0 000244/104-8146884-9815145?v=glance Sells Answer
These are the top 4 results for "Why are Flamingos Pink?" (entered without quotation marks) in the top 3 search engine providers Yahoo!, MSN, and Google.
Yahoo!
http://www.finelinefeatures.com/pink/
http://www.shopping.com/xGS-Pink_Flamingos~FD-113
http://www.thewildones.org/Animals/flamingo.html Contains Answer
http://www.overstock.com/cgi-bin/d2.cgi?cid=48422
MSN
http://199.216.204.14/project04/legends2004/why_f
http://home.nycap.rr.com/useless/pink_flamingo/
http://www.cat1234.com/id56.htm
http://www.straightdope.com/columns/010518.html Contains Answer
Google
http://www.straightdope.com/columns/010518.html Contains Answer
http://www.thewildones.org/Animals/flamingo.html Contains Answer
http://www.amazon.com/exec/obidos/tg/detail/-/006
http://webexhibits.org/causesofcolor/7.html Contains Answer
As we can see... google out preforms all three, offering 3 sites that actualy contain the answer in the top 4 results, two of which are in the top two. CLEARLY providing better results on at least this topic than either Yahoo or MSN.
Anders
"Circa" notwithstanding, ZIP Codes came to existence in 1963.
Do you love freedom??? Do you love freedom!!! DO YOU LOVE FREEDOM!!!!!!!!
heh... too far down the tree... /me shrugs
I just think it would be interesting to have some more people post similar posts. You know... pick a topic you might search for, post what you wanted to find, and which search engines provided the best results. Barring mental retardation for setting up the search it seems at the very least some kind of data could be collected to talk about.
Anders
Bad name, good search results:
Clusty, aka, Vivisimo: http://clusty.com/
This one has succeeded when Google has failed.
WWW
(Slightly premature announcement coming up.. but hey - it's Open Source so that's okay, right?)
I've just started a (Java) project to interface to a number of search engines. It might be a good place to start if you feel like doing some coding. See https://argos.dev.java.net/ - there is no release yet but the code is in CVS.
It currently supports Blogdigger, Feedster, Del.icio.us, Google, MSN and Yahoo (and Google Desktop search). I'd like to include Ask.com, too, but they don't provide a programatic interface and I refuse to screen-scrape.
In my opinion none of the other search engines are close to Google in quality of results. I've found (to my surprise) that Ask.com gives me the second best results (they bought the old Teoma search engine, which was always okay. It had an index almost the size of Google's, which neither MSN or Yahoo can match yet.)
Interesting that you searched for "Why are flamingos pink? and got good results but I searched for "why are flamingo pink" and got mostly irrelevant results. Just the absence of the "s" on flamingo gave quite different results. /Mark
I don't read your sig. Why are you reading mine?
What if you don't know what you're looking for?
Then you could try keyword mapping
Sometimes I have to research a new technology but don't know the what to look for. If you know what are important keywords in a particular technology, you could get better search results on Google.
Kwbrowse can help you get the right keywords to search for.
I know that this probably is caused by algorithms and how the different search engines treat stop words, but still: It seems as if no one search engine is best at everything yet, although Google currently (probably) scores better most times.