Slashdot Mirror
BBC Magazine's Search-Engine Shootout
An anonymous reader writes "On BBC Online's excellent Magazine, there is a shootout between Google, MSN, Yahoo! and Ask Jeeves. Search tests were conducted on five criteria: an obscure fact; multiple meanings of "raleigh"; speed; and current time in Sydney. Yahoo! is the fastest of the lot. Google has the cleanest interface. MSN Search fared worst of all. Jeeves is the apparent winner for features like related search. (Author claims to be a Google nut.)" This may be the nicest thing anyone's ever said about Jeeves.
Within the next two years max. That spells trouble for Google and its shareholders, who still place a very high premium on what is quickly becoming a common service. Fortunately for Google insiders, they should be able to cash out long before the regular dopes in investorland figure this out.
and want karma, just steal it from the previous thread which discusses this article.
oops--cynical mode off. If you are interested in the article, please seach through the above thread for other interesting comments & so that your own comments can be even more interesting or insightful
Granted Teoma is operated by AskJeeves, but it's hardly the same engine, but it's too bad that they left it out. I think it's one of the better searches out there -- it tops MSN's new search, easily.
The most important thing for me is a clean interface, and thus I don't care much if Yahoo is faster. Just compare hotmail with gmail, hotmail is like a newspaper. I just want to read my email! I hope MSN search doesn't load thousands of images to display your query.
Send email from the afterlife! Write your e-will at Dead Man's Switch.
I find it highly suspicious that Incredifind and Lycos Sidesearch were left out. I mean, c'mon, search "helpers" that install themselves automatically and lead me to fascinating products I never knew I needed deserve some mention, right???
It helped me track down a couple of old friends, even when all others had failed. To be fair, it was the last one that I had turned to - I'd even tried dogpile and lycos before that.
"How like you to drag your keyboard to a gun fight." - Aaron Bedard (BANE)
This is hardly fair as MSN search is in very early beta while Google and the rest have been around for several years
If I recall correctly, Google does a lot of its indexing and listings by how many web pages link to the page in question, the more popular the page, the closer it is to being first in relevance of a search.
I've found this to be quite troublesome when it comes to searches for information that instead give me commercial sites trying to sell things - "samsung 753df monitor review" gives me one actual review and then a couple of pages worth of links to sites that simply include user reviews.
Google really needs a better way to filter out these pages than having users type in "-consumer, -resale -'buy now!'".
Jeevus, I forgot the search existed.
Ever since google became a verb, it's all I use.
Jeans are common things, and yet, people still buy Levis.
Brand recognition is key in any market. And dont think that when search engines become "common", they'll be better than Google. Running www.Google.com isnt in the reach of most... hardware costs, knowledge, etc.
Even Microsoft can't seem to catch up with them, and it's been many years Google is #1 in my book.
Eureka Science News - automatically updated
I've read multiple comparisons between all these search engines, and most of the time (all the ones I've read, anyway), MSN Search comes out dead last. The question I have, is what does Microsoft have to say about all of this? I mean, they promised everyone the best, but now people are saying that their search, in summary, kind of sucks. Have they released an official statement justifying what's happening? Also, we do have to remember that this is only a beta test of their search engine (not that I'm defending it).
- dshaw
Perhaps someone can enlighten me as to why speed really matters in these cases. Can you really tell the difference between a .18 second return and a .97 second return once you account for varying connection speeds, internet traffic, etc? Is speed really a relevant criterion? Obviously I'm missing something.
http://www.rootstrikers.org/
From TFA: Yahoo: 0.18 seconds. Google: 0.97 seconds. MSN: 0.92 seconds. Others not available.
I think the author got these numbers from the search page, where the time is shown along the number of search results. However, the meaning of this number is not well-defined. I don't think it's even guaranteed to be accurate.
(btw, I tried on Google for "raleigh" and I got 0.30 seconds. Yahoo gives me 0.12 seconds. Trying jumping around the search results page, Yahoo gives me about 0.25 seconds, and Google sometimes goes as high as 0.70 seconds. However, I must note that Yahoo doesn't let me jump over as many as +/- 5 pages, but Google lets me jump over +/- 10 pages.)
I once had a signature.
Well, I started to find Google exteremely furstrating now. Whenever I try to serach for documents/specifications/data sheets of a certain protocol or an idea that has a large commercial use, google would send you pages, and pages, and pages of products that matched these keywords. It's very obvious that companies are buying these keywords. Thus, they have higher priorities on our first few pages in the search results.
"Evil thrives when good men do nothing"
I only visited Jeeves for the easter eggs. Sadly, it seems that most of them are now gone.
Cheers,
Adolfo
If somebody involved with this story can't count to five, how seriously should we consider it?
Sponsored by SILA -- Search for Intelligent Life in AmericaThe three most important words in a relationship are "I love you." The two most important are "Humor me."
When yahoo gave me 0.25, that was a rare worst case scenario. Same as when I said when Google gave me 0.70. Actually, I just hit another worst case of Google that gave me 0.88 seconds.
I once had a signature.
Google has problems coming up with accurate results for the phrase "to be or not to be". Altavista and others produce 100% accurate results in the top 10, while Google will have between 1 and 3 results of pages not containing the phrase.
Don't blame Durga. I voted for Centauri.
As long as the pages being returned contain the keywords (in other words, are accurate results), how is this a problem?
Don't blame Durga. I voted for Centauri.
The fact that Google has close to a monopoly on searching means there is too much of a chance that people will try to exploit its ranking system to push pages up the listings. Of course, I'm sure they are continually tweaking the ranking system to try to detect these pages, but if there were more search engines in popular use then it would be harder to find a way of exploiting them all at once.
It has nothing like one. Yahoo search, msn, altavista, dogpile, and many others exist and are used. Monopoly means something, and Google sure does not fit the definition of one.
Don't blame Durga. I voted for Centauri.
The five test criteria were relevance (the raleigh test), speed, the look (cleanliness of returned results), the obscure fact search, and the question (the time in Sydney).
If you can't count to five yourself, or double-check basic facts, well, that's really sad.
"Accept that some days you are the pigeon, and some days you are the statue." - David Brent, Wernham Hogg
Each search engine has strategically placed and promoted themselves in particular countries to make them the first choice.
Hence, if the test was done at a time when users from a country were flooding an engine, considerable changes in return times would be there (still staying inside 1 sec).
We are well aware that there isn't a single search engine that rules supreme in all the countries' networks.
>[...] Google will have between 1 and 3 results of pages not containing the phrase.
What are you talking about?
All the hits on the first page contains the phrase, one is to a site containing the complete works of Shakespeare. Why would you say this is not accurate?
Belief is the currency of delusion.
No, the third result is irrelevant, as it does not contain the phrase. If I had wanted where the quote came from (who made it), I would have asked for that. Instead, I asked for a pages containing a phrase, and Google failed to come up with completely relevant/accurate results. There are plenty of other "real world" examples of when I do phrase searches and Google fails.
Right now, by the way, it looks like there is only one bogus result (90% accuracy). Pretty good.
Don't blame Durga. I voted for Centauri.
Wired recently had an article recently about how brand names along just aren't cutting it any more.
Consumers are wising up the quality is more important than name.
Which of the companies -- MSN, Google, Yahoo!, and AskJeeves -- is willing to stand up to the Chinese barbarians and to refuse to censor any results from the search engine?
We know that Google is willing to succumb since it hires a large number of Chinese H-1B workers.
It was in the test, did quite well, and was the only one not mentioned in the /. story.
That is how I searched, and Google failed. When I point this out, people usually say how hard this is to do. Yet, it is easy using SQL logic to return strings containing substrings. It is so easy, in fact, that most other search engines have been doing this accurately for years. Finally, Google admitted that this is a bug and that they will fix it some day.
Don't blame Durga. I voted for Centauri.
I think Google is already in trouble.
I didn't need a team of British reporters and weeks of research, I just typed "Who has the fastest search engine" into the search engines. Duh! And the results are...
Google says "AlltheWeb"
Yahoo says "Ixquick"
MSN says "Ixquick"
Jeeves says "AlltheWeb"
To break the tie I asked the winners:
Ixquick said "AlltheWeb"
AlltheWeb said "Ixquick"
So there you have it: A tie.
I agree with this post and the article. Furthermore, I can't remember the last time that I went out of my way to buy Levi's jeans. If it's on sale and it fits well and looks good, that's good enough for me.
There's no http://search.suprnova.org/ on the list too:) And nobody bothers
Signature Pro version 1.13.2-3 release 83.5 beta3try7 after-breakfast edition
What a stupid post.
Internet advertising was commodidized years ago, and that's where Google makes most of it's revenue, one way or another, according to the financial papers I read.
The fact that Google has profit now, even after 99% of businesses in the mythical "internet sector" have gone belly up, is a testament to the fact that they probably know more than you do about this whole business thing.
It's been a long time.
Since when have people actually *used* Ask Jeeves? I've driven by their office (in LG) before, and it looks pretty sad. I can understand a shootout between MSN, Google, and Yahoo, but to incorporate Ask Jeeves is like inviting a Christmas Island to nuclear talks with North Korea.
I didn't ask for someone's subjective definition of "relevance". I ask for something that is the only thing I ask for with search engines: pages containing the phrase. Google has problems doing this well, that is all. If you think that "to be or not to be" is the same phrase as "tobeornottobe" and that "now here" is the same phrase as "nowhere", perhaps you deserve the flame.
By the way, until very recently, a search on the phrase came up with the page www.be.com. We know that this is a bogus/inaccurate result as well, but even you might admit that this one is not even "relevant" by your standards.
Don't blame Durga. I voted for Centauri.
This is the whole reason why Google has managed to survive.
(Assembled) computers, for example, are a commodity. Anyone can buy parts off Pricewatch, build them, and sell them. This doesn't stop companies from making money by doing it.
A better example would probably be Logitech. They make keyboards, mice, and speakers--certainly commodity items--and make money from selling them.
Same with Google. Anyone else can, and many other people do, provide relevance-engine searches, but Google does it better for cheaper than anyone else.
This is how capitalism is supposed to work. Google doesn't survive because they don't have relevant competition (like Microsoft), they survive because they can stand up to their competition (like Logitech).
And we all get better, cheaper stuff for it.
You are probably the only person on the planet who is bothered that one of the results for "to be or not to be" is the site tobeornottobe.com containing the works of William Shakespeare, but nevertheless this really does show that Google isn't suitable for your special requirements. I think we all understand that now. Thanks for letting us know.
The real Ralph Yarro posts as Anonymous Coward. Anyone else is an impostor.
From Google:
;-)
"Your search - "lynndie england" - did not match any documents."
"Your search - "lyndie england" - did not match any documents."
nuff sed.
Mark
and the competition is not far behind http://www.mooter.com/
-every shot makes somebody happy!
i always use http://www.jux2.com
it grabs results from google+yahoo+ask jeeves, and then ranks stuff according to how it appears in those. since there's only 35% overlap the results are usually better than using 1 engine alone.
Actually, this is just one of the most obvious ways to show that Google has problems doing phrase searches. I've had this happen on many other "real world" search examples.
"but nevertheless this really does show that Google isn't suitable for your special requirements."
My "special requirment" is not that special: it is to have relevant, accurate results. Is it really too much to ask for: search engine results that *gasp* contain what you asked specifically for? Google is buggy in this, while altavista works. However, Google is my search engine of choice because it has a lot more results than Altavista. Even if some of the results are incorrect....
Don't blame Durga. I voted for Centauri.
It may not be Google, but I love its simple web|news|pictures|video|audio interface.
- Just my $0.02, take with a grain of salt, your mileage may vary.
Type "fuck microsoft" into Microshit's engine and then type "fuck google" into Google.
Which search engine is better? Try it at home and you decide!
---Technology will liberate us if it doesn't enslave us first.
Those snobby brits... Any tenuous connection to their fading culture and they're all over it!
On a completely relevant note, I have a whole playlist of P G Wodehouse audiobooks on my iPod.
Inspired, I'm going to start my own search engine called AskWooster. It will search for send its agents, called Gussie Finknottle, Tuppy Glossup and Bingo Little, out to seek for interesting stuff and return 3 cats, a fish and a size 14 top hat.
Whoosh...! The sound of all this going completely over the heads of the /. crowd.
I can not use any other search engine other than google. It's just impossible for me
~Ilyanep
To get message, take amount of carrier pigeons at each stage mod 2. Then decode binary.
Millions of people, and the slashbot hivemind, tend to find google to be a very useful search engine. You are not alone in your opinions, but you are outnumbered. Give up, nitpicker.
Self-referential sigs are rarely entertaining.
cauz you're talking about average, but i'm talking about worst case. Try clicking on the 15th page, then the 23rd, then the 6th, etc. You get the idea. Search engines that give out search results "lazily" (evaluate only upon request) tends to be hit hard by this random access method.
I once had a signature.
If brands are irrelevant, howcome you are posting a link to wired on slashdot? Surely there are lots of other and even better web magazines and forums... just the names of sites escape my mind.
signatures pending - ansa@kos.to - (dont mail there)
Personally I like Jeeves. There have been several times when searching for something involving a common word that Jeeves' context sensitive approach came through. Sure, it makes a lot of strange suggestions, but hidden among them is an occasional gem.
I'd like to defend Microsoft this time... Their new search engine is barely through the "technical preview" (aka alpha in this case) stage and has been given limited resources as it's only available for beta testing purposes.
"The new competitor MSN has had teething problems and we should bear in mind this is a test mode."
Yeah, so why didn't you test the finished product? If they would've tested MSN Search, at least test search.msn.com and not beta.search.msn.com as they apparently have done. It's not interesting to me as a reader to see the performance of a search engine where very little fine tuning has been done. It's not like I'd use a search engine with little hardware resources causing more time-outs than succesful searches anyway, even if it would win in the test.
Beware: In C++, your friends can see your privates!
What part of "it is my preferred search engine" did you miss? (By the way, it is not my "opinion" that the phrase-search is buggy. It is just a fact. I showed the bug).
Don't blame Durga. I voted for Centauri.
I did the same thing. Lots of results. Then, I clicked on the images (instead of web search). Nothing. Unless I did something wrong. You try it.
Don't blame Durga. I voted for Centauri.
... but not before offering several entirely unsuitable search returns in the form of potential marriage partners.
Suprisingly, I didn't see anyone mention it. So, may I suggest:
http://vivisimo.com/
Much easier to navigate than Google, thanks to clusterings that actually make sense (usually)!
These sites don't give the time it took them, so I could only measure how fast the page loaded. My connection is relatively slow (google loads in 2-3 seconds, Yahoo in 7 seconds), so speed measurements are not very reliable or useful, but I gave them anyway.
It's not clear from the BBC article what was the exact query for the second test. I used "What's the reported IQ of an Alsatian" (without quotes) for the first attempt (later I tried this at Google and it didn't work, so consider this attempt invalid). After none of the search engines gave anything, I tried "Alsatian dog IQ" (without quotes).
Teoma:
- No ads, no clutter, to the right search refinements and relevant links from catalogs.
- 3,272,000 results. City is No 1 (as well as 2,4...), bikes are No 3 (and 6), explorer is No 5, charity is not on the first 6 pages.
- 7 seconds
- No result on the first attempt. No results on the second attempt.
- Direct link to Timeanddate.com's page for Sydney is No 1.
Vivisimo:- Original interface with clustered results (frame-based), metasearch. 2 sponsored links.
- Top 249 results only. City No 1 (6), bikes No 2 (3), charity 11 (there are 20 results per page), explorer No 17.
- 10 seconds
- No result on the first attempt. During second attempt using the "Shepherd" cluster and the 6th result I found out that Alsatians are the 3rd smartest breed (after border collies and poodles), but no exact IQ estimate.
- Direct link to Timeanddate.com's page for Sydney is No 1.
AllTheWeb:- 3 sponsored results (marked as such) on top, no clutter, search refinements.
- 8,350,000 results. Bikes No 1 (and 2), city is No 3 (4,5...), charity No 9, explorer No 13.
- 5 seconds
- No result on the first attempt. On the second attempt it listed the relevant page at No 11 (although unlike at Google, the answer itself wasn't in the site summary).
- Direct link to Timeanddate.com's page for Sydney is No 3.
Conclusions: AllTheWeb is excellent for searching, basically as good as Google (from this very limited test). Vivisimo is good for searching, clustering is very good. Teoma no as good - didn't find the charity.Refinements at Teoma are almost as good as Jeeves. Refinements at Vivisimo the clustering is not as effective as at Jeeves (because the number of search results is smaller), but still good. Refinements at AllTheWeb, though there wasn't any for explorer or charity.
Interface is great everywhere, no gimmicks, like at A9 (which has a monstrously huge 200Kbyte page), everything is slick. Frame interface at Vivisimo is good. Not too much ads, at Vivisimo they are marked, at AllTheWeb they are marked too, but not as well, and Teoma doesn't have ads.
Next I will try some visual search tools (Grokker, Kartoo, etc.) and will post the results in the reply to this post.
Future Wiki -- If you don't think about the future, you cannot have one.
Also if brand names don't matter, why are you posting on slashdot instead of mycoolblog.blogspot.com?
----
Write Code That Writes Code
Another remarkable study found that just 4 percent of consumers would be willing to stick with a brand if its competitors offered better value for the same price.
Also there is a difference between people saying they will switch brands, and people actually doing it. Market research is right up there with astrology and tarot cards in predictive accuracy.
----
Write Code That Writes Code
These are the searches I made in order and nothing cut out except for the cases where a question didn't give anything on page 1 for either of the engines.
:-)
:-)
Who created Slashdot?
Ask Jeeves: CmdrTaco as #1 (correct answer)
Google: CmdrTaco as #4
Who designed Ford?
Ask Jeeves: Henry Ford Academy as #1 (HF is correct)
Google: Nothing about Henry Ford on first page
What wavelength is red?
Ask Jeeves: 622-780 nm from search result description of #1
Google: "about 650 nm" if you follow link and read on in the page of #1
Sometimes we see negative side effects from this question system...
Where can I download Nero?
Ask Jeeves: First links to download service that did indeed have Nero as a download, but versions from 2000. Ahead.de as #3
Google: Ahead.de as #2 (a completely unrelated site as #1)
I tried to refine...
Where can I download the latest version of Nero?
Ask Jeeves: Got confused, but finally got the proper link (official download page) as #8
Google: Nero 6.6.0.1 (correct!) at unofficial site as #1, official site as #2.
Do SCO have a case?
Ask Jeeves: Groklaw.net as #1
Google: NewsForge analysing article as #1 and other SCO related news articles.
When do Revenge of the Sith have its premiere?
Ask Jeeves: "has its worldwide premiere on May 19, 2005" according to search result description for #2.
Google: Nothing in search result descriptions, and nothing I could find by following a few search result links on page 1.
Beware: In C++, your friends can see your privates!
The reviewers added the disclaimer that MSN was having "teething problems" (ie. still in progress), so the informed reader needs to keep this in mind as they compare MSN to the other more established competitors.
There are plenty of Gmail reviews out there, even though it is still in beta. I think it's perfectly acceptable to review/test a beta product that has been put online (for the express purpose of testing) if for no other reason than to provide feedback for MSFT to use if they so desire.
"Nokia is not a country, it's the capital of Finland!" -Moderated "Informative". Yeesh.
flash-based Kartoo and HTML-based Mooter. First I searched for "raleigh" and tried exploring the visual results. The next search was for "Alsatian dog IQ", the last for "what is the time in Sydney". I didn't expect to see the results on the first page, since these engines are not really page-oriented, I was willing to quickly refine the results using their special facilities.
Grokker:
Kartoo:
Mooter:
Future Wiki -- If you don't think about the future, you cannot have one.
Want to find an evil corporation?
How about a monopolist?
Or, for something a little different, the greatest threat to innovation in our world?
All straight from the horse's mouth. (-:
Got time? Spend some of it coding or testing
You do have a point.
"Well-managed brands have extraordinary economic value and are the most effective and efficient creators of sustainable wealth."
The real problem is that the brands are poorly managed. One of the reasons brands became powerful is that the products made stood up to a certain level of quality. If they cannot retain that level of quality, buyers will buy something else.
They mentioned Sony DVD players, well, they've been hit by stupidly making a couple models (330 and 560, I think) that not only couldn't read CD-Rs, CD-RWs, had only a 90 day warranty, they also had a very rediculous failure rate, especially with the crap they make in Mexico. Other Sony products have had sketchy reliability, and have long had an uncompetitive warranty length or absurdly priced.
Note, I do own some Sony products, but I only buy them after a good amount of research.
Search tests were conducted on five criteria: an obscure fact; multiple meanings of "raleigh"; speed; and current time in Sydney.
Am I missing something or is this only 4 things? Maybe the "multiple meanings of raleigh" counts as two?
The behavior is really not that obscure, 'get pages that have what I am looking for'. That is how I always search anyway.
Don't blame Durga. I voted for Centauri.
When I tried that the first hit returns:
That seams like quite an good answer to me.
Wel I guess noone will see this but it still is worth mentioning.
He didn't say that brands were irrelevant, but rather that they were insufficient to generate profits. It taks a strong brand, high quality product, and continuous marketing efforts to consistantly outsell your competition.
A good brand certainly helps. That's why Coke outsells store-brand soda and Lays outsell store-brand chips (in blind taste tests, consumers actually prefer most store-brand items). However, unless the quality of branded items improve, they could soon see their dominance come to an end.
Thank you Mario! But our princess is in another castle!
You're new around here, aren't you? Because, and I am saying this with a lot of love, Slashdot readers are not exactly the best example of the public's clothes buying habits.
~~~~~ BigLig2? You mean there's another one of me?
This may be the nicest thing anyone's ever said about Jeeves
Piffle. Bertie Wooster has many nice things to say about Jeeves. Why just the other day, I popped into the Drones for a dash of the t. and g. and no other than Bertram Wooster himself was holding forth at the billiards table, singing high praises of Jeeves, his valet. I believe the tone was something like, "The old boy has a large protrubence at the top-back of his head. That's where the brain is. It comes from eating fish and reading some crime novelist named Spinoza."
Comparing it to Windows will be a moot point, since El Dorado is going to have a 40% larger code base than XP.
I would of liked to see a comparisation against the clusty search engine with those other search engines. I've found clusty to be very usefull with it's categorial search.
Although their "sponsored results" (taken from overture) tend to anoy me, as they look part of the search results.
Change is certain; progress is not obligatory.
yahoo's now got 1 billion images.
where's the slashdot story on that?
No surprise that Google won that shootout. They conduct more effective R&D than any of their competitors. Google's workforce is almost completely PhD, while Microsoft for one employs mostly BS and MS graduates. Guess who delivers the cutting edge performance?
Interesting point, but unfortunately, your reasoning is pure paranoia . Do you have any reference to Google censoring? Tibet.org doesn't even mention Google. Don't you think they should mention it if they thought Google censored them?