Slashdot Mirror
Microsoft Lost Search War By Ignoring the Long Tail
Art3x writes "When developing search engine technology, Microsoft focused on returning good results for popular queries but ignored the minor ones. 'It turned out the long tail was much more important,' said Bing's Yusuf Mehdi. 'One-third of queries that show up on Bing, it's the first time we've ever seen that query.' Yet the long tail is what makes most of Google's money. Microsoft is so far behind now that they won't crush Google, but they hope to live side by side, with Bing specializing in transactions like plane tickets, said Bing Director Stefan Weitz."
Company releases an inferior product, much later to the game than competition, makes excuses for failure, water still wet.
"we've got trenchcoats and bad attitudes" - John Constantine, HellBlazer
.
Search engines are all about people looking to find stuff. A good portion of what people look for are probably new things that are happening now.
So, Microsoft goes off and designs a brand new "bet the ranch" search engine, without even knowing how its customers use such a service. Yes, that sounds like Microsoft.
Someone should tell Medhi that it also helps when you don't game the search results to fit your corporate agenda.
From time to time, I try out the following query on Bing: "Why is Windows so expensive?"
The day that the first result returned is NOT a site about Macs being expensive is the day I'll start to take Bing seriously. Until then, I'm sticking with Google, which is at least honest enough to properly index anti-Google queries.
Nope, I'm sorry but you didn't get the frist post. On the other hand you did get the first post! Congratulations!
Now be a good boy and go back to the main page to wait for the next article so you can try and be the first one to post something again.
As long as there are search engines and choices, the war isn't over. A war of unskilled attrition, ( like Microsoft plays ) can take a long time to end.
---- Booth was a patriot ----
It's not the 20 relevant ones that's the trouble, it's the thousands of irrelevant ones.
Only three things are certain; death, taxes, and apocryphal quotations - Ben Franklin.
Microsoft is so far behind now that they won't crush Google, but they hope to live side by side...
The same way the Zune lives side by side with the iPod.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
I would say they lost by:
1. Being too late. Search engines have been around for many years. You can't easily launch a search engine now without a massively improved user experience over what is already available.
2. Not being trusted, I don't want to use Microsoft's search engine as it may subvert the results to promote their wares.
3. Stupid name. Every time I hear "Bing" I think of Ned Ryserson from the film Groundhog Day.
4. OTT interface, I don't need a big background when I'm looking for stuff.
world of the Internet is not just a place to find information, but it is also important, the Internet is about "make money". microsoft should have realized it.
http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html
Perhaps Microsoft just cannot think like that? To be clear, Microsoft saying that maybe Google and Bing can perhaps exist side-by-side is a clear admission of defeat. Microsoft never says that, so you know the situation is bad. I just can't understand why they got a bee in their bonnet and wanted to chase Google in the way that they have. It was clearly a knee-jerk thing and they hadn't clearly thought about it. The only major difference they did was change the name from the stale MSN Search name to something they thought was cooler - Bing. Nothing else changed.
To not take into account that people search for many random and obscure things put together that won't have been recorded before (language is a very broad thing and what people search for is also time-based i.e. NOW), and not to have some sort of logic to aid with that, is utterly unforgiveable. What the hell are Microsoft Research doing?
I don't think Bing will ever out-Google Google. So it's strange that they don't try to identify problems with Google and address them. They seem to start out with the assumption that Google is perfect, so the best path forward is to do everything just like Google, only more so.
The big problem with Google is privacy. Why not try to make a search engine that doesn't track what you do? I'd pay a subscription for such a thing. Maybe most people wouldn't, but I would. Search is such a big market that 5% of it is still huge. Maybe 5% of the people in the US would pay for private searching.
MS has had a kind of bullying culture for a long time, and they've declared war on open source, so we've viewed them as the bad guys for a long time. But windows is a heck of a lot more open than the iPad, and their business model isn't based on data mining. In a lot of ways, they've been left behind by many of the most toxic trends in the industry. They should listen to some of the things that we linux folks have been saying, and try to fit them into their pitch when they can. Talk about the value of controlling your own data, of privacy, of letting anyone who wants to write a program and distribute it, of being able to install your software on whatever hardware you want. That's not snake oil -- it's good stuff.
The strange thing is that they've missed those toxic trends not because they value the good alternatives, but because they're big and sluggish and not very agile. They've just been left behind. And all they want is to catch up so they can turn the same screws on us that Apple and Google turn. It doesn't occur to them to make the kinds of arguments I'm proposing here.
Maybe this is actually quite useful. If they'd do exactly the same as Google, then there's no advantage for consumers, both do the same.
When they're different, that means Bing may actually have a use, namely for these cases where you specifically need something from the "short tail".
P.S. I've never used Bing so far.
Why not try to make a search engine that doesn't track what you do
maybe because knowing what you do leads to much better answers...
I'd pay a subscription for such a thing.
http://www.ixquick.com/ -- there ya go.
You can even google it ;-)
Bing can't perform as well as Google because for one, it doesn't have the same data to begin with.
For example, have you ever released a new website and watched how long it takes for Bing to index it compared to Google?
Why not try to make a search engine that doesn't track what you do? I'd pay a subscription for such a thing.
How would they keep track of who has subscribed if they're not tracking people?
The big problem with Google is privacy. Why not try to make a search engine that doesn't track what you do?
This is Microsoft we're talking about. If you believe they'll ever do that, I've got a bridge to sell you.
The big problem with Google is privacy. Why not try to make a search engine that doesn't track what you do? I'd pay a subscription for such a thing. Maybe most people wouldn't, but I would. Search is such a big market that 5% of it is still huge. Maybe 5% of the people in the US would pay for private searching.
Microsoft doesn't have a problem with google abusing privacy. Their only complaint is that they want to be the ones doing it, not google! Ask me how I feel in a year or two but for now I still trust Microsoft a whole lot less than Google.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne
>The big problem with Google is privacy
No it's not. Maybe it's a problem to you and a few other privacy nuts, but no one else minds, and it's good to Google, that's why they do it.
Why did they need to be in it? I realize that doing something because you can and because you want to be the one with all the marbles and such is part of competition but at some point it becomes obsession. It did so with Microsoft a decade ago. Everything someone does they want to mimic. That's the idiocy, they mimic and essentially have from the beginning. They're the Chevrolet of technology.
They can't except that Google is just better at search. Period. Why can't they just accept that and stop stalking the search market? At some point you have to accept that the chic just doesn't dig ya and move on.
Oh, so since they screwed up, they're not going to be able to completely destroy Google, so they'll settle for even competition? It's this kind of thinking that's gotten Microsoft into trouble in the past, the philosophy that they can be the only one, so they have to destroy anything that remotely competes with them.
The day I searched (a few months ago) for information on the Toyota recall and got an automatically scrolling box of Twitter posts was the day I switched to Bing.
(That said, Bing really isn't as good as Google... but most of the time it's almost as good, and I really don't want anything to automatically scroll, and I really really don't want any results from Twitter.)
You're comparing oranges to apples, so to speak. An operating system is not equivalent to a single product put out by a company.
Tell me, is the Xbox more open than the iPad? Because those two products are the ones you should be comparing. Closed, tightly regulated ecosystems in both cases, although I'd still give the iPad the edge for ease of developer access.
On the other hand, is Windows more open than OS X? Clearly, the answer to that is a resounding NO, as you quickly realize as you jump through Microsoft's "Genuine Advantage" license code hoops.
I have my own website which is absolutely authoritative on its rather narrow topic. This website is easily findable by its unique keyword that identifies the topic (similar to searching for "slashdot", you won't find any cooking websites or shopping, only tech stuff). I'm at #5 on Google and my number one competitor is at #6. Neither of us shows up on a Bing.com search, I quit looking after page 10 of results. The results just have a bunch of websites that I've never heard of before. Even more galling, Bing.com tries to play games with my results because I'm overseas. I search for "mykeyword" and select "Only English". Bing.com helpfully comes back with "Results are included for XXX XXX (foreign word that is the translation of my keyword)". Two of the sites on the first page say "Parse error: syntax error" as their preview. Yes, my site is in Bing's index and regularly submits XML sitemaps.
In conclusion, Bing sucks if it can't put my site in the first 10 search results. Hell, it should at least be in the top 100. I don't game Google, either, other than some basic SEO that any responsible business owner should do.
Shutting down free speech with violence isn't fighting fascism. It IS fascism!
There's a difference between keeping track of who has subscribed and keeping track of what subscribers search for. Of course, in this scenario, subscribers would have to blindly trust Microsoft.
"Oppression and harassment is a small price to pay to live in the land of the free." -- Montgomery Burns.
First Era: Human-Powered Search (1997-2002)
"Search isn't some relatively new effort that dates back to 2003 at Microsoft. Search, especially web search, is something the company has seriously pursued since 1997. In its first era, Microsoft started out with a crawler-based search engine, one that creates listings by using automation to harvest material from across the web. It then migrated to building a very good service that relied primarily on human power, human beings to either catalog the web or customize top search results with hand-picked answers. Bill Bliss was the person in charge during most of this period. Here's how it unfolded, over the years"
"I was always told "Search is not core to our business, Google is not a competitor, Yahoo is not the competition, AOL is the competitor to beat, subscription services is how we're going to win.", Bill Bliss, Former Microsoft Search Chief
Everybody is jumping on you as though you'd cut yourself in a shark pool because you committed the error of being factually inaccurate in a forum full of geeks. Which doesn't make you right, but still. . .
I think you inadvertently raise an interesting point.
You just assume that MS is being sneaky. And you have EVERY reason to believe this to be the case. Can you imagine a world where search is ruled by the MS totalitarian approach to everything they touch? I have a very hard time trusting Google, and they've got a pretty good track record, but MS. . ?
~Shudder~
I wouldn't trust them with anybody's info for two seconds. You KNOW they'll abuse it for profit the moment Game Theory recommends that as the best option for world domination and monetary gain. That's simply how they work.
So your automatic distrust of MS, while factually off the mark, is based on more than two decades of rotten corporate behavior. That aspect of the human instinct will often point at the wrong specifics (because the conscious mind is over-eager to interpret the warning bells) but usually in the right overall direction.
-FL
It is like everyone around here is too young to remember the last what 3-6 failures MS made at "new" search engine or too old and their memory does not work anymore.
There is no reason to waist time and effort on bing as webmaster, until bing (or whatever they want to relabel it) starts moving traffic I don't care about bing as a search engine.
Living in Chile
Comment removed based on user account deletion
If they didn't have innovative companies like Google, Apple, Sony, etc, etc, etc to lead the way, they would not have a business model at all. Current model: wait for someone else to come up with a good idea, see if it makes money, imitate the innovator, use monopoly power to grab share of market, and if possible, overwhelm the innovators so they disappear as a future threat to the status quo (ie, MS hegemony).
Why not try to make a search engine that doesn't track what you do?
If a search engine doesn't track you it will give so bad results that you would stop using it.
In the 90's you could type "Linux" into a Microsoft search engine and get half a dozen hits when Infoseek (this was before google...) gave a million-odd.
That was when Microsoft lost me as a customer for their search engines - past, present, future. I really haven't bothered to try Bing and never will.
No sig today...
...the one I made in 2004. We will never see an interesting, competent or remotely innovative product from Microsoft, ever again. It's over, and they will continue to turn the money crank on bloated "enterprise" software for years to come as they slowly but certainly slip into irrelevance. It's the business model. It doesn't matter what engineers they hire. And it's over.
Performance is comparable to Google, but I get the impression that Bing's results are more skewed for people looking to purchase things on line. Try some basic 'how to' queries (how to caulk a window, how to make pancakes, etc) and see if you get more product related hits returned near the top with Bing than with Google.
This isn't a criticism, just an observation. It could be a smart thing for MS. It will help them squeeze more advertising dollars out of smaller market share.
It seems to me that Bing may be a better tool for shopping than Google is, but Google is a better tool for searching than Bing is.
Bing's problem then becomes that there are several better tools for shopping and comparing prices than Bing offers.
The thing is, when it comes to searching, it is almost impossible to complain about the quality of service that Google provides. You may have other issues, like concern for privacy etc, but Google loads quickly and returns accurate results. It does exactly what a search engine should. Google gives no reason for its users to walk away from it.
In the article, it goes on to talk about generic queries and "consumer dialog"; the thing is that these were done by sites like Ask(Jeeves) or whatever and it still got them nowhere. We don't want a search engine to put its own spin on results; we want it to point us to (preferably authoritative) sources of information.
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
Why not try to make a search engine that doesn't track what you do? I'd pay a subscription for such a thing.
No you wouldn't. Seriously, let's be real - you absolutely would not pay for a subscription to a search engine.
And neither would anyone else. Nobody.
There are simply too many free alternatives out there (Google, Yahoo, Alta Vista, etc., etc., etc.) - only a complete and utter twit who was absolutely new to the internet would pay for a subscription to a search engine.
If you're going to suggest a business model, at least suggest one that has some vague, remote possibility of being successful.
That's not true in my experience. I have switched from Google to Scroogle (.org) which doesn't track me. I lose the geolocation (hits from my own country first), but everything else is fine.
"Oppression and harassment is a small price to pay to live in the land of the free." -- Montgomery Burns.
Too much eye candy and/or AJAX nonsense. This is especially true for image search. My laptop chokes trying to render it sometimes. I shouldn't have to buy the lates, greatest, desktop replacement laptop just do do an image search. Also, it seems to want to popup or something when you click an image. Maybe there's a fix for that which doesn't involve dumbing down my IE security settings. I have to admit I never tried it on Chrome, I just gave up on it and went back to Google.
Because of their non-simple interface, I never even got to realize they were missing depth.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Google makes it money with ads. Search is one of their means to display said ads. Kill their search, kill their ads, kill their income, kill them putting more and more of productivity on the web, stop them killing MS products.
MS is not directly intrested in seach, but they are intrested in keeping control over where applications run. The more they can control that, the more they can keep selling their products.
Take gmail. Nobody who wants to be taken serious uses hotmail anymore, but that is not the point. With gmail for businesses, how many companies have lifted themselves OUT of the need for exchange? And with that windows on the server AND with that windows on the client for Outlook?
Gee, all of the sudden you can use a Mac or Linux machine without paying MS a dime. It is being used more and more, and it is not just that lost revenue that MS fears. The more people do NOT uses the latest word/outlook to generate their office documents, the more you as a MS shop cannot do it, because nobody can read your documents.
And that could be the beginning of the end for MS. Not because nobody uses Word anymore to create doc files, but because they use the old version they have, with the format everyone can read.
The biggest enemy of MS is NOT people to stop using their product, but not buying the latest version.
Software after all does not run out. If XP works, you can use it for years to come. Decades even.
Googles Search is not the enemy, it is the income Google has that allows it to launch other products.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
"When developing search engine technology ...." maybe the first problem was calling it "developing search engine technology". Sounds pretentious to me.
Especially compared to what came out.
Stephan
http://stephan.sugarmotor.org
Remember Cuil? They were originally talking about the "long tail"; they wanted to have a bigger index than Google. Cuil is mostly ex-Google people, and they thought they could re-do Google at lower cost.
Didn't help Cuil.
There's ongoing effort in search engine development. Unless you pay close attention, though, it's invisible. A few years ago, around 2007, Yahoo introduced about fifty specialized search sub-engines. These understood weather, stocks, sports, celebrities, movies, and similar popular search topics. They focused on areas that have a strong structure, and need a lookup engine that understands that structure. For about six months, Yahoo was way ahead of Google on such searches.
Didn't help Yahoo. Google implemented something similar and caught up. Now everybody does that.
It's not clear that the Twitter search is a win. Bing announced they were going to do Twitter and Facebook searches, and a day later, Google announced they'd do that too. Google implemented Twitter search, and apparently Bing didn't. Twitter search just seems to clutter up Google results.
In the last year, Google has become much more aggressive about interpreting queries. Google tries hard to infer from the query words what the user is really looking for. This tends to work for popular queries (since it's based on statistics from other queries) and doesn't work too well for unusual queries. For hard queries, you need to use explicit operators ('+' and '"') with Google more than you did a year ago.
The big search engines are still doing badly at de-rating sites which are basically link farms. When you're searching for a product, and you get a hit that's just some site with ad links to other sites, that's a fail. Search for auto parts, and you're likely to get "parts.com", "thepartsbin.com" and "who-sells-it.com", which are just "portals". They don't even return pages that are actually about the part in question. ("thepartsbin.com" pages are all essentially the same, except for keywords inserted for SEO purposes.) Search engines need to look at the business behind the web site. If a business has a million commercial-looking web pages, and a total business volume of a few million dollars, they're probably bogus. That's a part of the "long tail" you don't need to visit.
That's not true in my experience. I have switched from Google to Scroogle (.org) which doesn't track me. I lose the geolocation (hits from my own country first), but everything else is fine.
I think you are sorely missing the point. If Google doesn't track a few users like you that doesn't hurt. If a search engine stopped tracking *everyone* (which is what you are suggesting) then you would get terrible results. Read this blog post if you want to learn more why this data is useful and what you would miss out on.
Surely ignoring the least common queries is the most stupid thing to do... For the most common things, most people generally know where to go anyway... Search engines are for when you're actually looking for something.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
I see what you mean, thanks for the link. So it's a scale between features/convenience/usefulness and privacy. Personally I value privacy higher, and I'm glad I can "opt-out" so to speak.
"Oppression and harassment is a small price to pay to live in the land of the free." -- Montgomery Burns.
with Bing specializing in transactions like plane tickets
Huh?!? A search engine that specializes in plane tickets...
RIGHT.
I don't think Bing will ever out-Google Google. So it's strange that they don't try to identify problems with Google and address them. They seem to start out with the assumption that Google is perfect, so the best path forward is to do everything just like Google, only more so.
But this is what Microsoft does, isn't it? It's what they've always done -- see what the competition does that works, then do the same thing and leverage their Windows monopoly to make it big. Even Windows itself borrowed heavily from Mac OS; DOS before it was a CP/M rip-off. (As an aside: I'm not the only person who noticed how much Windows 7 looks like KDE, am I?) Microsoft almost never the first-mover in anything, and they never were.
But people have recognized this now. It's no longer the cool place for the kids to go work after getting their degrees, so they're having a much more difficult time recruiting the best and the brightest. The people behind their successes have mostly left, and are elsewhere in the industry.
This isn't predicting the "death" of Microsoft -- rather, it's the fall into mediocrity that happens to all large, mature corporations. Microsoft will need a new CEO to break this pattern, and he will be a very different kind of CEO from the kind who got them here.
You're comparing oranges to apples, so to speak.
Shouldn't that be lemons to apples?
Windows->open? Who are you kidding? What gets sent when you start a Windows PC? What goes in the registry that is NOT GUID Based?
Business model not based on data mining? What about Windows (and Office) Genuine Advantage? Just what do they collect from all of those PC's?
And ye shall know the truth, and the truth shall make you free.
John 8:32(King James Version)
I encountered Bing about a year ago after buying a new Dell laptop. By default, IE of course takes you to Bing. When I tried to find a copy of Firefox, Bing directed me to places like "DownloadFirefoxHere.com" instead of mozilla. I don't know if they intentionally suppress competing sites (i.e. mozilla competing with IE), but I have to wonder what sort of adware and other garbage one gets from third party Firefox download sites.
I wouldn't mind if they gave me "as good as but different" results. I draw the line when they give me egregiously bad results, apparently for their own benefit.
Over recent weeks we are seeing more and more about how people within the 'bing' team are all talking about how 'we' are all trying to get along and live along side google... The reality seems to be very very different if you talk to the other microsoft divisions.. If you take all of Steve Balmers comments about wanting to 'kill google' and 'bing will take over' there seems to be a big change in the reality.. I suspect that its all part of the 'poor microsoft story' that's focused on making them out to be the poor underdog when in reality they are investing billions in taking peoples choice away by creating agreements with carriers etc to either include only bing or do other 'techniques' to try and artificially win search transactions and therefore advertising revenue.
I'm not saying that Google does not pay for people to use their search, there are agreements for revenue sharing etc the key is that googles services for the most part are soo good that people want to use them.. bing still has a long way to go despite being an improvement over 'live' search crap.
The NSA (and probably several law enforcement agencies) track your internet usage, and you're worried about Google?
bing seems to think you have an infinite number of search results for everything.
go try a search for something like "inopportuneness" and keep clicking on the "Next" button. You'll eventually get a blank page
"Yeah, Yeah, Yeah." - Lennon, McCartney
Businesses pay for "search engines". When their business depends on the information, you can bet that the price for a massaged engine/database is absolutely a cost of doing business. Lexis/Nexis is in business for exactly that, and I'm sure that there are many others out there, with some perhaps that are quite obscure & specialized.
sr
His ignorance covered the whole earth like a blanket, and there was hardly a hole in it anywhere. - Mark Twain
Microsoft!? Who would trust them to keep your information confidential?
Rethinking email
I find Google ads (the ones that show at google.com) quite usefull. Sometimes even more so than the search results. I guess more people would agree.
Rethinking email
Anything out of Microsoft about Bing is PR with limited accuracy or any usefulness to predicting future value. Bing can not be considered of value so long as MS continues to pump out bogus queries to websites in an effort to falsify stats for the purpose of making website owners believe Bing provides more traffic than it truly does. Of my 4 websites, the number of fake Bing visitors has hit as much as 9% of total traffic. It's unfortunate that Microsoft would rather fake traffic than provide a truly valuable search tool to attract consumers.