Slashdot Mirror
MSN Search Engine Favors IIS
Scud writes "It appears that if you want to rise up in the rankings over at the MSN search engine you would do well to host your page on IIS. Ivor Hewitt has done a study and it appears that by using IIS, you are likely to increase your odds of a higher listing by several percent."
It's clearly biased towards Internet Explorer too, the results I get back in Firefox are mostly irrelevant blogs and pages full of adverts.
"Those who cast the votes decide nothing; those who count the votes decide everything." (attrib. Joseph Stalin)
So what's going on? I have no idea, I doubt it's all a big conspiracy... but some possible explanations spring to mind: Perhaps the MSN search has simply been coded by developers used to talking to IIS machines and so it just does that job better? Perhaps the MSN spider is taking advantage of some specific IIS features to provide enhanced indexing?
In other words, there are some explanations out there other than "MS is biased and there's a conspiracy and they are trying to take over the world"...
One man's Funny is another man's Offtopic.
... to think ms wouldn't use all it has. Obviously it hasn't yet learned from google, that being evil is bad. And bad guys get punished.
Is there any truth to the rumor that having a picture of Bill Gates on your site makes you #1 in your category?
I'm a big tall mofo.
... and they still think they can beat Google to the game. When are they going to realize that what made Google so successfull was the fact that is has been so unbiased in all ways imaginable, including not accepting payments to get higher rankings.
Google makes money by prioritising quality. Microsoft makes money by prioritising money.
Go figure.
My page is titled "San Andreas Radio" and if you Google it, comes out #1 or #2 every time.
MSN it and it comes out about #7. Either they're being paid to reduce its rank (it's a bit subversive), or they don't like the fact I'm hosted on Linux, or they simply don't have a very good search engine.
If I put the exact unique title of a page into an engine, I expect that page to be #1.
To be conclusive, it needs to be a controlled experiment with the same text and same outgoing/incoming links.
Just the webserver alone changing. This can happen by taking a popular site and then changing what it reports to the MSN search robots.
But until such an experiment is done, the data is open to too many interpretations.
My site is first or nearly first in google using relevant search terms. But in MSN it never shows (even if listed). Maybe also the use of PHP is harmful for MSN ranking? M.
For years now, the company where I work has had all it's Apache systems reporting that they are IIS 5.0 systems. Just a quick change in a single file before compiling and there you go!
IMO the true test would be to take his site which is hosted on Apache, move it to being hosted on IIS and watch and see if his ranking goes up or down after the next time it is indexed.
Do you Gentoo!?
- Crow T. Trollbot
Just change the server response line if the GET or POST comes from Redmond, WA to say you are some version of IIS. I can already see the recommendations coming from the SEO folks.
Most of those useless keyword, domain parking/hijacking, and spam sites out there run on Linux+Apache because the owner can host thousands of those domains fairly inexpensively, and that's the key to all spam: minimization of operating expenses so you only need 1 out of 100,000 users to click/buy to turn a profit.
These sites don't have any real content, they just point to other sites and/or exist to spam you with advertisements. Some of them have googlebombed their way higher into the rankings.
My guess is that MSN does a slightly better job of filtering those useless sites out of the index at the present time, OR the "googlebombing" techniques they use aren't as effective with MSN's indexing. Since they almost exclusively use Apache that would have the false appearance of favoring IIS.
This is just a guess, but it seems plausable.
Natural != (nontoxic || beneficial)
MSN Search should be banned for being dishonest.
Banned from where and by whom?
MSN search can do whatever they like. I don't know anybody who actually uses it. Even non-tech oriented people that use IE (against recommendations) set their startup page to something else. Google, mostly, but also "My Yahoo" and their webmail or portal of preference.
No sig
...Googled for anything using MSN!
If you disagree with me on social issues, then it's pretty clear that you are a narrow-minded bigot.
- Google 7873
- Yahoo 3163
- MSN 199
- AOL 65
- Dogpile 44
- Unknown 41
- Earth Link 28
- AltaVista 16
- Excite 14
- A9.com 9
- Others 77
...which comes out to about 2% MSN.
And then you would also need to move it _back_ to Apache to see if the ranking declines again.
I thought moving it back to apache would be the natural thing to do after running a webpage on IIS for a short while... *shudder*
Do you Gentoo!?
Remember, the guys working on the MSN search engine certainly use IIS to host their intranet sites, and whatever internal webservers they use to test against are probably IIS as well, at least in the most cases. They are likely to consider bogus results for their own sites (both internal and external) more critical... that's not malice, that's just human nature. Even if they consciously work against that, they're more likely to notice problems there first.
And search engine tweaking is more an art than a science. It's an evolutionary process, with feedback loops and strange attractors. So if there's any difference in the behaviour or design of Apache or IIS that would be visible to a search engine, it's likely to lead to a slight bias in favor of the server software that the servers they pay more attention to run.
Leads me to think: is it significant? That is, can we exclude (to a reasonable certainty, that is, p>0.95) the possibility that the effect seen cannot be attributed to chance or some other criterion MSN uses?
Ivor says at some point The initial set of words indeed showed a significant difference between the results from Google and the results from the Beta MSN search..
But what does he mean? I would be interested in what kind of significance test was applied, what the exact results were. Just looking at the ratio of percentages doesn't tell me enough... One should go back at the original data (seems provided, good) and check if the effect is actually trustworthy or just, in Ivor's words, "Odd. Pure coincidence perhaps."
Before seeing some analysis of significance, I don't believe anything...
Have they gone ahead and implemented that thing about assigning you a hosting provider at birth then? What a shame. Back in my day, we used to be able to pick our hosting provider based on what they provided and what they charged for it.
Ah, the good ol' 1900's.
--MarkusQ
In other words, there are some explanations out there other than "MS is biased and there's a conspiracy and they are trying to take over the world"... "
It's called plausible deniability. "Why, no, we had no idea this would happen. You say it's an interaction with an IIS feature that causes this to happen? Heavens to Betsy, we never thought of that."
Microsoft people aren't stupid, and they ARE trying to take over the computer world, or haven't you been paying attention to what they say and what they have done? The engineers that built MSN Search would certainly be aware of any interaction that fits with IIS features to provide enchanced indexing. They would have been all over it from the beginning. And a side-effect means that IIS sites come out higher? Great! It's a feature that benefits us, they would think.
Of course MS is biased. Of course they would have noticed this. Of course they like it.
Then, in 2007, came Longhorn, with integrated web search using not Google, but MSN. Joe Sixpack didn't care, but MSN was so damn convinient he forgot about Google - effectively forcing Google Inc. with its costly development department out of business. Later - oh surprise - all results you got for "Linux" on MSN were advisories to ditch it for Windows. He who controlled the search result, controlled the industry. (Maybe I should put some fake Frontpage-Meta-Header to my webpages to increase Rankings on MSN ... just to be sure)
Screw the FSM - Real geeks believe in the Invisible Pink Unicorn
Believe it...
First off, I looked at the difference in means for Apache rankings in MSN and Google. 61.5% (MSN) vs. 64.3% (Google) for 970 observations Right there, you ought to be able to eyeball it and see significance. But, to make sure, here are the results of a t-test which checks the likelihood that two matched sets have different means (forgive the crappy formatting):
M G
Mean 0.615061856 0.642948454
Variance 0.01100624 0.008740111
Observations 970 970
Hypothesized Mean Difference 0
df 969
t Stat -10.51551356
P(one-tail) 7.26569E-25
t Critical one-tail 1.646427658
P(two-tail) 1.45314E-24
t Critical two-tail 1.962415113
As you can see, the P is 1.45 x 10^-24, which at least makes us think the results are not pure coincidence. I don't intend on speculating on the causality, though...
Add something like this pseudocode to your server:
if $Browser = "MSNSearchBot" then $Server = "Microsoft-IIS/6.0"
One line blog. I hear that they're called Twitters now.
Try typing "online music".
On Google the top two references are iTunes and iTMS. On MSN you'll have to go through a few pages before you'll see anything about iTunes.
Yeah, I trust Microsoft to provide unbiased search results. Sure I do.
m.m.