Search Engine Results Relatively Fair

perkr writes "The Economist and PhysicsWeb report on a study from Indiana University claiming that search engines have an egalitarian effect that gives new pages a greater chance to be discovered, compared to what would be the case in the absence of search engines. Based on an analysis of Web traffic and topology, this result contradicts the widely held 'Googlearchy' hypothesis according to which search engines amplify the rich-get-richer dynamics of the Web."

Comments

My experience bears this out also

[Analogue Kid]

I've made sites with fairly mainstream content before, which were totally ignored by google. But then, I put an article on my blog about the history of a certain group of elite English schools in Taiwan. Previously, this information had not been on the internet anywhere. Now, if you type the name of the original school of that group (Modawei) into google, my article comes up #1.

Let's try a thought experiment...

[NickFortune]

Suppose there were no search engines.

Most web newbies would form their impressions of the web from their ISP's portal site. That would give a lot of power to corps like AOL, who for a long time tried to persuade their subscribers that there was no web outside of AOL hosted content.

There might still be blogs and social networking sites, but the take up would be slowed since fewer people wold have heard of them, and both might have failed to ignite into the movement we see today.

Which would probably mean that if you wanted something outside of the main ISP channels, you'd be reduced to digging through the spam on USENET to find it.

Google as an egalitarian influence on the web? I think it's a bit of a no-brainer, personally.

Impact of Search Engines on Page Popularity

[Omar El-Domeiri]

There is another paper out of UCLA that is similar to this one except with somewhat opposing results. In which, the authors show analytically that the rich-get-richer phenomenon does exist. http://oak.cs.ucla.edu/~cho/papers/cho-bias.pdf

It seems tough to reconcile these two sets of findings, and this new paper even makes mention of this:

"The connection between the popularity of a page and its acquisition of new links has led to the well-known rich-get-richer growth paradigm that explains many of the observed topological features of the Web. The present findings, however, show that several non-linear mechanisms involving search engine algorithms and user behavior regulate the popularity of pages. This calls for a new theoretical framework that considers more of the various behavioral and semantic issues that shape the evolution of the Web. How such a framework may yield coherent models that still agree with the Web's observed topological properties is a difficult and important theoretical
challenge."

Re:An example of poor Google performance

[baadger]

Too true. How about some serious search innovation from Google?

- Effective (but switchable) web spam filtering, as parent mentions.
- The ability to search for strings like "-x flags" (note the quotes) and actually get meaningful results.
- More complex patterns (mathematical expressions, anyone?)
- Sort search results by the date pages were modified, they were discovered by Google? (useful in circumstances when you're looking for the latest information on a topic).
- Semantic sensitive search bots.
- Better results for filetype: operator. Why can't Google index all major filetypes even if it can't make them searcheable?

Anyone got any others?

Google could be working constantly behind the scenes on their engine but perhaps they should start making more noise about it. When was the last time Google's web search engine trod some new ground? Or any search engine for that matter (I refer to Google because they are 'innovating' so much).