Slashdot Mirror
Search Engine Results Relatively Fair
perkr writes "The Economist and PhysicsWeb report on a study from Indiana
University claiming that search engines have an egalitarian effect
that gives new pages a greater chance to be discovered, compared to
what would be the case in the absence of search engines. Based on an
analysis of Web traffic and topology, this result contradicts the
widely held 'Googlearchy' hypothesis according to which search engines
amplify the rich-get-richer dynamics of the Web."
First of all any time you want to analyze Google, you have to realize that they've had ten PhDs crunching the problem already for years. Google is designed to give the best results for whatever its users are searching, thus any apparent bent towards egalitarianism, monopolism, antidisestablishmentarianism, or what-have-you, is purely incidental.
If you're searching for something obscure, Google will instantly tell you the one startup company building it. On the other hand, if you want something mainstream, they'll give you a prioritized list of the best sources. There's no alterior motive it seems - they just give you what you searched for... imagine that! I've seen a business through from obscure geek hack to the mainstream consumer, and Google has been there at every step of the way, working exactly as users expect. To accuse them of favoring any particular stratum of that chain is awfully unfouned IMHO unless there are some specific examples. Indeed, answering users' needs instead of pandering to the status quo seems to be he most valuable bit of what google does.
I've had worldwidewingtour.com live for about 3 days and I have a good google ranking. Even a search like "hooters wing tour" places me at number 7 on google.
Those who know, do not speak. Those who speak, do not know. ~Lao Tzu
I've established a number of websites primarily for small groups of users, and every one of them has been ranked, even one set up for a friend strictly to put up family pics for his brother to see. If it's out there it's googlable. And no, I don't care if it's not a word :)
I've made sites with fairly mainstream content before, which were totally ignored by google. But then, I put an article on my blog about the history of a certain group of elite English schools in Taiwan. Previously, this information had not been on the internet anywhere. Now, if you type the name of the original school of that group (Modawei) into google, my article comes up #1.
I'm a gnu world man.
Google only updates their publicly visible Pagerank data every couple weeks or so, to make it more difficult to game the system. New pages will show up as PR 0 until the next public update, but (of course) Google updates their private Pagerank database more often. That being said, WTF?! World Wide Wing Tour?!
Most web newbies would form their impressions of the web from their ISP's portal site. That would give a lot of power to corps like AOL, who for a long time tried to persuade their subscribers that there was no web outside of AOL hosted content.
There might still be blogs and social networking sites, but the take up would be slowed since fewer people wold have heard of them, and both might have failed to ignite into the movement we see today.
Which would probably mean that if you wanted something outside of the main ISP channels, you'd be reduced to digging through the spam on USENET to find it.
Google as an egalitarian influence on the web? I think it's a bit of a no-brainer, personally.
Don't let THEM immanentize the Eschaton!
There is another paper out of UCLA that is similar to this one except with somewhat opposing results. In which, the authors show analytically that the rich-get-richer phenomenon does exist. http://oak.cs.ucla.edu/~cho/papers/cho-bias.pdf
It seems tough to reconcile these two sets of findings, and this new paper even makes mention of this:
"The connection between the popularity of a page and its acquisition of new links has led to the well-known rich-get-richer growth paradigm that explains many of the observed topological features of the Web. The present findings, however, show that several non-linear mechanisms involving search engine algorithms and user behavior regulate the popularity of pages. This calls for a new theoretical framework that considers more of the various behavioral and semantic issues that shape the evolution of the Web. How such a framework may yield coherent models that still agree with the Web's observed topological properties is a difficult and important theoretical
challenge."
Search Engines give new pages a greater chance to be discovered
This just in - Yellow Pages give new businesses a greater chance to be
discovered.
Some kind of cypher communication, using /. as a way to exchange data pretty anonymously? It's certainly interesting to observe that each block is repeated once, the ones between the "HELLO WORLD" maybe identifying a key, though I don't believe in a complex scheme (I'd bet you can en- & decode this thing without using a sheet a paper even), probably just subtract them. Any crypto-gurus around, I'm not interested in wasting my time on this one? Could be just trolling :-P
Here's how: the wealthy get to decide who receives their spending, and those people in turn decide how strongly to weight their suppliers' votes in the allocation of resources. This perpetuates through in a cycle that reaches a very rough, shifting equilibrium that very much resembles Google's "pagerank", IMO.
Compared with outright hierarchy, this kind of inequality is still going to appear relatively fair, but it doesn't measure up to equally weighted votes. That is, it isn't democratically fair. However, this, or at least some inequality appears to be essential to making useful discrimination, if you're going to use the "intelligence" of the web itself to do it. Ideally, the results would be based upon the quality of the content itself, no matter how obscure, but the artificial intelligence required to do that would be mind-boggling.
Besides, people often want to find something that they were surfing the other day (ie. relatively more likely to be strongly linked), or else read up on what others are talking about, so that they need the same points of reference... An objectively better site might actually be inferior for socialising with one's peers, or engaging in political tribal virtual warfare: a third point of reference in such cases leaves you out of the discussion!
Wikileaks, no DNS
Too true. How about some serious search innovation from Google?
- Effective (but switchable) web spam filtering, as parent mentions.
- The ability to search for strings like "-x flags" (note the quotes) and actually get meaningful results.
- More complex patterns (mathematical expressions, anyone?)
- Sort search results by the date pages were modified, they were discovered by Google? (useful in circumstances when you're looking for the latest information on a topic).
- Semantic sensitive search bots.
- Better results for filetype: operator. Why can't Google index all major filetypes even if it can't make them searcheable?
Anyone got any others?
Google could be working constantly behind the scenes on their engine but perhaps they should start making more noise about it. When was the last time Google's web search engine trod some new ground? Or any search engine for that matter (I refer to Google because they are 'innovating' so much).
It looks more like something which is trying to look like a cypher. The repeated blocks are redundant, nothing more.
Perhaps it is a social experiment, designed to draw out paranoid theories.
(Accepts pat on tinfoil hatted head)
http://michaelsmith.id.au
If you RTFA you'll notice some of the arguments against it.
But beyond that, common sense alone tells you winner takes all, and it continues to be that way, with google or with anyone else.
The entire pageranking algorythm is there to point you to the most likely result you're looking for. They base that on popularity, number of links coming in, and the importance of the referring sites linking you. The net effect is, the more popular you are, the more relevent you become and the higher ranked you are.
Also, when you type in say "windows" Google automatically assumes you're talking about the Windows OS. What if you were looking for real windows? The search engines are always assumming based on popular demand. This steers people's thoughts and pushes them in a non-neutral direction. As a word's context changes to favor a certain direction, search engines rank that as more relevent, which leads to it being more favorable, etc. Cycle repeats.
eTrade SUCKS
It makes no attempt to filter spam, which like email will soon account for about 80% of content.
Not true at all. Insider screenshots of Google's special internal interfaces to employees show that they actually have a human driven spam filtering services. They basically display a page at a time and have the user rate how likely they think it is spam. I can't remember where I saw the screenshots, so I can't find them.
When Google claims to have 28,600 results, in fact there are only 36. Now that's a con.
Did you read the disclaimer at the end? Google excludes many results if they're too similar. You can click that link at the end to show all of the 28k+ results if you like, but it would be rather pointless. This tends to happen when google indexes things like web forums. Because of the way links to forum threads work, you get a lot of overlapping content, and google is simply smart enough to identify it as overlapping content and just count it as one hit.
Mod (-5) Google bashing. No. This applies to all the search engines.
You'll probably be moderated down for spreading misinformation and not actually bothering read this message at the end: "In order to show you the most relevant results, we have omitted some entries very similar to the 22 already displayed. If you like, you can repeat the search with the omitted results included." If those are spam as you say, I think google did a good job by omitting many thousands of spam sites.
Of course. Doesn't anyone remember what the web was like before search engines became popular, when the main way to find pages was by following links there from other pages? If you could get someone to link to your page who in turn was listed prominently on the Humor, Jokes, and Fun page on akebono, then you were all set, but otherwise, it would take *months* for anyone to find out about your page, if they ever did.
Don't even bother replying to this unless you know the significance of akebono.
Cut that out, or I will ship you to Norilsk in a box.
So... what's your point?
It's also not Egalitarian because Egalitarianism assumes all people are equal, so company of 100,000 employees is 10,000 more important than a company of 10 employees.
Except corporations are legal entities in their own right (hell, some courts even grant them human rights!) and therefore only count as one, and not as the sum of theiur employees. That's always assuming you want to analyse the matter from the corporate perspective. Personally, I don't think is the most useful approach.
Think in terms of people who want to find the web pages best suited to their requirements (as opposed to the narrow range of pages the cartels want to push) and you'll find those individuals have much freer access to the information they seek. Similarly, consider the individuals who publish the web pages and who have a greater chance of having their pages read when people search using Google. I think you'll find Google an egalitarian influence from both those perspectives,
It's more like positive discrimination, you discriminate against big companies for your own benefit and pretend its for some greater moral purpose.
Bizzarre. Weren't you earlier defending the right of a company to run it's own business as best suits its business model? Or does that only apply when other corporations can pay money to distort the listings to their own gain? Anyway, it's like the corporate shills are always saying: they're free to set up their own engine and take their advertising revenue elsewhere.
Don't let THEM immanentize the Eschaton!