Slashdot Mirror
Modeling Linking on the Web
An Anonymous Coward writes "Amazon has a much greater market share among online bookstores compared to the greatest market share for offline stores. How is this possible? Because the web changes how people find information. There are millions of links to Amazon on the web, which makes it more likely for people to find Amazon when surfing the web, or when using search engines which typically use link popularity in ranking. This makes it harder for new businesses to compete. Researchers have discovered that across the entire web, links are distributed according to a "power law" which leads to "rich get richer" or "winner's take all" behaviour where a small number of sites get the vast majority of links and traffic. A new study just released by NEC shows that this behaviour varies in different communities, and shows how to predict competition in different areas. For example, you can see how much tougher competition is among booksellers compared to photographers."
This is true of Webcomics as well. Ask someone what their favorite Webcomic is, and they will almost invariably respond with one of the following: User Friendly, Penny Arcade, PvP, Sluggy, Sinfest, Megatokyo or Exploitation Now. With the exception of Penny Arcade, I have found the total combined quality (art + writing + humor) to be fair at best, and atrocious at worst (guess what the worst is; hint: think of a little dustball with feet). But these sites are linked to from all over, and they often link to each other, creating "flash crowds" from Slashdot, other comic sites, personal home pages, etc.
There is a class of "second tier" comics which have nice little followerships: Little Gamers, Sexy Losers, Polymer City and Cool Cat Studio (really, any Keenspot comic that isn't Sinfest or EN) are among these. Everyone else, myself and my comic included, is "third tier", i.e., tumbleweeds rolling across their allotted server space.
Then there is Pokey, which stands conspicuously on its own. HOORAY.
N4st0r, trixx0r h0bb1tz0rz! Th3y st0l3 0ur pr3c10uzz!
The difference between a Pareto distribution and a power law distribution is that in a Pareto distribution, the probability P[X > x] ~ x^-k, (that is, the probability that a observed value is greater than x is proportional to the inverse power of x) whereas a power law is P[X == x].
And a Zipf law is a power law on ranks, rather than values.
Lada Adamic of HP has an excellent how-to on power law distibutions you might find interesting.
Is called VisIT. It produces a graphical representation of how sites link together, based around any given query. It was used quite sucessfully to demonstrate how Scientology had spammed Google, by creating multiple domains all linking back to their main web page.
It's a freebie download and you can get it here.
Alas gallinaceas de urbe bovis volo
There are a quite few papers on this topic (behaviour of disordered networks) by Barabasi and one of his research students, Reka Albert (now probably graduated), most of which are available from his research group's website or from arXiv.
Particular highlights:
A-L. Barabasi and R. Albert, Emergence of scaling in random networks, Science 286, 509, (1999)
A-L. Barabasi, R. Albert and H. Jeong, Scale-free characteristics of random networks: The topology of the World Wide Web, Physica A 281, 69-77 (2000).
A-L. Barabasi and R. Albert, Topology of evolving networks: local events and universality, Physcal Review Letters 85 5234 (2000).
This work is an interesting counterpoint to the 'small world' networks of Watts and Strogatz:
D.J. Watts and S.H. Strogatz, Collective dynamics of 'small-world' networks, Nature 393, 440-442, (1998).
D.J. Watts, Small Worlds, Princeton University Press, (1999).