Slashdot Mirror

Nah, they'll just build out municipal broadband that's ten times as fast for half the price.

Yeah, because all of the earlier governmental undertaking proved so superior to private enterprises. To wit:

• Public schools, which cost 4 times more today than in the 60ies
• Public roads, which suck by all accounts, particularly in California
• Public transit, which sucks by all all accounts
• And last, but the most germane to the topic, the glorious Municipal WiFi — which sucked so bad, it got abolished by most, who attempted it

Yeah, let's build even more success on that glorious track-record — all the while letting the governments know even more about our online behavior and policing any misbehavior not as ToS-violations, but as civil infractions. What can possibly go wrong?

WebGraph is a framework to study the web graph. It provides simple ways to manage very large graphs, exploiting modern compression techniques. More precisely, it is currently made of:

1. A set of flat codes, called codes, which are particularly suitable for storing web graphs (or, in general, integers with power-law distribution in a certain exponent range). The fact that these codes work well can be easily tested empirically, but we also try to provide a detailed mathematical analysis.
2. Algorithms for compressing web graphs that exploit referentiation ( la LINK), intervalisation and codes to provide a high compression ratio: for instance, the WebBase graph (2001 crawl) is compressed at 3.08 bits per link, and a snapshot of about 18,500,000 pages of the .uk domain gathered by UbiCrawler is compressed at 2.22 bits per link (the corresponding figures for the transposed graphs are 2.89 bits per link and 1.98 bits per link). The algorithms are controlled by several parameters, which provide different tradeoffs between access speed and compression ratio.
3. Algorithms for accessing a compressed graph without actually decompressing it, using lazy techniques that delay the decompression until it is actually necessary.
4. A complete, documented implementation of the algorithms above in Java, contained in the package it.unimi.dsi.webgraph. Besides a clearly defined API, the package contains several classes that allow to modify (e.g., transpose) or recompress a graph, so to experiment with various settings. The package relies on fastutil for a type-specific, high-performance collections framework, on MG4J for bit-level I/O, on the COLT distribution for ready-to-use, efficient algorithms and on GNU getopt for line-command parsing.
5. Data sets for very large graph (e.g., a billion of links). These are either gathered from public sources (such as WebBase), or produced by UbiCrawler.

In the end, with WebGraph you can access and analyse a very large web graph, even on a PC with as little as 256 Mbytes of RAM. Using WebGraph is as easy as installing a few jar files and downloading a data set. This makes studying phenomena such as PageRank, distribution of graph properties of the web graph, etc. very easy.

You are welcome to use and improve WebGraph! Installation

You just have to install the .jar file coming with the distribution, and download the jars WebGraph depends upon (i.e., fastutil, MG4J, COLT and GNU getopt). You may find useful to refer to the JPackage Project if you own an RPM-based distribution. In the same vein of the packages above, WebGraph is also distributed as a Jpackage-like RPM.

WebGraph is a framework to study the web graph. It provides simple ways to manage very large graphs, exploiting modern compression techniques. More precisely, it is currently made of:

1. A set of flat codes, called codes, which are particularly suitable for storing web graphs (or, in general, integers with power-law distribution in a certain exponent range). The fact that these codes work well can be easily tested empirically, but we also try to provide a detailed mathematical analysis.
2. Algorithms for compressing web graphs that exploit referentiation ( la LINK), intervalisation and codes to provide a high compression ratio: for instance, the WebBase graph (2001 crawl) is compressed at 3.08 bits per link, and a snapshot of about 18,500,000 pages of the .uk domain gathered by UbiCrawler is compressed at 2.22 bits per link (the corresponding figures for the transposed graphs are 2.89 bits per link and 1.98 bits per link). The algorithms are controlled by several parameters, which provide different tradeoffs between access speed and compression ratio.
3. Algorithms for accessing a compressed graph without actually decompressing it, using lazy techniques that delay the decompression until it is actually necessary.
4. A complete, documented implementation of the algorithms above in Java, contained in the package it.unimi.dsi.webgraph. Besides a clearly defined API, the package contains several classes that allow to modify (e.g., transpose) or recompress a graph, so to experiment with various settings. The package relies on fastutil for a type-specific, high-performance collections framework, on MG4J for bit-level I/O, on the COLT distribution for ready-to-use, efficient algorithms and on GNU getopt for line-command parsing.
5. Data sets for very large graph (e.g., a billion of links). These are either gathered from public sources (such as WebBase), or produced by UbiCrawler.

In the end, with WebGraph you can access and analyse a very large web graph, even on a PC with as little as 256 Mbytes of RAM. Using WebGraph is as easy as installing a few jar files and downloading a data set. This makes studying phenomena such as PageRank, distribution of graph properties of the web graph, etc. very easy.

You are welcome to use and improve WebGraph! Installation

You just have to install the .jar file coming with the distribution, and download the jars WebGraph depends upon (i.e., fastutil, MG4J, COLT and GNU getopt). You may find useful to refer to the JPackage Project if you own an RPM-based distribution. In the same vein of the packages above, WebGraph is also distributed as a Jpackage-like RPM.

Ok, maybe not quite that, but Ken MacLeod is the best thing to happen to science fiction in a long time. All four of his books are unbelievably great and those not available in the US are well worth special ordering from the UK.

I've written reviews of all of them, available on my web site:

The Star Fracion

The Stone Canal

The Cassini Division

The Sky Road

Note that the Star Fraction is available in bookstores in Canada. A $10 paperback edition is also available in the US via mail order from Laissez-Faire Books

Ok, maybe not quite that, but Ken MacLeod is the best thing to happen to science fiction in a long time. All four of his books are unbelievably great and those not available in the US are well worth special ordering from the UK.

I've written reviews of all of them, available on my web site:

The Star Fracion

The Stone Canal

The Cassini Division

The Sky Road

Note that the Star Fraction is available in bookstores in Canada. A $10 paperback edition is also available in the US via mail order from Laissez-Faire Books

Ok, maybe not quite that, but Ken MacLeod is the best thing to happen to science fiction in a long time. All four of his books are unbelievably great and those not available in the US are well worth special ordering from the UK.

I've written reviews of all of them, available on my web site:

The Star Fracion

The Stone Canal

The Cassini Division

The Sky Road

Note that the Star Fraction is available in bookstores in Canada. A $10 paperback edition is also available in the US via mail order from Laissez-Faire Books

Ok, maybe not quite that, but Ken MacLeod is the best thing to happen to science fiction in a long time. All four of his books are unbelievably great and those not available in the US are well worth special ordering from the UK.

I've written reviews of all of them, available on my web site:

The Star Fracion

The Stone Canal

The Cassini Division

The Sky Road

Note that the Star Fraction is available in bookstores in Canada. A $10 paperback edition is also available in the US via mail order from Laissez-Faire Books

Here's a link to my review.

I mentioned my review of Ender's Shadow on my website. I am pretty much a major SF fan (though you won't find me at cons) and started a series of books reviews earlier this year in an attempt to revive the dead rec.arts.sf.reviews newsgroup. I put them up on my web site as well, and you can see them at:

arenn's SF Book Reviews

To keep it marginally on topic, of course there is a link to my review of Ender's Shadow.

Just posted to the web this week:

Ender's Shadow

I already had trouble getting through, so here's a mirror:

http://www.urbanophile.com/arenn /script-kiddie.txt