Slashdot Mirror

← Back to Stories (view on slashdot.org)

Computing PageRank on your PC?

Posted by CmdrTaco on from the for-the-curious-hacker dept.

An anonymous reader writes "A group of CS researchers of the University of Milan has found a way to compress web graphs at 3 bits per link, and to access them in compressed form. They provide data sets representing real snapshots of portions of the web with one hundred million nodes and 1 billion links. You just need some bandwidth to download a few hundred megabytes of data, and you can compute PageRank with your PC. All the code involved is GPL'd, and the data are public: everybody can grok PageRank now!"

7 of 186 comments (clear)

  1. The major thing missing from Mozilla by Anonymous Coward · · Score: 5, Interesting

    Is a way to look at Google's pagerank. That's the only real thing the IE Google toolbar has over the Mozilla alternative.

  2. Dumb Question: by Xesdeeni · · Score: 5, Interesting

    What's Page Rank? Does this indicate how often my page is visited?

    Xesdeeni

  3. Some webmasters/SEO's are obsessive by Anonymous Coward · · Score: 5, Interesting

    If Google tweaks one thing, causing result 97 to shift to result 98, they notice. They'd be doing this daily to check on their pages.

  4. Which sites are the Root(s)? by amembleton · · Score: 5, Interesting

    When these Web Graph or Page Rank things are drawn up which sites do they use as the roots?

    I mean they've got to start with some site(s) and then go through each link from there.

  5. Google patents? by PaulBu · · Score: 4, Interesting

    All the code involved is GPL'd, and the data are public: everybody can grok PageRank now!

    GPL'd? Hmm, I thought that Google did patent the PageRank algorithm (correct me if I am wrong), so re-implementing THEIR algorithm even more efficiently would be incompatible with GPL. OTOH, if it is not THEIR algorithm, it can not be called 'PageRank'
    Oh, the evils of software patents...
    Paul B.

  6. I wonder... by crashnbur · · Score: 4, Interesting

    ...how this can be used to discover the percentage of broken links on the web at any given moment in time.

  7. Re:why is rank/rating necessary? by baka_boy · · Score: 4, Interesting

    I really shouldn't rise to this bait, but I can't resist: yes, given the choice between those networks, I would choose PBS. Just as I would take a non-profit-driven Internet, public radio over Clear Channel and its ilk, and community mesh wireless networks over 3G mobile phone service.

    Google has been, so far at least, a rare exception in the world of privatized communications utilities, by consistently showing a amazing lack of intention to lock people into their service, using either exclusivity agreements of some sort or the simple expedient of proprietary technology (i.e., "increase your PageRank by 10% if you support new encrypted GoogleML tags on your site!"). Nothing is permanent, though, and as we all know, single points of failure are a no-no.

    So, to bring all this back somewhere in the general neighborhood of the main story: further distributing the capability to build "mini-Googles", or specialized, community-maintained (but still fairly large-scale in terms of number of pages and links indexed) search tools is very interesting, and a useful body of technology to perpetuate.

    Or, even more generally, the technology needed to do large-scale storage, analysis, and manipulation of directed graph structures is a very useful tool. Software analysis often relies heavily on large graphs showing dependencies, caller-callee relationships, variable accesses, etc., as do any number of AI subdomains like knowledge representation and planning systems.