Slashdot Mirror

← Back to Stories (view on slashdot.org)

Computing PageRank on your PC?

Posted by CmdrTaco on from the for-the-curious-hacker dept.

An anonymous reader writes "A group of CS researchers of the University of Milan has found a way to compress web graphs at 3 bits per link, and to access them in compressed form. They provide data sets representing real snapshots of portions of the web with one hundred million nodes and 1 billion links. You just need some bandwidth to download a few hundred megabytes of data, and you can compute PageRank with your PC. All the code involved is GPL'd, and the data are public: everybody can grok PageRank now!"

17 of 186 comments (clear)

  1. The major thing missing from Mozilla by Anonymous Coward · · Score: 5, Interesting

    Is a way to look at Google's pagerank. That's the only real thing the IE Google toolbar has over the Mozilla alternative.

  2. This sounds cool.. by xchino · · Score: 5, Funny

    Now if I can just think of a reason why I would need this..

    --
    Everyone is entitled to their own opinion. It's just that yours is stupid.
    1. Re:This sounds cool.. by Daniel_Staal · · Score: 5, Funny
      Now if I can just think of a reason why I would need this..

      And you call yourself a geek. *Sigh*.

      It doesn't matter why you need it. It's technical, GPLed, and has to do with Google. That's all the reason you need.

      --
      'Sensible' is a curse word.
    2. Re:This sounds cool.. by (trb001) · · Score: 5, Funny

      It's technical, GPLed, and has to do with Google

      It's a geek hattrick!

  3. Dumb Question: by Xesdeeni · · Score: 5, Interesting

    What's Page Rank? Does this indicate how often my page is visited?

    Xesdeeni

    1. Re:Dumb Question: by Chris_Stankowitz · · Score: 5, Informative

      Do you mods ever stop to wonder if this guy could have been asking a legit question? Its possible he doesn't know. Also possible that others don't. I know...I know..., this is /. how could he not know right. It is still very possible. I'm not saying he should have been modded up, but by modding him down someone may miss the chance to read his post and reply to it with an intelligent answer. All of that being said. I would answer his question. But now that I think about, I'm not sure what it is. I 'think' I know. But, I think he and I are in the same boat. I also thought about posting this as an AC, but I won't. Then surley someone will just think that it was the original poster posting as an AC. He may be trolling. He may not be. It won't hurt to answer the question.

    2. Re:Dumb Question: by Anonymous Coward · · Score: 5, Funny

      Jesus, you created a second account just to defend yourself!

  4. Tee Hee by teamhasnoi · · Score: 5, Funny
    I bet the Searchking is steaming right about now...

    "Finally, proof!!"

  5. Some webmasters/SEO's are obsessive by Anonymous Coward · · Score: 5, Interesting

    If Google tweaks one thing, causing result 97 to shift to result 98, they notice. They'd be doing this daily to check on their pages.

  6. Which sites are the Root(s)? by amembleton · · Score: 5, Interesting

    When these Web Graph or Page Rank things are drawn up which sites do they use as the roots?

    I mean they've got to start with some site(s) and then go through each link from there.

    1. Re:Which sites are the Root(s)? by warkda+rrior · · Score: 5, Informative

      It is a graph, not a tree, so there is no one root. Maybe you are looking for the seed site, i.e. the first site added to the webgraph they construct. You can choose any site you prefer, although something well-connected is better. It seems to me that Yahoo! would be a good starting point.

      --
      You need to install an RTFM interface.
  7. PageRank is part of Google's algo by Anonymous Coward · · Score: 5, Informative

    It's basically how well linked to your page is, and how well linked to the pages linking to you are, and so on. It's an advanced form of link popularity. The idea is that the more people that link to something, the more influential/important it is. Some sites have high PageRanks of 10 (like Google), while Slashdot is something like an 8. Many pages are in the 4-6 range. Every link you create is like a "vote" for another web page.

  8. I can see it now... by AyeRoxor! · · Score: 5, Funny

    "[...] even on a PC with as little as 256 Mbytes of RAM."

    Somewhere in 1980, milk shoots out of Bill Gates' nose for no apparent reason.

  9. This is good, but... by Prince_Ali · · Score: 5, Funny
    This is good, but I'd rather have the google cache compressed to 3 bits per page.

    "I'll be there in a minute! I'm downloading the Internet!"

    --
    Slashdotter are stupid and biased.
  10. Doesn't actually calculate PageRank? by Vultan · · Score: 5, Informative

    As best as I can tell from the website, the API is only for storing and interacting with a large graph. Nothing there is actually involved with PageRank. You could use this API presumably to write your own PageRank code, but to say "everybody can grok PageRank now!" is misleading at best.

    Moreover, IANAL, but isn't the PageRank algorithm patented by Google? Wouldn't this prevent anyone from releasing GPL code that computes PageRank?

  11. Proof of concept only by Saganaga · · Score: 5, Informative

    I think this project is really just a proof of concept. As another post pointed out, to make this really useful you'd need to regularly update your local data set, which isn't very practical for most people.

    Also, if the downloadable dataset only covers a small portion of the web, how can this system's utility really compare to Google's?

    That said, I think computer science proof-of-concept type project are very useful and serve a valuable purpose in getting the ideas out there for others to improve upon.

  12. Re:can anyone explain what a web graph is? by lordbrain · · Score: 5, Informative

    In a graph is made up of two things, edges and vertices.

    In a web graph, vertices are webpages and edges are hyperlinks.

    PageRank determines how many incoming edges a vertex has. Given the nature of the web, this is a nontrivial problem because a vertex only knows its outgoing edges.

    The assumption for PageRank is that the more incoming edges a vertex has, the more popular it is. So you would use this to figure out how popular a particular vertex is.

    Given this you could do like Google and combine it with a search engine to prioritize the results.

    --

    Thank you. Thank you. Please no applause; just throw money