Slashdot Mirror

← Back to Stories (view on slashdot.org)

How the Wayback Machine Works

Posted by ryuzaki0 on from the very-big-hard-drive dept.

tregoweth writes: "O'Reilly has an interview with Brewster Kahle about how The Internet Archive's Wayback Machine works, with lots of juicy details about how the biggest database ever built works."

2 of 134 comments (clear)

  1. Google? by kenneth_martens · · Score: 4, Interesting

    It's an interesting idea, but the real problem is not storing the 100 TB of data, it's figuring out how to search through it to find what you're looking for. Now, apparently they write a lot of their own software, but it might be better if they could team up with Google and have Google index their sites on a special database. We'd have www.google.com for regular searches, and wayback.google.com for the Wayback Machine's sites.

    Something else I found interesting: according to the article, they "use as much open source software as [they] can." That makes sense when they've got between 300 and 400 computers, and with the number growing all the time. Licensing all those with a non-open OS would be quite expensive.

  2. Try this instead.. by CptnHarlock · · Score: 4, Interesting

    http://web.archive.org/web/*/http://slashdot.org

    --
    $HOME is where the .*shrc is
    -- silver_p