Internet Archive Opens Crawler Code Under LGPL

Posted by Cliff on Wednesday January 7, 2004 @03:40AM from the preserving-our-digital-culture dept.

ramakant writes: "It looks like the Internet Archive, which hosts the infamous Wayback Machine has opened its newest in-development crawler code under the LGPL. From the announcement: 'Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix , or misspelled or missaid as heratrix / heritix / heretix / heratix) is an archaic word for inheritess. Since our crawler seeks to collect the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt.'"

1 of 186 comments (clear)

Min score:

Reason:

Sort:

Unless the Archive caves in... by turambar386 · 2004-01-07 04:25 · Score: 5, Informative

"Since our crawler seeks to collect the digital artifacts of our culture for the benefit of future researchers and generations..."

That is, unless the digital artifacts in question are, like Operation Clambake opposed to rich and powerful sects. In which case, they are blocked by the Wayback machine after the Archive caves in to DMCA notices.