Building a Bigger Search Engine

skreuzer writes "Wired is running a story about a distributed web crawler called Grub. People who choose to download and run the client will assist in building the Web's largest, most accurate database of URLs. This database will be used to improve existing search engines' results by increasing the frequency at which sites are crawled and indexed. Conceivably, Grub's distributed network could enable state information to be gathered on every document on the Internet, each and every day."

Comments

And in related news . . .

[ubernostrum]

The architects of the GRand Unified Bootloader posted to the mozillazine forums today, flaming the choice of the name "grub" for this new system and calling for spamming of all grub-related discussion boards in retaliation.

Or not. What a difference maturity makes.