Slashdot Mirror

← Back to Stories (view on slashdot.org)

Building a Bigger Search Engine

Posted by ryuzaki0 on from the size-isn't-everything dept.

skreuzer writes "Wired is running a story about a distributed web crawler called Grub. People who choose to download and run the client will assist in building the Web's largest, most accurate database of URLs. This database will be used to improve existing search engines' results by increasing the frequency at which sites are crawled and indexed. Conceivably, Grub's distributed network could enable state information to be gathered on every document on the Internet, each and every day."

1 of 278 comments (clear)

  1. Yeah what's the corporation bashing? by Anonymous Coward · · Score: -1, Flamebait

    I find it hypocritical these posters all have some anti-business bias, but yet we never see any of them refuse a paycheck?

    From the smallest Mom and Pop store to the largest conglomerate, business are what employ people. Socialist whiners do not. If you don't like businesses, QUIT YOUR FUCKING JOB! Don't depend on Business' Teat for your livelihood, be a beggar or something.