Slashdot Mirror

← Back to Stories (view on slashdot.org)

WebCrawler Turns 10 Today

Posted by ryuzaki0 on from the young-at-art dept.

Brian Pinkerton writes "WebCrawler, one of the first search engines on the 'Net, turns 10 today. You can read a short history of WebCrawler. When I wrote WebCrawler, one could do a credible job of crawling, indexing, and searching the Web from a single desktop PC. Today, the reality is a little bit different."

1 of 136 comments (clear)

  1. Webcrawler is a crawler no more.... by Camel+Pilot · · Score: 1, Redundant

    I just did a search on webcrawler for "digital camera" and the results where 70% pay-per-click advertising with a small and hardly noticable "Sponsored by:" disclaimer. Worse yet the paid links are intermigled with the indexed hits.

    Looks like Webcrawler is now more of a pay-per-click dispensor than a search engine... No thanks!

    I think google has done a good job of clearly identified what is relevant and what is paid for.