Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Build a Web Spider On Linux

Posted by kdawson on from the five-eyes dept.

IdaAshley writes, "Web spiders are software agents that traverse the Internet gathering, filtering, and potentially aggregating information for a user. This article shows you how to build spiders and scrapers for Linux to crawl a Web site and gather information, stock data, in this case. Using common scripting languages and their collection of Web modules, you can easily develop Web spiders."

2 of 104 comments (clear)

  1. Hmm... by joe_cot · · Score: 5, Funny

    Yes, but does it run on ... damn.

  2. The 90s called by dave562 · · Score: 5, Funny

    They want their technology back.