Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Build a Web Spider On Linux

Posted by kdawson on from the five-eyes dept.

IdaAshley writes, "Web spiders are software agents that traverse the Internet gathering, filtering, and potentially aggregating information for a user. This article shows you how to build spiders and scrapers for Linux to crawl a Web site and gather information, stock data, in this case. Using common scripting languages and their collection of Web modules, you can easily develop Web spiders."

3 of 104 comments (clear)

  1. Hmm... by joe_cot · · Score: 5, Funny

    Yes, but does it run on ... damn.

  2. The 90s called by dave562 · · Score: 5, Funny

    They want their technology back.

  3. Re:yes, I did RTFA by Faylone · · Score: 4, Funny

    You RTFA? Are you sure you're in the right place?