Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Build a Web Spider On Linux

Posted by kdawson on from the five-eyes dept.

IdaAshley writes, "Web spiders are software agents that traverse the Internet gathering, filtering, and potentially aggregating information for a user. This article shows you how to build spiders and scrapers for Linux to crawl a Web site and gather information, stock data, in this case. Using common scripting languages and their collection of Web modules, you can easily develop Web spiders."

1 of 104 comments (clear)

  1. ATTN: Windows/Linux Refugees! by Anonymous Coward · · Score: -1, Offtopic

    The only thing more pathetic than a PC user is a PC user trying to be a Mac user. We have a name for you people: switcheurs.

    There's a good reason for your vexation at the Mac's user interface: You don't speak its language. Remember that the Mac was designed by artists, for artists, be they poets, musicians, or avant-garde mathematicians. A shiny new Mac can introduce your frathouse hovel to a modicum of good taste, but it can't make Mac users out of dweebs and squares like you.

    So don't force what doesn't come naturally. You'll be much happier if you stick with an OS that suits your personality. And you'll be doing the rest of us a favor, too; you leave Macs to Mac users, and we'll leave beige to you.