Slashdot Mirror

← Back to Stories (view on slashdot.org)

Robotcop: It's the Law

Posted by michael on from the you-have-ten-seconds-to-comply dept.

Voivod writes: "Inspired by the recent Slashdot and Evolt discussions about Blocking Bad Spiders, we set out to write an Apache module that solves this problem. The result is Robotcop and it's ready for action. We believe that it's the best solution to protecting Apache webservers from spiders currently available. Install it and help us make life hell for e-mail harvesting software!"

1 of 54 comments (clear)

  1. Re:What about spoofing spiders? by Voivod · · Score: 4, Informative

    You just put hidden links in your HTML which only a spider's HTML parser would notice and follow. This technique is already widely used by wpoison which is a Perl CGI solution to the spider problem.

    Check out the robotcop.org site. It has examples of how to set all this up.