Slashdot Mirror

← Back to Stories (view on slashdot.org)

How to Get Rid of Referrer Spam?

Posted by Cliff on from the get-a-net-bouncer dept.

wikinerd asks: "I have recently opened my own community website. Everything was fine until spammers found it, which happened quite quickly. As usual they filled up my mailboxes, but SpamAssassin can take care of that when it is needed. Then, they discovered my blog and my wikis and employed their bots to fill them up with spam comments. I solved this problem by moderating all comments. Now, however, they employed another evil trick: Referrer spam. They caused my webserver statistics to grow up by orders of magnitude by making their stupid websites to show up on my referrer lists. Unfortunately now my webserver usage statistics are full of viagra, poker, casino, porn, spyware, and pharmacy sites. I am afraid that this is a problem I cannot solve with the knowledge and the tools I have at the moment. So, I came here to ask Slashdot readers: How can I fight referrer spam and what tools are available in a GNU/Linux environment to ensure clean and spam-free usage statistics?"

1 of 56 comments (clear)

  1. Here's how for Apache by Anonymous Coward · · Score: 5, Informative

    I'll assume you're using Apache and have access to the .conf, or someone that does.

    First, you need to setup the log you'll use for statistics to exclude requests marked with a "nolog" environment variable.

    CustomLog logs/access_log-www.example.com combined env=!badreferer

    The following requires Apache's SetEnvIf module. You can put these lines in .conf, or even in .htaccess so you can change them without a restart. If you don't have/want SetEnvIf, you can also use mod_rewrite (E=badreferer:1 at the end of your RewriteRule) to do the same thing.

    #Blacklist (adjust as you need)
    SetEnvIfNoCase Referer ".*(credit|hold-em|holdem|mortgage|money|cash|gb.c om|4free|teen|pussy|discount|inkjet|fuck|hasfun|ca sino|gambling|poker|porn|sex|paris|nude|xxx|hilton |adminshop|devaddict|iaea|peng|just-deals|pisx|tec rep-inc|learnhow|phentermine|terashells|psxtreme|f reakycheats).*" badreferer
    #Whitelist (optional)
    SetEnvIfNoCase Referer ".*(google|yahoo|alltheweb|search|excite|aol.com|l ycos|msn|altavista|XXXX).*" !badreferer

    Additionally, you can use the same blocks to deny them access to your site:

    <Limit GET HEAD POST>
    Order Allow,Deny
    Allow from All
    Deny from badreferer
    </Limit>

    <LimitExcept GET HEAD POST>
    Order Deny,Allow
    Deny from All
    </LimitExcept>