Slashdot Mirror
Extending SpamAssassin and Amavis
An anonymous reader writes "Spam filtering solutions are a necessary evil in today's e-mail climate. There are many different tools and systems available for the filtering and removal of spam e-mail. Tools like SpamAssassin and more detailed agents, such as Amavis use a variety of different methods to identify and capture spam. An IBM article shows how you can extend SpamAssassin and Amavis, providing additional filtering facilities to lower the amount of spam hitting e-mail boxes."
bwah. dan, jucam o dota?
I recently developed a new technique to completely stop spam from reaching my inbox. I reject all incoming mail. Yes, there are obviously some false positives with this technique, but here's the great part, the more spam they send me, the lower the percentage of false positive I get!
Or you could just use an email provider like GMail that has good spam filtering and let them worry about it.
I know this solution isn't fit for "average user" but I use white list aliases, when I write my email in some website or whereever, I do @domain.com , as soon as I start to get SPAM, two things happen :
1. I know WHERE the SPAM is comming from and who rat my email out.
2. I can close that specific alias and no more SPAM.
Using the above technique, I see ZERO SPAM... like good old days of emailing.
Anyone that's done much exploration of spam filtering already knows the basic architecture of self-learning filters. This article has nothing new.
Recently I've implemented greylisting on my mail server. The drop in spam has been enormous, though there have been a couple cases where email didn't go through on the first try.
Essentially it's a step inserted into the SMTP transfer. The first time a given email+ip address attempt to send you mail, the server responds "try again later". If it tries again after 5 minutes, the mail is accepted. If it tries again before 5 minutes, it gets another "try again later".
The only problem is a small number of SMTP servers won't try again after a "try again later" message. The number of poor SMTP servers is fairly small, I'd estimate 1%-2% of all users.
AccountKiller
So what about public boards where a spider can pick up your email address? That being said, this is a really good idea; do you have it automated at all? For instance, do you have a catch all that is white listed so long as the incoming email address isn't black listed? Or do you manually add a white list entry for each new email?
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
If you need a real flexible smtp daemon, and can program in perl, I would recommend qpsmtd. Give it a try, you can create your own plugins (to handle spam or whatever you need) so easily you won't believe it.
SA is not a bad application but in my own experience it is a huge memory eater. I personally find DSPAM and/or CRM114 and/or OSBF-Lua better suited to catch spam/ham.
I use Postfix 2.5.0 and have on top of it running DSPAM, DSPAM RABL, SQLGrey, policyd-weight, postfwd, DKIM-Milter, SID-Milter, DCCM (DCC as a milter), SPF, AMaViS-New, a bunch of Postfix own anti-UCE techniques and other small things. All nice installed and glued together. It's working like a Swiss watch. And I am able to filter more mails per second then with SpamAssassin on the same system. I have less false positive/negative and I give full control to the end user (thanks to DSPAM) to influence his/her filtering.
For me SA is a nice program and I know it well but it takes way way to much resources from my system for simple filtering. When ever I do benchmarking I realize that in SA I need to count the SECONDS PER MESSAGE and in other tools like DSPAM/CRM114/OSBF-Lua I count the MESSAGES PER SECOND. That is a huge difference.
Speed is not everything but the memory usage is a problem. SA just eats and eats and eats memory. Why can't the developers make SA to stay below 10MB or 20MB memory usage? My DSPAM uses less then 4MB memory. From time to time when it needs to process a huge mail, then memory usage jumps up but gets back down again. On SA I can watch the memory being taken over by SA. Every hour this monstrum uses more and more. I hate that!
I have the following config in my sendmail.mc:
And I haven't had any persistent problems with legitimate emails coming through, which means that this setup works relatively well. I can't claim that this list is the ultimate or that it's perfect, but it works for me. The disadvantage is that it requires Sendmail, but for any *NIX hacker this shouldn't be a problem.There isn't even any problem doing a secure setup for persons roaming, in which case it's possible to set up a SMTP AUTH on a different port. I have at the same time elected to use SMTPS (SMTP over SSL), which means that any password and information sent over the net is encrypted.
Below is the code I use for listening on a secondary port (465/smtps) with AUTH and certificate handling for encryption.
You will have to hack the path "...ssl" into something real if you are going to use the above. And be familiar with OpenSSL.At least the actions I have taken discourages the spammers good enough and makes me feel reasonable safe. (there is always another leak, but you have to find it first).
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.