Slashdot Mirror
What E-Mail Validation Tools Do You Use?
morcego asks: "As we are all too much aware, spam is an increasing problem. Each of us has our own set of tools and methods to try and reduce the amount of spam we receive, each with different pros and cons. Also, on a more broad front, we have options like SPF (+ SRS), Microsoft's own Caller-ID, and
Yahoo's DomainKeys that we can use. These days, it is incredibly easy to implement any (or all of these), using publicly available frameworks and libraries (libspf2, and milter, to name a few). I have been using SPF for quite some time now with some measurable results, although nothing earth shattering. Which of these are you using, if any? Why, or why not? Do you think any of them really contribute anything to fight spam?"
I wrote my own Bayesian filter, Mailvisa, to gain a better understanding of how Bayesian filtering works, and to be able to tweak the parameters. When I last measured it, it caught 93% of spam. Of all the filters I tried at the time (I think it was all filters in Debian sarge), only Bogofilter scored better. This applies to both the amount of spam caught and the filtering speed. The closest thing to false positives I've gotten over the years were a few advertisement mails from my domain registrar.
I have only two problems with it: 1. I have to train it regularly, and 2. nowadays, lots of mail slips through, because it contains words related to programming languages.
Please correct me if I got my facts wrong.
After trying to tune SpamAssassin to work well for months, and being unimpressed by the hit/miss rate, I tok to forwarding all of my incoming email to GMail. I then forward all my email from GMail that is not spam back to my other account :0
I find this way I get 99.95% accuracy - things that GMail misses as spam, my local SpamAssassin catches. As a side bonus I have GMail's awesome interface to read my mail when on the road (much better than the Squirrel Mail I was using, and still better than RoundCube).
This brings up another point - I don't know why Google doesn't add IMAP connectivity to GMail, soyou could use it's interface to read email from other hosts. I don't see why their ad technology would not work with this scheme.
While not necessary e-mail validation tools, greylisting and SBL+XBL blocking lists by Spamhaus have eradicated nearly all spam I used to get through all of the other filters.
Greylisting alone helped to lower e-mail traffic drastically and blocking lists take care of known spamming hosts. I'd recommend using both to anyone running a e-mail server.
all that SPF CallerID and DKIM does is validate the sender !
this cuts out about 70% of (stupid) spammers
you also need to blacklist people who send you spam (and you can be confident that you get them because of the above technologies)
if you Ever want to send lots of mail to hotmail users you need to have callerID setup yahoo and gmail both trust you more if you have domainKeys
so things are moving on and there is no reasson why people should not have at least one of SPF CallerID or DKIM setup on their domain !
you will note that people here also use filtering but the question is does the filtering feedback to the blacklists ?
regards
John Jones
p.s. I work in the mail vendor world...
I use the OS fingerprinting options from pf to block windows machines from delivering mail on the primary mx. This saves approximately between 300 and 1600 spams a day. Beside that, rejecting mail from hosts without an A record, blacklisting all hosts sending mail to spamtraps with spamikaze, rejecting hosts which falsely claim to be a host in my domain and filtering with bogofilter.
I use spamd on OpenBSD to do greylisting. That cuts an enormous amount of spam out.
For those who aren't familiar with greylisting, when an smtp server attempts to deliver an e-mail the from address, to address, and IP address of the sender are put in a database and the mail is refused with a non-permanent error code.
Assuming the smtp server sending the e-mail follows the RFC, it will try again later. When it tries again after at least 20 minutes from the original attempt, it accepts the e-mail and adds the IP address of the source to a whitelist. For the next 30 days, any e-mails from it are white-listed. After that, the server is verified again.
I also keep a seperate white-list for non-RFC compliant servers and for frequent senders. Some servers only try one to three times and quit. Another problem is e-mail from some large e-mail farms may make each attempt to deliver the e-mail from a different server with different IP addresses, so I'll add their e-mail addresses to the white-lists as well.
One method I use for adding IP addresses of selected senders that send a lot of legitimate e-mail to the whitelist is to look up their SPF records and use that to identify the usual e-mail servers for the domain.
A few ISPs appear to put their entire address space in the SPF record. For example, panix.com's SPF record is
Needless to say they don't get whitelisted since I only want to whitelist e-mail servers, not their users spam-zombie computers.
In other words, I use the SPF records to identify legitimate e-mail servers from selected domains only.