Slashdot Mirror
How To Fight Spam Using Your Postfix Configuration
hausmasta writes, "In this guide you will learn how to tweak your virtual Postfix setup to better combat spam by stopping the mail before it hits SpamAssasin, using RBL (Realtime Blacklists) and RHBL (slightly different), greylistings, and Helo Checks." A clear, step-by-step guide to a complex subject.
The HowtoForge guide is great for everything however if you have a very busy mail server running Spam Assasin on 1000s of message per minute is a CPU killer. The best answer is to stop the mail before it hits Spam Assasin with a range of RBL (Realtime Blacklists) and RHBL (Same but different), Greylistings and Helo Checks.
... aren't blacklists still a bit... tricky?
Ignore this signature. By order.
If you're running the mail servers for a business, how prudent is it to run a spam filter in the first place? While using something that relies on checking the content of the mail may be useful in getting rid of the most egregious spam, you don't want to block all items identified as spam. You can't run the risk of blocking your customers.
RBLs are notorious (especially SpamAssassin) for blacklisting entire domains when only a small subset of those users are actually sending spam.
If you're running your own mail server at home, then a whitelist would probably be more useful than a blacklist since you already know who you want to contact you.
But you gotta hand it to the Unix folks for making the task of setting up a spam filter this difficult. I am curious how difficult it would be to set up a spam filter on an Exchange server.
a single page article that tells you how to filter internal domain mail from external ?
there is nothing on this page about how to use RBL (which is a lame way of filtering as it is) or whitelists or anything else, did the submitter actually what this article covers ?
useless, now wheres my point and click interface ?
The layout of the linked site is awful and sticking configuration snippets in text areas is awful. I didn't even see any warnings and if you're doing this, there are plenty of things that could go wrong. IMHO, setting up an MTA to reject spam at SMTP time is an involved process that requires the admin know what they are doing, blindly following a poorly written howto is a recipe for disaster.
You clearly have no understanding of how SpamAssassin works. Blacklists are only one of many negative as well as positive factors that SpamAssassin takes into account when evaluating whether a message is spam or not. You can adjust the score threshold to a level you're personally comfortable with.
I have 3 tiers, ham (non-spam), possible spam (that gets moved into my junk folder), and true spam (really high scores) that gets rejected at the mail server. Also, for anyone else running anti-spam software at the server, DO NOT BOUNCE EMAIL BACK TO SENDERS if you're not rejecting at connect time. If someone used my email address to send spam (not to be confused with my server), I don't want to receive the bounces for it.
I started using postgrey on my mail server a few months ago and my spam dropped from several hundred per day to about 10 per week. I wrote up this great how-to but figure that if spammer's got wise my advantage would be lost so I never published it.
If you do this, and you're an ISP (which you likely are if you're getting 1000s of spams a minute) you're very likely to block legit mail from ever getting to subscribers. Realtime Blacklists aren't reliable enough to use alone in blocking spam. You'd serve customers better by buying more/faster machines to handle the load.
If you DO decide to do this, be VERY carefull about the RBLs you use. Some RBLs list entire blocks of IP addresses of ISPs that have hosted spammers, and some even list every DSL/Cable modem static IP address. I've had me mail blocked by overzealous ISPs before (for just being on a cable modem static IP), so I'm not a big fan of using RBLs or other simple techniques to block spam at the postfix/sendmail level.
AccountKiller
Don't filter by RBL. Use them to give scores to Spamassassin, but don't reject mail on basis that some host is in some RBL.
* There are RBLs nearly impossible to get out from, and you might actually get an IP assigned months earlier to somebody who never requested a removal.
* False positives. Mails misidentified as spam (typically: newsletters which the signed up person no longer wants, vacation-mails in foreign languages) might bring you onto an RBL.
* Collateral damage. A shared server with 1000 users and 2000 domains might turn up in an RBL because one of those users had an inscecure formmail running a night long. And even after removal by the sysadmins in the morning, 1499 users can't mail you the next 18 hours.
* Spurious criterions for getting listed. Like "unsolicited bounces" or "sent mail to secret spamtrap"
So while RBLs are really a useful tools for deciding whether a mail might be spam, using them as THE ONLY reference on whether something is spam or not is just foolish.
"The more prohibitions there are, The poorer the people will be" -- Lao Tse
Here is a link setups to build servers with postfix, amavisd, SpamAssassin, ClamAV, Razor (you could also use these settings just for postfix to add better spam resistance).
I not only am I the President, I'm a user.
http://www.freespamfilter.org/
Enjoy...I love it...
I am aware there's definitely a fair number of over-zealous blacklists, but I've had an extremely good experience using cbl.abuseat.org as a blacklist source, and haven't encountered any false-positives while perusing my mail logs.
SIG: HUP
Rejecting mail outright based on RBL results is an invitation for loads of false positives. If you're getting "1000s of messages per minute," chances are you are a large business or an ISP. Either way, your customers sending mail from foobaz.net (for example) won't be happy when their no longer gets through because some boneheads at spamcop or spamhaus put them on the RBL.
If you don't have the proper setup to accept that many messages and do adequate "tag-and-forward" spam analysis, or at least reject later in the delivery cycle based on cumulative results from multiple RBLs, you need to beef-up your mail architecture. Of course, you could just set it to randomly drop 80% of all of your incoming email into /dev/null and you would likewise solve your traffic problem.
Who is General Failure? And why is he reading my disk????
There are more explaination on the postfix.org than this FA (fine article). Wisely using good RBL (for open proxies) and graylisting and some helo/header checks should reduce a lot of spam.
I use a combination of greylisting (10 mins for normal IP addresses and an hour for dynamically assigned IP addresses). I also do a very quick check on a few sender domains such as yahoo.com A "from" yahoo.com must be from a machine with .yahoo.com. If
not, it is rejected. Works quite well with less than one spam
per day. Given that yesterday had 466 spam's that got rejected,
I'm pretty happy. I cannot recall anyone complaining they can't get in.
Greylisting works really well because real mail servers try hard
to deliver whereas spammers don't try that hard for delivery.
You should be able to send email between any 2 routable IP's. It's unfortunate that we have a minority of criminals and amoral profiteers ruining it for everyone. I myself reject residental hosts on networks operated by irresponsible ISP's (comcast, netvision, road runner, verizon). That doesn't mean I agree with forcing people to use a smarthost to send legitimate email.
A far more effective and less faulty way to filter out some spam can be done by using the new features added in sendmail 8.13.
.mc file.
FEATURE(`great_pause',5000)
That one is given in your
Wait's 5000ms to say HELO (EHLO) and all MTA's starting to send data (spambots not being all that RFC-aware) before that is discarded.
I've measured that it atleast cuts 15-20% of the total amount of spam.
All generalizations are false
Warning, I used to use RBLs to reject mail, but occasionally all of yahoo or somethnig gets added to a blacklist, and so mail from yahoo would start bouncing (including yahoo groups -- they really ought to use a seperate network for yahoo groups as their free webmail!). So instead of rejecting mail, I started just inserting a warning header, X-Spam-Blacklists:, which is easy to do with Exim. (I assume also with Postfix). Then I could filter the message in Thunderbird into a "probably spam" folder. You could also set up SA to increase a messages score if it's in a blacklist, I think, but I've never tried it.
So if you do set up RBL rejection, make sure you pay attention to what it's rejecting. Skim through the log file a few times in the week or two after doing it, otherwise you'll never no that it's being rejected.
Spammers are having their zombies dig through the windows configurations to find the owners email relay and using that to send their spam. It's not that difficult and combats greylisting.
Here is an insightful blog posting from Justin Mason about using RBL's (and bl.spamcop.net in particular which this HOW-TO mentions) for filtering spam. You could see a staggering amount of false positives unless any rbl is only one part of a scoring system which decides whether a mail should be rejected.
Never underestimate the dark side of the Source
For those of us who like details, here's an alternative spam filtering guide: http://www.flakshack.com/anti-spam/wiki/index.php.
I'm using the setup presented to filter about 1000 messages per day before they hit our exchange server. It works very well and so far has had no false positives. However, it's difficult and time consuming to set up and needs to be trained with sa-learn before using.
http://assp.sourceforge.net/
We use ASSP its an SMTP mail proxy that sits infront of your mail server.
The advantage of assp is it can reject emails without having them processed and it has a greylist and penelty box. The offical site explains it better than i can. All i know is since we started using it (and after training the bayes database) ive had no spam for several months....
collateral damage.
Greylisting will always be useful, at least to a certain degree. When you get email from an unrecognized source, one that you can't judge as being a source of legitimate email or if it is a spam source, greylisting lets the reputation systems like DNSBLs and DCC/Razor/etc. catch up. Also, good ISPs and mail admins have tools to watch for likely spam runs. Again, greylisting gives those tools time to act and purge the spam.
Sure, greylisting also gets rid of spam from spambots that don't retry email, and that's a plus but, as you mention, making sure the email gets retried isn't that hard.
Greylisting is not a Final and Ultimate Solution to the Spam Problem, but it will always be a useful tool.
SPF support for most open source mail servers can be found at libspf2.
I for one, welcome our spamfighting overlords, and think that we should see more of this kind of thing on the front page of ./
Although we've got our share of trolls and fanboys ruining it for the rest of us, I daresay that there are at least a few slashdotters out there who could write guides at least as good as this on a number of geeky and related subjects and present them on the front page, in much the same vein as the original content presented in book reviews etc. It might even be worth paying the author a few bucks for the privelige.
I know that the news presented here is usually linked-to content, but I think some original guides with relevant links would be content at least as worthy as the 'Nanocosmetics used since Aincent Egypt' article posted by that little troll Roland last night.
When the posters fear their moderators, there is tyranny; when the moderators fears the posters, there is liberty.
At my last company, we had a big problem with spam. But because the company did some online marketing, there were people in our marketing department that WANTED to get some of the spam so they could see what tactics the competition was using. At one point our mail gateways were so clogged, mail was being delayed by up to 4 or 5 hours at our gateway.
We fixed the problem by having postfix discard any mail that was not addressed to a legitimate email account. We ran a script that would pull all valid email addresses from exchange and told postfix to reject anything else. We set that script to run every hour to catch any changes to email addresses. Our emails that went through spamassassin dropped by almost 90%.
Our only trouble was that if we rejected with NDR, there were tens of thousands of Non-Delivery Reports being sent out, and if we rejected without an NDR, people who mispelled email names (sean@domain instead of shawn@domain) never got a report stating that their email did not go through.
Overall, it was a huge success and we didn't have to mess with greylistings or blacklists.
Great civilizations have lived and died on false theories. Don't mess up mine with a few facts.
I wrote basically the same article a year ago for Free Software Magazine. We've been using this setup at my office for the last year, and it's been working perfectly - to the point that an upset coworker told me yesterday that she's received four spams last week and wondered if something was wrong.
Dewey, what part of this looks like authorities should be involved?
Ok, I recognize that you are probably trying to promote Sendmail. But for a variety of reasons, there are folks out there that can't or don't want to use Sendmail. I don't use Sendmail because I consider it overly complex. I use postfix and yes, I use the RBLs before it gets to SpamAssassin (gasp). If you tune it properly, it works just fine, thank you. Also the email list archives are extremely helpful in this matter, as is the list itself if you've done your homework first.
The simple truth is that interstellar distances will not fit into the human imagination
- Douglas Adams
The better place to looks is the Howtos and FAQs.
x t
One of my favorites: http://jimsun.linxnet.com/misc/postfix-anti-UCE.t
Has anyone found a good open source gateway solution that incorporates a web front end for user configs and quarantine? Basically an open source Barracuda appliance?
Idiot.
Yes, greylisting has become less effective. Now its only blocking ~90% of the spam instead of ~95%. Oh no! Considering it is essentially a freebie, with no real downside, and no false positives ever, that's damn good. Of course you still need to filter spam after that, but cutting out 90% of it for free is a great help.
Comment: (paraphrase)" X doesn't work because Y happens"
Rating: +5 "insightful"
Reply to above comment: "X works because Y happens"
Rating: +5 "insightful"
And neither post showed any actual insight; just observation.
My Suburban burns less gasoline than your Prius.
If you implement SPF, SPF-aware mailers won't send you those reject messages. And for the most part, spammers will stop using your address in their From: fields, because it decreases their spam's penetration.
Don't confuse SPF with Microsoft's fake "sender-ID" crap, though. That's essentially useless, you want SPFv1 aka "SPF classic", then DKIM as soon as it's ready.
Worked for me...
So, you can drop a connection at HELO if the HELO parameters are malfed, but you can't drop it just because it's unverifiable - you have to wait and drop it at RCPT if you want to be RFC-compliant. It's generally a good idea to be RFC-compliant whenever possible.
RBLs are great for weighted systems like SpamAssassin, and for blocking Open Relays. If you use an Open Relay, everyone should reject your mail, because you are either clueless or a cantankerous ass-hat.
I wrote a policy server for Postfix that will allow you to trigger responses to a message based on rbl status, which means you can use presence on an RBL to trigger greylisting of an email, which gives you the best of both worlds.
maRBL
Can anyone recommend a good open source antivirus and antispam gateway that allows the user to manage their prefs and quarentine via a web page. Basically like an open source Barracuda appliance.
.... it's not like this is awesome breaking news. I know of at least one institution that's been doing this since the 20th century (as a matter of fact, they likely invented greylisting before anyone had thought to call it that).
Martian Software has an interesting (if somewhat dated) piece on using statistics to cause spammers pain. Essentially it's a real-time spam filtering system that slows down messages according to how "spammy" they seem to be. For individual messages, a few seconds or more delay would cause little if any problems, but when sending millions of spams, this is a very big issue indeed.
Personally I'd like to make the spammers pay - in CPU cycles - using methods like this. Of course I'd rather beat them witha baseball bat, but that's a wee bit illegal/impractical.
As long as the receiving mail server rejects the message with a permanent failure notice, a correctly configured sending mail server will give the user an appropriate response, and if it rejects it with a temporary failure notice, a correctly configured sending server will keep trying, and eventually let the user know if delivery hasn't happened in some time period.
If the receiving mail server accepts the message, it's obligated to send a reject email if it fails. That doesn't mean you *actually* do that with spam, but it means you either fail to notify a legitimate sender that the mail has been dropped, or else successfully notify a forged sender that the spam with his name forged on it has been dropped, or do something else inappropriate.
This doesn't mean the reject code needs to be *honest* - your smtp receiver can lie to the sender about the recipient's mailbox not existing, or being full, or about the sender's mail server smelling of elderberries. If the email really was sent by a human, and the sender's mail server correctly notifies the human of problems sending it, it's up to the human to decide what to do.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks