Slashdot Mirror
Interview With The SpamAssassin
comforteagle writes "Howard Wen has conducted an interview with Daniel Quinlan of SpamAssassin. In it he explores what keeps Daniel motivated in the face of the unrelenting torrent of spam and new spamming techniques, as well as, what is working - what is not, and what he predicts spammers have up their sleeves next for defeating spam detection." From the interview: "If you don't mind deleting spam manually, that's your prerogative, but don't complain about it. If your ISP doesn't do a good job fighting spam, then switch ISPs or install your own anti-spam software. There are a lot of choices out there."
When I got to over 300 spam a day was just about the time I tried gmail (google mail). So far this is the best spam protection I have come across. My spam folder is getting about 400 a day now but I can't remember the last time a "good" message went in there. I still get about five spam a day that I need to manually deal with.
http://www.busyweather.com/
Disclaimer: No interest in the company. Just a satisfied customer.
v1agr@ r0g@1n3
Who has noticed a decrease in the effectiveness of Spam Assasin. I have! Anyone else?
"If you don't mind deleting spam manually, that's your prerogative, but don't complain about it. If your ISP doesn't do a good job fighting spam, then switch ISPs or install your own anti-spam software. There are a lot of choices out there."
How the hell do you think the national do-not-call list came about? Because people bitched and complained! I agree there are spam solutions out there but I still think there should be an easier, more fool-proof, and legally backed way of opting out of spam.
I find laziness to be an excellent motivator.
Quinlan: That would probably be advance fee fraud, also known as "Nigerian" or "419" scams. These messages are often literally sent individually to each recipient, mutating each time, by scammers typically located somewhere in West Africa. Because they often are sent in low volume, and almost every one is somewhat different, they are a bit tricky to catch.
An easy solution for home users who don't happen to know anyone from West Africa is to just block all e-mail from there. But even without that, I have had decent success in the past with a combination of SpamAssassin tagging e-mails and Thunderbird filtering. Stay away from OE. Far, far away.
The SURBL can be found here: http://www.surbl.org. It's a very good thing, so much so that spammers are starting to try to get around it by doing stuff like this:John.
IT IS THE BOMB. Spam loads to my work account dropped by orders of magnitude. Now, Mail.app identifies maybe 2 per day, instead of 200+.
Charles
There exists no way of exchanging information without making judgments. --Bene Gesserit Axiom
I've said it before, but I have to promote PopFile (http://popfile.sourceforge.net/) again. Since doing a bit of training, it now correctly sorts about 99% of my e-mail. I get about 600 messages a day not including mailing lists, and my accuracy is 99.65%. It is generally not susceptible to new spam techniques unless they can match the subject matter that my e-mail typically covers.
When they start spamming "Linux IPF Apache LOOK! Vi@GR@ makes your peNi$ PHP Bug CSS" I will be concerned.
Maybe I'm the lucky minority here, or my mail host has some crazy filters I don't know about, but I very, very rarely recieve any type of spam. Now, I don't go handing out my email address either. If I'm signing up for something shady, I use another address at a web-based email account, which does get a lot of spam... but otherwise I use the mail host that comes with my website http://www.surpasshosting.com/ and Thunderbird as a client, and never see any type of spam.
From TFA:
The greater challenge is that the new techniques never stop coming. It's possible spammers will eventually run out of tricks, but it definitely hasn't happened yet. Most techniques backfire fairly in the long run, and make it more obvious that a message is spam.
You gotta wonder if there is a spam "bubble" that will burst pretty much like every other bubble. It started the same way, a few scammers got the idea of sending out scams via email and were quite successful, and everyone else started to jump on board. But soon enough(hopefully) people will learn their lesson and spam will slow....maybe I'm putting too much faith in people.
But it is interesting to see how many "me too" trends there are in spam. Up until about 2 years ago, I never received a 419 scam, but now I get at least one a week. Up until about a year ago, I never received a rolex email(typically the domain of brick and mortar(ok, urine soaked streetcorner) drifters), but now I get a few a day.
Monstar L
Two words: Spam Arrest. Zero spam, no filters to nurse, no lost mail.
Slashdot entertains. Windows pays the mortgage.
I bet he has cool business cards:
Daniel Quinlan - Spam Assassin
He can tell people his job is to kill spammers. Which reminds me, I wonder if anyone at the IRS actually checks what job title you put on your tax forms?
I Am My Own Worst Enemy
...God bless Daniel Quinlan and people like him. I have had a hell of a time with my daughter's email. A LOT of Web sites for kids have a "mail a friend" option. At one point my daughter wanted to use that option on a few sites. These are kid-oriented sites with privacy statements, so the sites felt trustworthy.
Fast forward to two weeks later, and one of those #@!&^ing sites has sold her email address to every spammer in the nation. My little kid got 196 spams yesterday -- for Viagra, lesbian cheerleader porn, you name it. So I have become heavily interested in every anti-spam product known to man. I've got 'em on the server, and got 'em on the client. Right now, with redundancy, this is 99% accurate, and my daughter gets only messages from friends and family. My biggest problem is not that spam gets through, but that false-positives block a legit message every now & then. That is the area I hope improves the most.
My Greasemonkey scripts for Digg &
"If you don't mind deleting spam manually, that's your prerogative, but don't complain about it. If your ISP doesn't do a good job fighting spam, then switch ISPs or install your own anti-spam software. There are a lot of choices out there."
It seems pretty simple to me: complaining leads to awareness, which leads to action. Maybe a bunch of people on Slashdot griping about spam won't amount to jack, but let Oprah or someone else with a grappling hook or two on the office/church/bar water cooler complain about it and they can make a difference in social attitudes.
SpamAssassin is a good step but the real problem is the social system which makes spamming possible. How else can you explain a 60-year-old grandmother 1) using her computer as a spam relay, 2) acknowledging it on television, and 3) not seeing it as a problem because it's "legal" and she's getting regular cheques to do so?
How is it that a social/legal system can be designed to bankrupt and scare the shit out of people who share a few movies or songs but barely put a dent in the people sending out millions of useless, offensive, and content-bordering-on-the-illegal emails? Is there nothing wrong with this?
A pop3 proxy works great. I recommened SpamBayes
http://spambayes.sourceforge.net/
What's wrong with personalized training? I get more spam than almost anyone I know, and SpamBayes does a fantastic job for me.
This has both good and bad aspects. First, the good news: responsible ISPs will be able to block a good portion of spam at their routers and mailservers; it's not hard to detect and blacklist a PC which is spewing the same email to 20,000 different recipients. Unfortunately, it only takes a few poorly-configured ISPs to provide a great deal of bandwidth to spammers. Couple this with Windows' known security holes, and home users' typical apathy regarding patches and security updates, and you have a large pool of potential spam-hosts which cannot be as easily targeted as open relays or specialized spam-spewing servers. After all, if spammers are using a legitimate ISP's mail server to send spam, a remote admin can't block that mail server without also condemning large amounts of legitimate email to deletion, which may well be unacceptable.
The upshot of all this? The onus of spam filtering is going to be, more and more, on ISPs rather than on recipients. While this has its good side - spam filtered at the source doesn't take up as much precious bandwidth - it also means that filtering will be more difficult for those not close to the source.
That's it. I'm no longer part of Team Sanity.
- Reject if on the spamhaus list
- Reject if claiming to be your mail server in the helo
- Reject if claiming to be RFC1918 space in the helo
- Reject if there isn't a '.' somewhere in the middle of the helo (simple way of checking for FQDN)
In addition, configure sendmail to do rcpt flood rejects, and even better, enable greet_pause. I've rejected quite a few with those.Anything that gets through all of that is then analyzed by spamassassin. WIth Bayesian training, my current threshold is 3.0. Anything legit is normally -2.0 or less. I Totally DROP through mimedefang anything greater than 7.0. Anything from 3-7 is dumped in a special folder on my local account via procmail. I analyze that stuff every now and then to see if it is time to once again lower the thresholds.
Also, continue to do the RBL checks in spamassassin (although it's a little redundant since I check spamhaus in mimedefang). That way you also get scoring based on SURBL..good stuff.
Spam Bayes with Outlook correctly handles over 95% of my spam.
In fact I've found it works great as a personal filter, if you configure it somewhat differently from the way the documentation suggests. That is, increase the weight of the Bayes filter, and have it train itself on every message it classifies. Then correct it on any mistakes it makes - which rapidly become few and far between.
Here's a paper showing that SpamAssassin can achieve as good results as others touted for personal use.
Unfortunately SpamAssassin is a bit hard to install and set up. But if you have RedHat or Debian Linux, it is available by rpm/apt and you can install a few scripts to make it work.
I wish I had a better shrink-wrapped version, but I don't. So I'm supplying the raw files for one user in the hopes that (a) somewhat technical people can reproduce the setup and be happy, (b) somebody will make a shrink-wrapped version, perhaps with plugins or extensions or macros for more mail clients.
Here is the Linux Personal Spamassassin setup.
With a full screen terminal window, I can mark spam based on the name and the subject header. I can recognize spam at a rate of about 10 per second this way. With the names spammer pick, and the mis-spelled subject headers, it is pretty easy to pick them out.
Using pine, I never give a spammer info by opening web bugs. I can look at the raw email by typing "h" to show the headers, so all those phishing emails are immediately obvious.
Keeping the email on the isp's server means that when I rebuild a machine, I don't have to worry about about backing up my email.
Since I implemented the above as a Postfix ruleset, I don't get spam anymore, and it's not exactly like I've actually kept my primary address secret. No, I'm not kidding or exaggerating - basically, my mailbox is my own once again. Viva Postfix! Viva greylisting!
Dewey, what part of this looks like authorities should be involved?
"Not spam" does not whitelist senders. It moves messages. Maybe I'm missing something.
I really should have just posted AC, having gotten three replies that went:
1) google radiates golden benefince, you suck for criticizing them
2) see that "not spam" button? the shiny one? don't lick it, click it! good boy!
3) Use another email client, you're not firewalled or anything, and you configure this client everywhere you go, right? Aren't I clever for knowing about its existence?
My blood pressure really cannot stand slashdot these days.
I am no longer wasting my time with slashdot
I manage a couple ISP incoming MTAs, they come looking for a anti-spam and anti-virus solution which is easy to provide them in OSS land.
...
First Qmail setup to use RBLs
cbl.abuseat.org sbl-xbl.spamhaus.org relays.ordb.org dynablock.njabl.org list.dsbl.org dul.dnsbl.sorbs.net
That bunch will block a whole lotta spam before it ever gets to discuss sending mail with the SMTP server.
Next, SimScan from Inter7.com, this little c app runs at the front end of the SMTP process, it will scan incoming mail at SMTP level with ClamAV and SpamAssassin, anything scoring over 10 in SA is dropped at SMTP level with a 5xx error.
SimScan allows you to fine tune settings on a per domain and per user level if you so desire, so it is easy to turn SA off entirely for a user who wants all the spam they can get, ditto for those who'd rather not be protected from viruses.
Using these features you stop a LOT of spam, likely in the 80% or higher range. Most domains we've applied this to have gone from hundreds per day to less than 10 per day.
It is imperative you also use the SURBL features in SA to stop more spam than ever, you should also use Razor2, DCC and Pyzor. I suggest upping the Razor2 scores a bit as well the defaults are quite low.
If you can't use your own address then your spam filters suck. I will not let spammers decide where and with whom I share my address. It is mine, and I'll do what it takes to defend it.
Dewey, what part of this looks like authorities should be involved?
Use SpamPal. It comes with blacklists, but you can turn it off because the reg expressions that came with it are very effective. There are also modules to decode base64, filter on spammed URLs, clean up web bug crap, block by country etc. & it's free.
From the article:
I can't believe no one has mentioned Yahoo! yet. Automatic, accurate spam-filtering? Yes. White-listing? Yes. Black-listing? Yes. And if you want to stick with the free account, use Yahoo!POPs to download messages into Thunderbird.
Personally, I have the upgraded (2GB) account so I can take advantage of what I consider the best anti-spam feature available anywhere: disposable email addresses.
Not sure if you want to divulge your address to for a free iPod contest? Give them a disposable address where email is directed straight past your inbox and into a separate folder. When you lose that iPod contest and the spam starts pouring in, just delete the disposable address.
Sure, you can set up a free "junk mail" address with Hotmail, Yahoo!, but I've found that "checking in" on my spam is a waste of time.
Of course, the best solution is to not give out your email address.