Slashdot Mirror
Aggressive Email Filtering Blocks Political Debate
Stephen writes "Many of us have spam blockers operating on our mail. But according to this BBC article, when British members of parliament starting having their emails filtered last month, it stopped them talking about genuine political business such as the Sexual Offences Bill, and prevented them receiving some constituents' emails." This problem has bit me on the bum a few times too. About 1 message in every 250 spam is a false hit. Course thats about once a day :(
that the best anti spam method is to block certain IPs. No filtering based on content.
Sometimes filtering CC entries works pretty good.
A similar problem happens with free Webmail or adversiting-supported e-mail accounts. The small advertisements attached to the bottom (I call them "spamlets") will sometimes trigger mail filters.
Watch out for this if you're sending a message from e.g. Yahoo! to Hotmail, who both attach spamlets and both filter incomming mail. They also will not send rejection notices to the sender, so you may never know if you message got through.
The only problem with this is... the servers who maintain the lists of trusted servers could start charging for a "email registration" type service. (Kinda like TLD's and the name registrar's out there)... Then eventually you'd start loosing the trust and we'd be back into the same steamy pile of pooh we're in now..
Just when you make it idiotproof, some idiot builds a better idiot.
According to the article the system was implemented without prior warning. What they should do is educated the users on how to implement spam filtering on their machines and not stop messages from going through at all.
In my e-mail client spam is marked in a different color, and by now the success rate seems pretty good, but I still don't trust it enough to auto-delete them. Spam sucks, but false positives not getting through might be worse than boobie mail getting blocked. In this case members of a governing body are affected. They should be working on legislation against spam, instead of having their hands held by the IT department.
Hank! White!
bit me on the bum
Taco, ol' Sod, I see you're hard at work addressing those complaints from our brothers overseas about the persistent American slant of SlashDot.
Good On Ya, Mate!
That said, and out of fear of being mod'd OT, let me add that I have had success training Evolution's filter system to recognize spam not based on the subject but on the domain name. Without ever bothering with public blacklists, I've just patiently built out my own Enemies List over the years. The "keywords," if you will, in so many of the spammers' domains are remarkably similar -- "email" "deals" "free" etc. Combine that with whitelisting based upon my address list, and I think I've had maybe 2 false plucks for as long as I can remember (receiving on the order of 150 spams daily)
When the FTC decided to try filtering
No, it doesn't. It only works well for those who can afford to pay for professional lobbyists & for those who pay to set up 'special interest groups' claiming to represent the vast public when they don't.
When I lived in the US, I can remember the 'Prop. nnn' ads on TV, paid for by 'Concerned Citizens Against Blah' written in small print at the bottom. It always turned out that 'citizens' were actually some big business (often the tobacco industry).
Alison
"It is a miracle that curiosity survives formal education." - Albert Einstein
Agreed, though SpamAssassin has one feature that is real usefull for this: the score is embedded in the email header in a row of '*'. This can be used to reduce the amount of skimming you have to do: I find I never get a false positive that rates above 13, so I /dev/null (or uce@ftc.gov) everything above that, and skim the rest. This gets rid of the worst spam without my intervention, and reduces the amount I need to skim to 10-20 messages a day.
'Sensible' is a curse word.
Not at all, and you're thinking about far too simple a model.
/dev/null. Mail would have to be carefully crafted to achieve a 20 unless it truly is spam. Such effort is not to be rewarded :-) Keep in mind that mail about laws on sexuality or other simple examples cited in this article would never get NEAR a 20.
/dev/null. This is a very nice way to identify that a) there's a consensus that this very message is spam and b) my local mail patterns indicate that this is spam.
/dev/null to a mailbox that I scanned carefully. None of the messages that I would have thrown out were non-spam.
With SpamAssassin, I deal with spam in 3 ways:
1. Mail that gets a score of 20 or more is sent to
2. Mail that triggers both the Bayesian and Razor2 tests is sent to
3. Anything else with a score of 4 or more is marked in the subject line and I have a virtual mailbox in my mail client that I use to glance at the from addresses. If something looks plausible, I check it out.
As of the development version of SpamAssassin that I'm using (about a week old out of CVS), I get a false positive rate of about 1:100-200 messages and during testing over the last couple of months, I copied the messages that would have gone to
I get a LOT of mail form lists, spammers, friends, random people on the net, machines spewing status, etc. I feel that I'm a reasonably good QAer for this sort of thing, and the new SpamAssassin will rock your world (and the spammers')!
Filtering based in very common words has proved dangerous. Some of the problems of this (for http proxies) has been covered here
A few weeks ago the Scunthorpe town council decided to implement a nasty words filter on all email received, just to reduce the volume of abusive email they were receiving.
The email filter worked out very well indeed - well, too well. Absolutely no mail was delivered. It took a while for them to realise that their own town name contained one particular rude word, and considering that their town name was part of their email address, all email had to have a certain word in it.
I started using a filter called Spammunition a while ago. It's a free Bayesian filter for MS Outlook. (Not my ideal mail client but it's what we use at work). It's great. No false positives, and catches all my spam.
It is important to distinguish between rejecting a message (in which case the sender gets a "550 spam" indication) and discarding a message (in which case neither the sender nor the receipient is notified). Only the SMTP server can reject a message, it is too late by the time it has gotten to the message user agent (client).
If the anti-spam software rejects a message it is usually trivial for the sender to modify the message or find another delivery method and little is lost. If a message is discarded, the damage might be much greater.
Bayesian spam filters usually run on the client, and have to discard messages but there is no particular reason they couldn't run on the server.
The client can't reasonably return a "DSN" via email since the envelope from (even if known to the client) is probably a forgery, so responding would just be creating more spam. The SMTP server
can reject the message before it is accepted with an error code, it doesn't have to send an email with the error message.