Slashdot Mirror
Aggressive Email Filtering Blocks Political Debate
Stephen writes "Many of us have spam blockers operating on our mail. But according to this BBC article, when British members of parliament starting having their emails filtered last month, it stopped them talking about genuine political business such as the Sexual Offences Bill, and prevented them receiving some constituents' emails." This problem has bit me on the bum a few times too. About 1 message in every 250 spam is a false hit. Course thats about once a day :(
These types of incidents may be good in the long run - if it makes law makers "wake up" to the problem of spam.
...We can only hope... Perhaps we could even start bombarding law makers with spam ourselves? - that would raise their awareness!
I can just imagine the outrage if this happened to the bush administation.
'what do you mean no one got my emails?'
'It seems your.. uh... last name is causing some issues with spam filters sir'
'That's it.. lets bomb the spammers'
I am the lord of the pun. Dance Knave!
...to eliminate all the dupe stories!
I think polititians shouldn't have any filters on their e-mail.
After about 2 weeks of what the average person goes through, we'd see stronger anti-spam legislation/penalties.
that the best anti spam method is to block certain IPs. No filtering based on content.
Sometimes filtering CC entries works pretty good.
it stopped them talking about genuine political business
thats because they no longer knew how to enlarge their penises and missed being notified that some russian woman wanted them so badly that it hurt.
that would certainly stop our gov't, at least..
xao
xao
http://TheHillforum.hopto.org
the problem is that just by knowing there could be a false positive, you have to examine all your filtered spam, which makes the spam filter useless in the first place.
I can easily see why this may be happening. The types of filters that use keywords can easily fall into this.
I understand that keywords and phrases such as
'free money' 'zero percent financing' 'win
million dollars' 'sex xxxxx pictures!' and so
on can trigger many filters.
I would like to think that the better designed
filters would use a combination of key words as
well as suspicious domain names and/or IP
address blocks to do filtering.
The spam filter that is used on my email account does not filter out, but it does add the word
'SPAM?' into the subject line of the email message. I can then see right away if it is
really spam or is something mistaken by the filter for spam. The message is not blocked, though.
Mark
Cleara
Rome taught me patience and assiduous application to detail. Virtues which temper the boldness of great, general views.
how to balance open access to constituents without being overwhelmed.
Perhaps Parliament could consider some of the steps that the American Congress has taken. The American Congress has a de facto filter built in to prevent Joe Random crazy from flooding their representatives with spurious requests. Most Congressional requests, letters, phone calls, faxes and emails are tossed out unless they come from certain designated people known as lobbyists. These lobbyists have worked hard to cultivate contacts in the Congress, and can get better results from one office visit than 1,000 letters from voters. In a way, they're professional access voters.
So, maybe the UK could restrict access to just professional lobbyists, it works very well in the US.
A similar problem happens with free Webmail or adversiting-supported e-mail accounts. The small advertisements attached to the bottom (I call them "spamlets") will sometimes trigger mail filters.
Watch out for this if you're sending a message from e.g. Yahoo! to Hotmail, who both attach spamlets and both filter incomming mail. They also will not send rejection notices to the sender, so you may never know if you message got through.
Perhaps they should start using pgp encrypted/signed stuff and filter out all non-encrypted/signed emails?
*shrug* That's what I do.. I hate getting email from somebody I don't know...
Just when you make it idiotproof, some idiot builds a better idiot.
Imagine that this system was widely used and every week/month everyone had to figure out which codes they had place in their subject line to communicate with people. Also, how many minutes do you think it would take for the spammers to write a script that scanned the bounced message and append the code to the next round of e-mail?
ich muß mehr Kuhglocke haben
You're confusing the right to free speech with the privilege of being heard.
Like woodworking? Build your own picture frames.
Because most of the people they should be talking to , i.e. constituents, also use this insecure system. In reality, most of the politicians I know use email aren't discussing state nuclear secrets or troop movements. If certain politicians are likely to leak sensitive information this way I would be far more concerned about what gets picked up from the far more insecure system - VoA (Voice over Air)
Specifically for the parliment, I dont see why they dont just whitelist all other parliment members.
People who think they know everything really piss off those of us that actually do.
...but not perfect.
Ok, here's the situation as I see it.
We have a problem: Spam
We need a solution.
So far filtering has been working good and is slowly getting better, but there's always gonna be the chance for false positives.
so how do we stop this?
I have no clue.
We should probably start cracking down on open relays, even use governmental pressure if needed (on spammers in our countries and on the governments of other countries). They serve no real purpose other than facilitaing spam.
What else can we do? Go after spammers legally. We need to make them pay. I bet if 1000 people sued ralsky for $500 a piece he'd start to take notice, but he still wouldn't learn. Some states, like washington, are doing that, and it seems to be working, or at least getting the spam recievers a little extra cash. If I lived there, I know I'd try it at least once. Hell, I might even pay for my braodband connection with the money I got from spammers:)
I've heard people recommend opt-out lists like they use for telemarketers- that's not gonna work because spammers are much more slimy- they'll use the opt-out list as a verified list.
We're not left with many choices, besides educating people to simply delete spam and DON'T buy from it. make it cost spammers money. if they sell even one thing, they they're winning.
I took a slightly fun approach. I'm building a list of 'legit' companies that sell your email address to spammers. What I did was bought a domain, and whenever I signed up for something, I used the companies name@ the domain, and had it all forward to one account. so when I get spam to musiccity@mydomain.com, I know that musiccity sold my email address (which they did).
Does anyone else have any Ideas how to stop spam? if so, save the redundant mods and reply.
Looking for Book Reviews? Check out Literary Escapism.
Without spam, how else would I be able to sit home every day and make $1,000 a week watching TV while playing with my 12 inch penis?
from Scunthorpe.
We offer SpamAssassin at the college where I work. I always tell new users that any spam blocking system, no matter how good, will eventually block something that was legitimate. That's why I don't write procmail recipies that redirect mail flagged as spam to /dev/null. You gotta put it in a seperate folder and you are asking to get burned if you don't skim the subjects and senders every couple days. Also, they should be whitelisting messages from addresses in their domain.
I don't see how this is news. It's just an example of bad system administration.
I'm sure the filters caused many problems with the "Hot, horny housewife" bill and the new "Extra six inches" tax debate.
Unfortunately, I am not Wil Wheaton
By definition filters are hit-and-miss and non-deterministic. I get almost exclusively SPAM with spoofed return addresses. How about this solution:
1. Sending mail server generates a tx content key based on the contents of an e-mail being sent.
2. Sending mail server uses the tx content key with a private key to create a confirmation key.
3. Sending mail server sends the e-mail, along with the confirmation key to the receiving server.
4. Receiving mail server generates a rx content key from the e-mail contents.
5. Receiving mail server sends the rx content key and the confirmation key back to the sending mail server.
6. Sending mail server uses its private key plus the rx content key to re-generate the confirmation key.
7. Sending mail server compares the confirmation keys.
8. If the keys match, the receiving mail server allows the mail to enter the recipient's mailbox.
9. If the keys don't match, the mail is bounced.
The keys are in place to keep the SPAMmer from tagging along on a valid return address with mail that address didn't send. This technique also keeps the second transaction to a minimum exchange of keys. The keys add traffic, but the eliminated SPAM traffic more than makes up for the penalty. As more and more mail servers are updated with this feature, spoofing is all but eliminated. The remaining "spoofable" domains can be explicitly severed from the net or blocked.
Xesdeeni
create a text file [&] zip it
Unless the recipient is expecting this they should just delete the message. I routinely delete any email that has zipped attachments unless I have previously agreed with the sender to send it that way. (That's assuming the recipients mailserver doesn't routinely strip zip files off as an enterprise virus protection measure in the first place.)
But one way your suggestion could be modified that will work for anyone whose email can view HTML is to print your message to graphic file, convert it to a GIF and embed it into a simple a webpage.
The reader will open the file and see what looks like a text message, but it actually will be the GIF image of your message.
Most filters don't block HTML and GIF files.
Work for Change & GET PAID!
i got in a fight with an ex-girlfriend and we ceased speaking for awhile
;-(
i became further incensed because she never contacted me after the fight
we didn't talk for 2 months
finally, i contacted her and said "why didn't you get back to me??!!"
she said, "you didn't get my email?"
i looked, and there it was, 2 months back, in my spam folder (yes, i keep all of my spam, the folder is gigantic)
although you could make a joke about emails from girlfriends being called spam, in this particular case, considering the chance at reconciliation that was lost and the feelings involved, it was definitely not funny at all
so i can say, with certainty, that my personal life has been greatly and adversely affected by spam.
you can hate spam for all sorts of reasons, but for me, it's personal.
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
If this were to hapen in the US, they would say "spam isn't really an issue these days" and no spam laws...
Boy would that be bad. Slow progress, is better than "no problem at all".
Lets push for a "no spam filter for Congress until Congress passes a no-spam law"
Then again, wouldn't be needed if enforced.
According to the article the system was implemented without prior warning. What they should do is educated the users on how to implement spam filtering on their machines and not stop messages from going through at all.
In my e-mail client spam is marked in a different color, and by now the success rate seems pretty good, but I still don't trust it enough to auto-delete them. Spam sucks, but false positives not getting through might be worse than boobie mail getting blocked. In this case members of a governing body are affected. They should be working on legislation against spam, instead of having their hands held by the IT department.
Hank! White!
bit me on the bum
Taco, ol' Sod, I see you're hard at work addressing those complaints from our brothers overseas about the persistent American slant of SlashDot.
Good On Ya, Mate!
That said, and out of fear of being mod'd OT, let me add that I have had success training Evolution's filter system to recognize spam not based on the subject but on the domain name. Without ever bothering with public blacklists, I've just patiently built out my own Enemies List over the years. The "keywords," if you will, in so many of the spammers' domains are remarkably similar -- "email" "deals" "free" etc. Combine that with whitelisting based upon my address list, and I think I've had maybe 2 false plucks for as long as I can remember (receiving on the order of 150 spams daily)
When the FTC decided to try filtering
So, maybe the UK could restrict access to just professional lobbyists, it works very well in the US.
Works well for who? I don't see how it helps the average joe citizen who wants to get his point across unless he donates money somewhere. Corporations have tons of cash to throw at it. So if Jimmy Lobbyist has more access than Joe Sixpack, thats a problem. repetition and filtering be damned. It is the duty of a representative democracy to represent those they are representative of, and if they aren't willing to take into account every email and letter and fax and phone call they get in their decisions, then it's a stone's throw away from not having elections at all, especially when you consider that when voting the only two candidates who generally have a chance is a lesser of two evils situation.
In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
Comment removed based on user account deletion
I attended the conference on spam at MIT. The conference would have been more accurately labelled a 'solving spam with the hammer we know about' conference since no other solutions were accepted - although several people besides myself submitted authentication based papers.
The big problem with the Bayes approach is false positives. Lots of great statistics were quoted but the claims were simply not credible. I don't believe that Spam is such a simple problem that the performance of naive Bayesian techniques is several orders of magnitude better on that problem than any other.
So really the trick is to swing the problem arround. START from the problem of making sure that anyone with a legitimate reason to contact me can do so without interference from statistical filtering techniques. The proper place to apply those is on the mail I cannot authenticate in that way.
I dislike the bounce-back loop as a filter for personal correspondence. I think it is great for the purpose of a lightweight authentication mechanism for mailing list subscriptions. I get very irritated when people use it to filter email, particularly since all my email is signed. People should not substitute their ad hoc authentication mechanisms without first supporting deployed standards.
The other problem with call back loops is that if they are used widely they will become a bigger problem than the spam, this is why I have been urging Microsoft et. al. NOT to support them. The trick that the spammers have developed to get round the callback loop is to steal addresses off mailing list archives and send forged messages to the other members of the list. So work out the effect that deployment of the naive bounceback hack would have.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Oh? Many people set their filters to tag'n'bag (or simply dump) any non plain-ascii email. I treat any email with HTML, base64, or an attachment of any kind as probably spam and potentially dangerous, and inspect it before reading it.
One line blog. I hear that they're called Twitters now.
I hate to do this because it's only partially complete. But I have a concept worked out on how to handle spam that works extremely well and removes the chance of false positives, especially from Real People.
It's not a money-making scheme, but it is prior-art <grin>.
The idea is a hybridization of SpamAssassin and tmda (tagged message delivery agent) wherein you accept all email into your inbox and the spam goes into a spam mailbox. Nothing New...
The cool part comes in when you start automating the spam_mail similar, at least conceptually, to what I have on my website. Shameless plug here
The idea is that you send out an email confirmation, similar to tmda, for only that email which is considered spam (by SpamAssassin). This means that most of your regular communications would go unhindered. But it would also make casual contact via email the easy and simple function that it is supposed to be.
These notions of having an email list of only your known contacts is a pain in the arse and most times met with extreme hostility. This is especially true if you are attempting to contact someone privately from an email list, or from a solitication from their website.
I have to warn you that if you use the code as described on my website you will probably break your server in the first day. I've rewritten it to scale much better (1,000 spams every 10 minutes). But I haven't had the chance to post the new code. But conceptually it rocks!
I've processed something like 20,000 emails without taking a single false positive, unless the original sender vegged... but then he didn't really want to talk to me anyways now did he?
The point is, it places the responsibility of delivering spammy mail to the sender. I do not have to receive it. However it allows the non-spammer to go about the internet unhindered.
Regular mail is anything SpamAssassin rates less than 5, I get a few false negatives- no real biggy.
Spam Low is anything SpamAssassin rates less than 8.
Spam High is anything SpamAssassin rates 8 or more.
My ratio of Spam High to Spam Low is 30:1. I easily scan my Spam Low in seconds. I glance at my Spam High only for entertainment before trashing it.
The false positives that end up in Spam Low are usally mailing lists that I have not white listed. When I spot one, I adjust my white list.
I am eagerly awaiting SpamAssassin 2.5 which has Bayes filtering to eliminate the very few false positives I get. As I understand it, this filtering in combination with SpamAssassin means I need to provide no feedback to the filter. Yet, spammers will have a different Bayes filter, therefore they will be unable to adapt their spam to go through my filter.
Net result: SpamAssassin Rocks!
A few weeks ago the Scunthorpe town council decided to implement a nasty words filter on all email received, just to reduce the volume of abusive email they were receiving.
The email filter worked out very well indeed - well, too well. Absolutely no mail was delivered. It took a while for them to realise that their own town name contained one particular rude word, and considering that their town name was part of their email address, all email had to have a certain word in it.
Your message was here?
Perhaps, with a flood of spam,
I deleted it.
I started using a filter called Spammunition a while ago. It's a free Bayesian filter for MS Outlook. (Not my ideal mail client but it's what we use at work). It's great. No false positives, and catches all my spam.
and one that is in use today, is to not accept any public email at all.
The US Senate and House of Representatives have their member's websites with a contact page utilizing a web form to submit letters. Since this email address is hidden by the web server, the only spam that could possibly get to senators is someone specifically writing a program to submit information for that specific web form.
Since no spammer would need to spam senators (unless someone tries to mail bomb them, but that is an other issue all together), nobody would spam them.
This also solves the problem with the post office mail and anthrax problems that happened just after 9/11. The quickest way to contact your senator is by fax, but even this web form is higher priority than snail mail.
Two infinite things: your stupidity and mine. But I'm not sure about the latter. If my sig offends you, I'm sorry.
If governments find spam unacceptable, and resort to spam filtering, and then find that unacceptable because of false positives, the next recourse is spam legislation. Therefore, false positives are good.
... "Give me a woman who loves beer and I will conquer the w
It is important to distinguish between rejecting a message (in which case the sender gets a "550 spam" indication) and discarding a message (in which case neither the sender nor the receipient is notified). Only the SMTP server can reject a message, it is too late by the time it has gotten to the message user agent (client).
If the anti-spam software rejects a message it is usually trivial for the sender to modify the message or find another delivery method and little is lost. If a message is discarded, the damage might be much greater.
Bayesian spam filters usually run on the client, and have to discard messages but there is no particular reason they couldn't run on the server.
The client can't reasonably return a "DSN" via email since the envelope from (even if known to the client) is probably a forgery, so responding would just be creating more spam. The SMTP server
can reject the message before it is accepted with an error code, it doesn't have to send an email with the error message.
From the article:
We receive over half a million incoming e-mails a month - so far the filter has blocked about 900 a week, which is about 1 in 180, much less than 1%
If only 1 in 180 messages are classed as spam, why are they using the filter in the first place? If the average amount of spam received across the board is less than 1%, then those MPs who complained of being inundated with spam must be few in number.
Why should the whole system suffer because of those MPs? They should implement their own filters if they have a problem.
The helpdesk has only received a handful of unblocking requests.
Not surprising. How are people supposed to know they're missing out on important e-mail messages if they never receive them because of the filter?
Incidentally, my ISP uses a spam filter which is completely transparent to the user. Any messages that get filtered, legitimate or otherwise, I never even know about. Most users don't even know the filter is in place. I'll be leaving them when my contract is up, being sure to first check up on the practices of any new ISP I choose.
...is that the MPs aren't filtering their e-mail, it's under the centralised control of Parliament's IT Services Dept.
Consequently, MPs are not receiving mail about e.g. the Sexual Offences Bill silently. They can't periodically check their "junk mail" folder for false positives, they have to know (via out-of-band communication) that they've had a false positive block and then go cap-in-hand to the IT Dept to ask for the mail to be released. Anything that gets caught that they don't know about, well, they won't know about.
This is why all spam filtering should be within the control of the user.
How it works:
... and then there were none