Slashdot Mirror


Aggressive Email Filtering Blocks Political Debate

Stephen writes "Many of us have spam blockers operating on our mail. But according to this BBC article, when British members of parliament starting having their emails filtered last month, it stopped them talking about genuine political business such as the Sexual Offences Bill, and prevented them receiving some constituents' emails." This problem has bit me on the bum a few times too. About 1 message in every 250 spam is a false hit. Course thats about once a day :(

29 of 392 comments (clear)

  1. Maybe good by jhouserizer · · Score: 4, Insightful

    These types of incidents may be good in the long run - if it makes law makers "wake up" to the problem of spam.

    ...We can only hope... Perhaps we could even start bombarding law makers with spam ourselves? - that would raise their awareness!

  2. Regulation by Marco_polo · · Score: 5, Funny

    I can just imagine the outrage if this happened to the bush administation.

    'what do you mean no one got my emails?'

    'It seems your.. uh... last name is causing some issues with spam filters sir'

    'That's it.. lets bomb the spammers'

    --
    I am the lord of the pun. Dance Knave!
  3. Maybe we Slashdot can buy this filter technology.. by gpinzone · · Score: 5, Funny

    ...to eliminate all the dupe stories!

  4. Remove all their filters by 3vi1 · · Score: 5, Insightful

    I think polititians shouldn't have any filters on their e-mail.

    After about 2 weeks of what the average person goes through, we'd see stronger anti-spam legislation/penalties.

  5. thats an easy one by xao+gypsie · · Score: 3, Funny

    it stopped them talking about genuine political business
    thats because they no longer knew how to enlarge their penises and missed being notified that some russian woman wanted them so badly that it hurt.
    that would certainly stop our gov't, at least..

    xao

    --


    xao
    http://TheHillforum.hopto.org
  6. Re:Maybe we Slashdot can buy this filter technolog by Anonymous Coward · · Score: 3, Funny

    ...to eliminate all the dupe stories!

  7. False Positive by propheci · · Score: 5, Insightful

    the problem is that just by knowing there could be a false positive, you have to examine all your filtered spam, which makes the spam filter useless in the first place.

    1. Re:False Positive by troc · · Score: 3, Interesting

      Kind of - I also find I end up reading, or at least checking, all teh stuff my mail app sends to the spam folder - and I guess I get one false positivie every 200 or so spams. However I find it much quicker browsing the spam in the spam folder, knowing it's probably spam than trying to weed out the spam in my inbox.

      So, the filtering perfomed by my mail app does save me some time but not as much as it could if the filtering were perfect.

      Troc

      --
      Troc's dubious podcast and blog: http://www.trocnet.net
    2. Re:False Positive by lessthan0 · · Score: 4, Interesting

      A better solution (the one I use) includes a summary report of spam filtered each day. The report lists the number of spams from each sender and I can usually spot valid mail in the list of "From"s without having to look at a single message.

      If I spot a false positive, then I dig into my spam archive for that day and check it out.

      I use the spastic filter:
      http://spastic.sourceforge.net

    3. Re:False Positive by ajs · · Score: 3, Informative

      Not at all, and you're thinking about far too simple a model.

      With SpamAssassin, I deal with spam in 3 ways:

      1. Mail that gets a score of 20 or more is sent to /dev/null. Mail would have to be carefully crafted to achieve a 20 unless it truly is spam. Such effort is not to be rewarded :-) Keep in mind that mail about laws on sexuality or other simple examples cited in this article would never get NEAR a 20.

      2. Mail that triggers both the Bayesian and Razor2 tests is sent to /dev/null. This is a very nice way to identify that a) there's a consensus that this very message is spam and b) my local mail patterns indicate that this is spam.

      3. Anything else with a score of 4 or more is marked in the subject line and I have a virtual mailbox in my mail client that I use to glance at the from addresses. If something looks plausible, I check it out.

      As of the development version of SpamAssassin that I'm using (about a week old out of CVS), I get a false positive rate of about 1:100-200 messages and during testing over the last couple of months, I copied the messages that would have gone to /dev/null to a mailbox that I scanned carefully. None of the messages that I would have thrown out were non-spam.

      I get a LOT of mail form lists, spammers, friends, random people on the net, machines spewing status, etc. I feel that I'm a reasonably good QAer for this sort of thing, and the new SpamAssassin will rock your world (and the spammers')!

  8. The type of spam filtering? by mrs+clear+plastic · · Score: 4, Insightful

    I can easily see why this may be happening. The types of filters that use keywords can easily fall into this.

    I understand that keywords and phrases such as
    'free money' 'zero percent financing' 'win
    million dollars' 'sex xxxxx pictures!' and so
    on can trigger many filters.

    I would like to think that the better designed
    filters would use a combination of key words as
    well as suspicious domain names and/or IP
    address blocks to do filtering.

    The spam filter that is used on my email account does not filter out, but it does add the word
    'SPAM?' into the subject line of the email message. I can then see right away if it is
    really spam or is something mistaken by the filter for spam. The message is not blocked, though.

    Mark

    --
    Cleara
  9. That's a real problem with a democracy by Hairy_Potter · · Score: 5, Funny

    how to balance open access to constituents without being overwhelmed.

    Perhaps Parliament could consider some of the steps that the American Congress has taken. The American Congress has a de facto filter built in to prevent Joe Random crazy from flooding their representatives with spurious requests. Most Congressional requests, letters, phone calls, faxes and emails are tossed out unless they come from certain designated people known as lobbyists. These lobbyists have worked hard to cultivate contacts in the Congress, and can get better results from one office visit than 1,000 letters from voters. In a way, they're professional access voters.

    So, maybe the UK could restrict access to just professional lobbyists, it works very well in the US.

  10. Webmail and "spamlets" by germinatoras · · Score: 4, Informative

    A similar problem happens with free Webmail or adversiting-supported e-mail accounts. The small advertisements attached to the bottom (I call them "spamlets") will sometimes trigger mail filters.

    Watch out for this if you're sending a message from e.g. Yahoo! to Hotmail, who both attach spamlets and both filter incomming mail. They also will not send rejection notices to the sender, so you may never know if you message got through.

  11. Re:Spam filter = Censorship by Kombat · · Score: 5, Insightful

    You're confusing the right to free speech with the privilege of being heard.

    --
    Like woodworking? Build your own picture frames.
  12. Re:Glad I don't live there by citog · · Score: 3, Insightful

    Because most of the people they should be talking to , i.e. constituents, also use this insecure system. In reality, most of the politicians I know use email aren't discussing state nuclear secrets or troop movements. If certain politicians are likely to leak sensitive information this way I would be far more concerned about what gets picked up from the far more insecure system - VoA (Voice over Air)

  13. Public email? by briancnorton · · Score: 3, Insightful
    It's truly amazing that anybody could make an attempt to use their public email for official business. In general, three accounts is all that you need to never have to read spam again.

    Specifically for the parliment, I dont see why they dont just whitelist all other parliment members.

    --

    People who think they know everything really piss off those of us that actually do.

  14. filtering is good.... by morgajel · · Score: 5, Insightful

    ...but not perfect.
    Ok, here's the situation as I see it.
    We have a problem: Spam

    We need a solution.
    So far filtering has been working good and is slowly getting better, but there's always gonna be the chance for false positives.
    so how do we stop this?
    I have no clue.
    We should probably start cracking down on open relays, even use governmental pressure if needed (on spammers in our countries and on the governments of other countries). They serve no real purpose other than facilitaing spam.
    What else can we do? Go after spammers legally. We need to make them pay. I bet if 1000 people sued ralsky for $500 a piece he'd start to take notice, but he still wouldn't learn. Some states, like washington, are doing that, and it seems to be working, or at least getting the spam recievers a little extra cash. If I lived there, I know I'd try it at least once. Hell, I might even pay for my braodband connection with the money I got from spammers:)

    I've heard people recommend opt-out lists like they use for telemarketers- that's not gonna work because spammers are much more slimy- they'll use the opt-out list as a verified list.

    We're not left with many choices, besides educating people to simply delete spam and DON'T buy from it. make it cost spammers money. if they sell even one thing, they they're winning.

    I took a slightly fun approach. I'm building a list of 'legit' companies that sell your email address to spammers. What I did was bought a domain, and whenever I signed up for something, I used the companies name@ the domain, and had it all forward to one account. so when I get spam to musiccity@mydomain.com, I know that musiccity sold my email address (which they did).

    Does anyone else have any Ideas how to stop spam? if so, save the redundant mods and reply.

    --
    Looking for Book Reviews? Check out Literary Escapism.
  15. False positives are a fact of life by sludg-o · · Score: 4, Insightful

    We offer SpamAssassin at the college where I work. I always tell new users that any spam blocking system, no matter how good, will eventually block something that was legitimate. That's why I don't write procmail recipies that redirect mail flagged as spam to /dev/null. You gotta put it in a seperate folder and you are asking to get burned if you don't skim the subjects and senders every couple days. Also, they should be whitelisting messages from addresses in their domain.

    I don't see how this is news. It's just an example of bad system administration.

  16. Many Issues by CleverNickedName · · Score: 3, Funny

    I'm sure the filters caused many problems with the "Hot, horny housewife" bill and the new "Extra six inches" tax debate.

    --


    Unfortunately, I am not Wil Wheaton
  17. Hit-and-miss filters by Xesdeeni · · Score: 5, Insightful

    By definition filters are hit-and-miss and non-deterministic. I get almost exclusively SPAM with spoofed return addresses. How about this solution:

    1. Sending mail server generates a tx content key based on the contents of an e-mail being sent.
    2. Sending mail server uses the tx content key with a private key to create a confirmation key.
    3. Sending mail server sends the e-mail, along with the confirmation key to the receiving server.
    4. Receiving mail server generates a rx content key from the e-mail contents.
    5. Receiving mail server sends the rx content key and the confirmation key back to the sending mail server.
    6. Sending mail server uses its private key plus the rx content key to re-generate the confirmation key.
    7. Sending mail server compares the confirmation keys.
    8. If the keys match, the receiving mail server allows the mail to enter the recipient's mailbox.
    9. If the keys don't match, the mail is bounced.

    The keys are in place to keep the SPAMmer from tagging along on a valid return address with mail that address didn't send. This technique also keeps the second transaction to a minimum exchange of keys. The keys add traffic, but the eliminated SPAM traffic more than makes up for the penalty. As more and more mail servers are updated with this feature, spoofing is all but eliminated. The remaining "spoofable" domains can be explicitly severed from the net or blocked.

    Xesdeeni

  18. Re:Work around by Cy+Guy · · Score: 4, Interesting

    create a text file [&] zip it

    Unless the recipient is expecting this they should just delete the message. I routinely delete any email that has zipped attachments unless I have previously agreed with the sender to send it that way. (That's assuming the recipients mailserver doesn't routinely strip zip files off as an enterprise virus protection measure in the first place.)

    But one way your suggestion could be modified that will work for anyone whose email can view HTML is to print your message to graphic file, convert it to a GIF and embed it into a simple a webpage.

    The reader will open the file and see what looks like a text message, but it actually will be the GIF image of your message.

    Most filters don't block HTML and GIF files.

  19. true story by circletimessquare · · Score: 5, Interesting

    i got in a fight with an ex-girlfriend and we ceased speaking for awhile

    i became further incensed because she never contacted me after the fight

    we didn't talk for 2 months

    finally, i contacted her and said "why didn't you get back to me??!!"

    she said, "you didn't get my email?"

    i looked, and there it was, 2 months back, in my spam folder (yes, i keep all of my spam, the folder is gigantic)

    although you could make a joke about emails from girlfriends being called spam, in this particular case, considering the chance at reconciliation that was lost and the feelings involved, it was definitely not funny at all ;-(

    so i can say, with certainty, that my personal life has been greatly and adversely affected by spam.

    you can hate spam for all sorts of reasons, but for me, it's personal.

    --
    intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
  20. They didn't even warn the MPs? by sebi · · Score: 3, Informative

    According to the article the system was implemented without prior warning. What they should do is educated the users on how to implement spam filtering on their machines and not stop messages from going through at all.

    In my e-mail client spam is marked in a different color, and by now the success rate seems pretty good, but I still don't trust it enough to auto-delete them. Spam sucks, but false positives not getting through might be worse than boobie mail getting blocked. In this case members of a governing body are affected. They should be working on legislation against spam, instead of having their hands held by the IT department.

  21. similar has happened in US by jdunlevy · · Score: 4, Informative
  22. No, That's a real problem with a democracy by intermodal · · Score: 3, Insightful

    So, maybe the UK could restrict access to just professional lobbyists, it works very well in the US.

    Works well for who? I don't see how it helps the average joe citizen who wants to get his point across unless he donates money somewhere. Corporations have tons of cash to throw at it. So if Jimmy Lobbyist has more access than Joe Sixpack, thats a problem. repetition and filtering be damned. It is the duty of a representative democracy to represent those they are representative of, and if they aren't willing to take into account every email and letter and fax and phone call they get in their decisions, then it's a stone's throw away from not having elections at all, especially when you consider that when voting the only two candidates who generally have a chance is a lesser of two evils situation.

    --
    In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
  23. Comment removed by account_deleted · · Score: 4, Insightful

    Comment removed based on user account deletion

  24. Re:Maybe Wake Up Call for "Filters" by Zeinfeld · · Score: 5, Insightful
    As stated lower in this thread, if you have to manually check your spam folder for "false positives", the filter is utterly useless since you are still checking for spam

    I attended the conference on spam at MIT. The conference would have been more accurately labelled a 'solving spam with the hammer we know about' conference since no other solutions were accepted - although several people besides myself submitted authentication based papers.

    The big problem with the Bayes approach is false positives. Lots of great statistics were quoted but the claims were simply not credible. I don't believe that Spam is such a simple problem that the performance of naive Bayesian techniques is several orders of magnitude better on that problem than any other.

    So really the trick is to swing the problem arround. START from the problem of making sure that anyone with a legitimate reason to contact me can do so without interference from statistical filtering techniques. The proper place to apply those is on the mail I cannot authenticate in that way.

    I dislike the bounce-back loop as a filter for personal correspondence. I think it is great for the purpose of a lightweight authentication mechanism for mailing list subscriptions. I get very irritated when people use it to filter email, particularly since all my email is signed. People should not substitute their ad hoc authentication mechanisms without first supporting deployed standards.

    The other problem with call back loops is that if they are used widely they will become a bigger problem than the spam, this is why I have been urging Microsoft et. al. NOT to support them. The trick that the spammers have developed to get round the callback loop is to steal addresses off mailing list archives and send forged messages to the other members of the list. So work out the effect that deployment of the naive bounceback hack would have.

    --
    Looking for an Information Security student project suggestion?
    Try http://dotcrimeManifesto.com/
  25. Re:Work around by AndroidCat · · Score: 4, Insightful
    Most filters don't block HTML and GIF files.

    Oh? Many people set their filters to tag'n'bag (or simply dump) any non plain-ascii email. I treat any email with HTML, base64, or an attachment of any kind as probably spam and potentially dangerous, and inspect it before reading it.

    --
    One line blog. I hear that they're called Twitters now.
  26. What a Sick Sales Plug!!! by tacocat · · Score: 3, Interesting

    I hate to do this because it's only partially complete. But I have a concept worked out on how to handle spam that works extremely well and removes the chance of false positives, especially from Real People.

    It's not a money-making scheme, but it is prior-art <grin>.

    The idea is a hybridization of SpamAssassin and tmda (tagged message delivery agent) wherein you accept all email into your inbox and the spam goes into a spam mailbox. Nothing New...

    The cool part comes in when you start automating the spam_mail similar, at least conceptually, to what I have on my website. Shameless plug here

    The idea is that you send out an email confirmation, similar to tmda, for only that email which is considered spam (by SpamAssassin). This means that most of your regular communications would go unhindered. But it would also make casual contact via email the easy and simple function that it is supposed to be.

    These notions of having an email list of only your known contacts is a pain in the arse and most times met with extreme hostility. This is especially true if you are attempting to contact someone privately from an email list, or from a solitication from their website.

    I have to warn you that if you use the code as described on my website you will probably break your server in the first day. I've rewritten it to scale much better (1,000 spams every 10 minutes). But I haven't had the chance to post the new code. But conceptually it rocks!

    I've processed something like 20,000 emails without taking a single false positive, unless the original sender vegged... but then he didn't really want to talk to me anyways now did he?

    The point is, it places the responsibility of delivering spammy mail to the sender. I do not have to receive it. However it allows the non-spammer to go about the internet unhindered.