Slashdot Mirror


Gmail Spam Filter Changes Bite Linus Torvalds

An anonymous reader points out The Register's story that recent changes to the spam filters that Google uses to pare down junk in gmail evidently are a bit overzealous. Linus Torvalds, who famously likes to manage by email, and whose email flow includes a lot of mailing lists, isn't happy with it. Ironically perhaps, it was only last week that the Gmail team blogged that its spam filter's rate of false positives is down to less than 0.05 per cent. In his post, Torvalds said his own experience belies that claim, and that around 30 per cent of the mail in his spam box turned out not to be spam. "It's actually at the point where I'm noticing missing messages in the email conversations I see, because Gmail has been marking emails in the middle of the conversation as spam. Things that people replied to and that contained patches and problem descriptions," Torvalds wrote.

32 of 136 comments (clear)

  1. Not sure if this is first comment by Mouldy · · Score: 4, Funny

    Or if the other comment's got hit by spam filters

    1. Re:Not sure if this is first comment by Anonymous Coward · · Score: 4, Funny

      No, you weren't the first as I posted a reply to this thread
      two days ago and it wound up in Linus' inbox as a kernel bug-fix.

  2. This Just In by Anonymous Coward · · Score: 5, Insightful

    Individual that differs more than 6-sigma from the population's mean has trouble with automated tools designed for the average person.

    Gmail's spam filter is why email is still useful.

    1. Re:This Just In by Anonymous+Brave+Guy · · Score: 4, Informative

      Gmail's spam filter is why email is still useful.

      I might not be six sigmas from the population mean, but the aggressive filtering of Google's mail service is annoying me more and more. I don't use it myself, but quite a few of my recurring professional contacts do, often behind their own domains so there's no way to know until it breaks. Aside from the privacy implications of that, I'm getting awfully bored of finishing a day's work, e-mailing the results to wherever they need to go, and getting in the next morning to find a nasty note from Google that was sent back after I'd left saying my mail had been blocked because they considered something in the attached file a security risk. This is particularly infuriating if I'm working in the UK and sending the results to a contact on US time, because it costs between half a day and an entire day to catch up.

      --
      If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
    2. Re:This Just In by Carewolf · · Score: 4, Interesting

      Individual that differs more than 6-sigma from the population's mean has trouble with automated tools designed for the average person.

      Gmail's spam filter is why email is still useful.

      In my experience it is crap. Not as bad as Linus experience, but it stil mistook on 1 in 200 emails just like google says and that is COMPLETELY UNACCEPTABLE. Having to find important emails in the thousands of spam emails is a problem, and haven't seen any other spam filter with that many false positives.

    3. Re:This Just In by techno-vampire · · Score: 2

      Having to find important emails in the thousands of spam emails is a problem...

      Maybe it wouldn't be such a problem if you deleted everything in your spam folder daily instead of just letting it sit there until it's 30 days old and gets removed automatically.

      --
      Good, inexpensive web hosting
    4. Re:This Just In by Solandri · · Score: 5, Informative

      Not sure why Linus and you are complaining. Gmail already has a tool for eliminating false positives. You set up a filter to automatically give any email from a particular mailing list a label for that list. It's actually a great tool for auto-organizing your email if you subscribe to multiple lists like he does.

      When setting up the filter, you make sure to check the "Never send it to spam" option.

    5. Re:This Just In by gl4ss · · Score: 2

      he was complaining because without notice the behavior changed and he started missing valid emails from addresses previously he was responding to, partially without rhyme or reason, since he started missing in-between emails and sometimes would get a later email but see that there was mail in the 'thread' that he had missed due to the spam filter.

      the point is, gmail changed the spam filter without notice (like starting to mark mail "this would go to spam next week") or whatever.

      --
      world was created 5 seconds before this post as it is.
  3. State the Obvious by freeze128 · · Score: 5, Funny

    Well, Linus doesn't *HAVE* to use gmail. There are other email providers.

    1. Re:State the Obvious by Herve5 · · Score: 2

      Yeah, like Microsoft or Apple!

      No.
      Like any national, free, reasonable email provider (in my country, the post office does this).
      Like *all* our hosting service providers do, too, at no delta-cost, and in a well controlled manner if you chose an associative hosting. Many come to mind in Europe, like the belgian All2all, the french Ouvaton
      Sorry for the bluntness, that's not you but the mere idea of Torvalds registering at Google that shocked me...

      --
      Herve S.
  4. Filters will do this by damn_registrars · · Score: 2

    It is pretty much a given that the FP rate would go up as filters become more ubiquitous. This is how the spammers are winning the spam battles when people place too much faith in filters.

    As I have said before, spam is an economic problem. We won't solve the problem with filters, or with any kind of punishment (legal or otherwise); we need to look at this rationally as an economic problem.

    --
    Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
    1. Re:Filters will do this by dpidcoe · · Score: 3, Interesting

      One of the ways of combating it economically is to make it require more effort to successfully deliver spam to the target recipients. i.e. using a filter.

    2. Re:Filters will do this by damn_registrars · · Score: 2

      One of the ways of combating it economically is to make it require more effort to successfully deliver spam to the target recipients. i.e. using a filter.

      The problem is that the spammers can acquire more opportunity to get past filters (by taking over more computers for their botnets, to send more spam from with more permutations designed to confuse filters) with more ease than the time it takes to train the filters on what is spam and what is not. When using filters, it becomes an arms race - and only the spammers can win.

      In other words it already costs the spammers almost nothing to send out a deluge of billions of spam emails. They already know how to tweak the message contents and parameters to get around many common filter rules. They also know that with the exception of people who primarily use "free" (rather than work or ISP provided) email addresses, their best potential customers aren't protected by the newest and "most clever" spam filter rules any ways.

      Basically, you cannot win the war on spam with filters. The filters just keep getting worse in terms of FP and FN; this is a win for the spammers. People keep putting more time and money in to adjusting the filters but this doesn't hurt the spammers either. If you want to stop spam, you have to interrupt the flow of money. Spammers will stop sending spam when they stop getting paid.

      --
      Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
  5. Works for me - whatever that is worth by sjbe · · Score: 5, Insightful

    Individual that differs more than 6-sigma from the population's mean has trouble with automated tools designed for the average person.

    Exactly. I use Gmail and I honestly haven't had a false positive (flagged as spam when it isn't) in over two years. I still get the occasional false negative (spam that isn't flagged) at a frequency of a few per week. It's good enough that I don't even bother to routinely check my spam filter. It also is pretty good on the training - once you've spent a little time telling it what is spam and what isn't for you in my experience it is pretty good after that. Frankly if you have to check your spam filter often it isn't a very good spam filter.

    I suspect Linus has rather unusual email requirements. Perhaps Gmail isn't the ideal solution for him. Very few tools are perfect for everyone. I'm a little surprised he's having that much trouble but stranger things have happened.

    1. Re:Works for me - whatever that is worth by Tx · · Score: 5, Interesting

      I read about this a few days ago on The Register, according to one user there, this particular issue is to do with DKIM and mailing lists (the stuff Linus had issues with was all Linux kernel mailing list messages);

      bhtooefr "Basically, Google's enforcing DKIM from certain domains, and if a message is "from" someone whose e-mail host provides proper DKIM, but it's missing it, Google (and Yahoo) servers reject it. Mailing lists aren't usually set up to properly handle DKIM (being, effectively, a relay), and therefore get rejected.

      The workaround that I saw one mailing list use was to resend the e-mail from the mailing list's address, append "via (mailing list name)" to the name on the from field, and just have both the mailing list and the original author in reply-to."

      Seems like people running mailing lists need to take a look at how spam filters work, rather than mail providers changing anything. If I understand correctly, the policy is sensible and blocks a likely spam vector, and legit mailing lists could easily be set up to not fail that particular check.

      For regular mail, I'm like you guys, Google's spam filtering does a fantastic job. I never check my spam folder any more, unless I'm expecting an email and it doesn't arrive, but it's been ages since I had a false positive.

      --
      Oh no... it's the future.
    2. Re:Works for me - whatever that is worth by war4peace · · Score: 4, Informative

      GMail started flagging Youtube newsletters as SPAM but gets confused by another filter I added manually to all e-mails from Youtube. I had created a filter which adds a label to that type of e-mail, and now GMail says "This message was not sent to Spam because of a filter you created." every time I am getting an e-mail from Youtube.

      Funny, 'cause Youtube is owned by Google.

      --
      ...gis sdrawkcab (usually not responding to ACs; don't bother posting as AC)
    3. Re:Works for me - whatever that is worth by omnichad · · Score: 2

      While I might agree with most of that, there's really no reason to flag a mid-thread reply as spam.

    4. Re: Works for me - whatever that is worth by AvitarX · · Score: 3, Interesting

      I've had this problem with small websites I run. A lot of contact forms default to using the submitter as from still, I have to edit the code that sends the mail in the module to be from the site's domain and use the reply-to.

      I started having to do this year's ago, yet very few modules let you take advantage of reply-to still. Very annoying.

      --
      Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
    5. Re:Works for me - whatever that is worth by Mr.+Slippery · · Score: 2, Insightful

      Seems like people running mailing lists need to take a look at how spam filters work, rather than mail providers changing anything.

      No, you're backwards. It's up to spam filter developers to understand how mailing lists work and not falsely flag legitimate traffic. If your filter breaks a mailing list, your filter is broken.

      --
      Tom Swiss | the infamous tms | my blog
      You cannot wash away blood with blood
    6. Re:Works for me - whatever that is worth by Coren22 · · Score: 2

      Or perhaps you don't understand how mail spam filtering works?

      If I send a message to a mailing list server, and it resends the message claiming that it is me, this is wrong.

      C22@mail.com - > ML@MAILlist.com
      ML@MAILlist.com -> user@list.com (as C22@mail.com)

      list.com will always say MAILlist.com isn't mail.com, why is it sending me mail. This is a misconfigured mail relay problem, not a spam problem. MAILlist.com is not an authorized mail server of mail.com, it is completely valid to reject it as a spoofed message as it is a spoofed message.

      --
      APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  6. Poor Linus needs a gofundme by RonVNX · · Score: 4, Funny

    Apparently he can't afford to purchase decent email service. Maybe someday he'll create something important and then he can get off the crap freemail.

  7. Already resovled by Anonymous Coward · · Score: 2, Informative

    You can tell this is old news given that it was this past weekend the Linus posted an update on Google Plus stating that false positive rates were back down to normal for him.

    https://plus.google.com/+LinusTorvalds/posts/dJdkRxUCRmK

  8. Re:Boolean filters are wrong by pem · · Score: 3, Insightful

    at some time it is going to be illegal [to throw away spam]

    WTF are you smoking, and can I haz some?

    No amendment, not even the first, makes it illegal for me to throw away shit that people decide to send to me.

    Spam is a valuable resource.

    Pigshit is a valuable resource. Spam is spam. The fact that you can look for similarities in it in order to trash more of it doesn't make it a valuable resource.

  9. Re:Karma is a bitch by Barsteward · · Score: 2

    you must be the only one, i've not seen anyone else complain about git in that fashion - did you email your bug report to Linus (or did it get put in the spam folder by google)?

    --
    "The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
  10. Re:Curious by Anonymous+Brave+Guy · · Score: 2

    The reason for the wave effect is at least in part because a relatively large proportion of the spam that gets sent actually comes from a very small number of sources. Someone figures out a formula for defeating the current spam filters on enough major systems to be viable and then exploits it heavily for as long as they can. The mail services note the changes in traffic, adapt, and block that traffic. On a really good day, a major spammer actually gets taken to court and removed from the system altogether for a while, though mostly that's probably just wishful thinking.

    --
    If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
  11. The bitching is slower than the fixing by Anonymous Coward · · Score: 4, Informative

    He posted about it in G+, a googler noticed and offered to look into it. One day later The Register is feeding off the echoes and the story is slashdotted.

    "Much better now.

    Of the 100+ messages caught as spam over-night, only two were false positives (and I reported them). My email is getting back to normal."

    https://plus.google.com/+LinusTorvalds/posts/dJdkRxUCRmK

  12. Already Solved by Anonymous Coward · · Score: 4, Informative

    On the next day, Linus wrote "My email is getting back to normal."
    https://plus.google.com/+LinusTorvalds/posts/dJdkRxUCRmK

  13. History repeating?... by DrYak · · Score: 2

    Maybe someday he'll create something important and then he can get off the crap freemail.

    Yup. Given his past success with both Linux kernel and with GIT distributed source management, I too think that out of anyone Linus Torvalds might be the only guy able to effectively solve the SPAM problem.

    --
    "Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
  14. he's using gmail? by hymie! · · Score: 3, Funny

    Somebody should tell Linus about this great new operating system I run at home. I have sendmail running on my machine, and it lets me control my spam filters and everything.

    It's called "Linux". I highly recommend it.

  15. other providers by Herve5 · · Score: 2

    I for one am extremely shocked that the above post ('use other providers') be flagged as funny.
    Torvalds is the last person I'd imagine registering an email address @ Google.
    Wise as he may have been, sorry, but to me he's a moron now just because of this.
    I just hope I won't evolve his way when getting older.

    --
    Herve S.
    1. Re:other providers by jenningsthecat · · Score: 2

      Torvalds is the last person I'd imagine registering an email address @ Google... I just hope I won't evolve his way when getting older.

      Age has nothing to do with it. I'm older than Torvalds, and I refuse to use Gmail.

      --
      'The Economy' is a giant Ponzi scheme whose most pitiable suckers are the youngest among us and the yet-unborn.
  16. domain issues by v(*_*)vvvv · · Score: 4, Interesting

    From his original post, there is a clear date he claims the FP rate to have gone up... so this isn't a blanket Gmail FP rate issue, but rather a Gmail or spam blacklist incident, which is quite different from what the summary would suggest. As of right now:
    http://mxtoolbox.com/SuperTool.aspx?action=blacklist%3aLKML.ORG&run=toolpage

    lkml.org Added to UCEPROTECTL2

    Uceprotectl2 Automatically Delists Entries

    This blacklist does not offer any form of manual request to delist. Your IP Address will either automatically expire from listing after a given timeframe, or after time expires from the last receipt of spam into their spamtraps from your IP Address.

    Uceprotectl2 Accepts Payments Or Donations

    This blacklist does support a manual request to remove, delist, or expedite your IP Address from their database upon Payment or Donation of fees to their organization. Please note the following; 1) MxToolBox does not in any way advocate the paying of removal from any blacklists. 2) Removal requests that are submitted without addressing the core problem will likely result in your IP Address being relisted in the database which can cause subsequent problems and extended listing periods without release.

    More information about UCEPROTECTL2 can be found at their website: http://www.uceprotect.net/

    Reason for listing - Net 146.185.176.0/21 is UCEPROTECT-Level2 listed because 36 abusers are hosted by RCN-ASN - Reality Check Network Corp./AS46652 there. See: http://www.uceprotect.net/rblc...

    UCEPROTECTL2 seems a bit shady, but I am not blacklist expert.

    Also as a side note, any spam filter that attempts context evaluation has a tendency to mark emails with code or special character formatting as spam. Even emails with links. So for someone like Linus to have a higher blanket spam FP rate is also not surprising.

    The best gmail feature is the "never treat as spam" filter.