Slashdot Mirror

← Back to Stories (view on slashdot.org)

A Look At Google's Email Spam Prevention

Posted by Soulskill on from the click-here-to-report-as-spam dept.

CNet has a story about the security measures Google employs to protect their email systems and fight the never-ending war on spam. Their Postini team, acquired two years ago, has a variety of monitoring tools and automated response systems to find and block undesirable messages. Quoting: "The system scores each message on numerous combinations of criteria, assigning a weight to each and then comparing the score to those in a database of several hundred thousand message types that have been flagged as good or bad from Postini honey pots and customer spam reports. ... To block fresh spam attacks not covered by existing heuristic technologies and viruses not covered by existing signature databases Postini relies on proprietary Zero-Hour technology to identify new outbreaks that show up in the traffic patterns and quarantine them for later rescanning. Customers can also create and build out their own white lists of message senders they trust and blacklist others they don't trust. It takes an average of 150 milliseconds for a message to be scanned by the antivirus engines that Postini licenses from McAfee and Authentium.

22 of 176 comments (clear)

  1. Don't care how they do it.. by Finallyjoined!!! · · Score: 5, Insightful

    I now get a couple of shed loads less spam. I used to check the apam directory for false positives. Don't bother doing that either.

    Go gmail :-)

    --
    If I had an Ass, I'd call it Fanny Bottom, then I could slap my Ass; Fanny Bottom, on the Arse.
    1. Re:Don't care how they do it.. by hansraj · · Score: 5, Informative

      Pfft.. the internet became sentient sometime ago and used to babble like a baby. Since whatever it said was pretty much garbage, it was impossible for anyone to correctly figure out whether the noise was the baby's (spam) or from the tv (non-spam?). Now that the internet speaks more coherently it is far more easier for Google to figure out stuff that is coming from the internet - spam that is. It is rather obvious actually.

      I wonder why yahoo has a miserable spam filter though; maybe Yahoo is like the careless parent who never gave a shit to figure out when the baby stopped babbling. And judging by the kind of spam I get in my hotmail box (it is all from microsoft), probably MS would be like those parents who insist on babbling themselves when the baby is around.

      There, mystery solved! Now no one has to RTFA. Now if only someone made this into a car analogy for the greater good.

    2. Re:Don't care how they do it.. by jo42 · · Score: 4, Insightful

      Don't care how they do it..

      Then I suggest that you don't really belong on /. ...

    3. Re:Don't care how they do it.. by Threni · · Score: 3, Informative

      I get loads more spam than I used to. Something broke in Google's spam prevention about 4 months or so ago, and it's not been fixed yet. I redirect my email to my phone, where I get a notification of new email, and I've had to turn the sound and vibrate alert off because I got too much spam coming through.

    4. Re:Don't care how they do it.. by DrXym · · Score: 4, Interesting
      Spam is now so bad for me on my home account that I reckon for every 100 messages, only two or three are legitimate contact. I literally get 200-300 spams a day. Bayesian filters will get rid of about 20%, and rules I've added such as deleting any email with cyrillics or other foreign characters still leave me with 100 or so to delete manually.

      I've set up GMail to filter my email and by comparison I'd say one or two spams get through. So I'm very happy with GMail's level of coverage. It's not perfect but it makes things tolerable. I'm not at all happy with Yahoo's level of coverage. Yahoo allegedly also has spam filters, but I've yet to see they actually work. It's not uncommon to find my email box filled with Nigerian and other scams.

    5. Re:Don't care how they do it.. by Anonymous Coward · · Score: 4, Funny

      STOP! The internet is not really sentient (yet).

      Am too!

  2. "Postini"? by John+Hasler · · Score: 4, Insightful

    My previous ISP switched me over to Postini with no advance notice (we got a cheery note from marketing after the deed was done). Blocked half the spam and half the ham. They told us how to disable the filtering "features" but it turned out that all the filtering could not be turned off.

    I'm not with that ISP any more.

    --
    Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
    1. Re:"Postini"? by icydog · · Score: 4, Interesting

      I have had a similar experience with Postini, but from a different point of view. I usually use my own mailserver to send emails, and in the beginning I was greylisted and occasionally blocked by a few servers here and there, but after just a few quick emails here and there to ask why I was blocked, I was always promptly unblocked. I just use it for personal email so I'm not sending commercial or bulk emails. And before someone asks, no it's not on a dynamic IP or anything, it's in a fairly large colocation facility.

      Google is the only mail service that I know of who still just won't accept my emails. They make it very difficult to contact them. There is a form buried somewhere in their help system, but it says that they won't respond unless they need additional info from you, which leads me to believe that they never actually read anything submitted through that form. (I have tried a few times.) They also specifically say they don't take whitelist requests. I have SPF records, I have correct reverse DNS, I'm not on any blacklists, etc.

      This means when I send emails to my friends who use Gmail, or comparies who use Postini, I get blocked without cause. Then I have to use a different server. It's kind of annoying.

      (Why do I use my own email server? Because I can. This is /., after all.)

    2. Re:"Postini"? by macraig · · Score: 3, Interesting

      Have you noticed? GMail gives one no way at all to sort the captured spam. Since I still endure false positives from the system and there is NO way to disable or bypass it, having means to sort all of it by From:, To:, and other criteria would make it easier to identify the false positives and rescue them from the trash bin.

      Well, I'll take that back, in part: that applies to the Webmail interface, but if ones uses IMAP with a local IMAP client, then the spam folder could be subscribed and sorted within the client. God only knows how GMail's system interprets the dragging of a message from Spam to Inbox via IMAP: does that automatically whitelist that sender in the future, or do I have to still log into the Web site and identify it as Not Spam manually?

    3. Re:"Postini"? by rm999 · · Score: 3, Insightful

      "there is NO way to disable or bypass it"

      Have you looked into filters? They added an option to "Never send it to Spam" about a year ago. You can create custom white lists with this, or just include everyone in the filter and totally bypass the spam filter.

    4. Re:"Postini"? by Jay+L · · Score: 5, Interesting

      Google is the only mail service that I know of who still just won't accept my emails.

      I had a similar experience; I run my own mail server, send no bulk mail whatsoever, and both Postini and GMail independently decided I was a spammer. No DNSBLs had me listed, ReturnPath was happy, etc. Meanwhile, I was blocked from sending mail to my lawyer, my financial advisor, my chiropractor, etc., all of whom turned out to be downstream from Google. Despite Google's claims that the customer is in full control of filtering, none of them were able to get at my e-mail without getting their sysadmins involved - which often required discovering that they had sysadmins at all.

      Worse, Postini's spam filtering takes its own output as input. Once it's scored a message of yours as spam, future messages will be more likely to score as spam - which of course makes any subsequent messages even more likely to score as spam. Brilliant. At one point, my spam score from a triple-signed (SPF/DK/DKIM) server was 98 out of a possible 100.

      Google's philosophy of "we don't do it unless we can automate it" works horribly when it comes to customer service. There's no feedback loop, no whitelisting, no channels, no nothing. It's SPEWS all over again, or perhaps the Kafka International Airport.

      But Google has no reason to worry about false positives; the more messages they call spam, the more spam they can say they blocked. Perverse incentives.

    5. Re:"Postini"? by SanityInAnarchy · · Score: 4, Interesting

      For what it's worth, Gmail has been just the opposite for me. It's Yahoo and AOL which randomly decide to block me -- sometimes with some cause, sometimes just because it's on a residential connection.

      Yet Gmail never so much as greylists me -- everything goes straight through, every time.

      --
      Don't thank God, thank a doctor!
    6. Re:"Postini"? by veganboyjosh · · Score: 3, Funny

      I used to get Snopes candidates from my mother-in-law a few years ago. I used to delete them without saying anything. Then I figured I'd try to teach her about the internet, and trusting things you receive in your inbox. I made an effort to track down whatever outrageous story she forwarded on snopes or wherever else, so that she'd see they weren't true, and stop sending them.

      Now, instead of getting emails from her with "I wonder if this is true. It sounds so amazing!", I get "I already checked Snopes, and while this one isn't real, it makes for a good story!" MLIA.

  3. Postini may or may not work, by Anonymous Coward · · Score: 5, Funny

    but what I really want to tell you is that I've inherited a great deal of money and I need someone to help me transfer it to the US. I live in Nigeria. You all seem to be great gentleman, so I will pay appropiately.

    Contact me.

    1. Re:Postini may or may not work, by Anonymous Coward · · Score: 3, Funny

      You all seem to be great gentleman

      You must be new here.

  4. Toughest spam by Pessimist+Cynic · · Score: 5, Funny

    They can filter out the obvious spam mail, but some spammers are so clever and so well hung - because they've taken some DrMaxMan to acquire an enlarged sexual wand with which you can perform better and be bigger for f.r.e.e - that they can actually embed their spam offers inside real messages in such a way as to be completely undetectable by filters.

  5. Praise Gmail by zhilla2 · · Score: 3, Interesting

    This is great for business mail too... small company where I work was literally BURIED with spam until we moved to gmail. Since their mail addresses were "in the open" on our website for years, some of them get 200+ spams a day. Now, if 1 in 1000 passes, it's a bad day. Also, in my private inbox, I had an VERY old mail address still redirected to gmail address... turned out that was the source of 1/2 spams (100+ / day). But those were filtered too without problem. So far so good... not a single false detection for ham. Nothing but praise so far. Disclaimer: I do not work for gmail. I am the genuine satisfied customer with smile on my face, from "after" picture, as seen on TV!

  6. But what about spam from "me"? by Peaquod · · Score: 3, Interesting

    At least 75% of my spam is addressed as though it was sent from *my* gmail account. Of course, it's easy to set up a filter to reject all such spam, but then I lose the ability to send reminder messages to myself. Seems like it would be extraordinarily simple for google to outright reject messages that claim to be sent from their servers that in fact were not. I sure wish they would!

    1. Re:But what about spam from "me"? by hidden · · Score: 4, Insightful

      Keep in mind:
      It's a perfectly legitimate (and common) for non-webmail users to have their outgoing server be their local ISP. So if google did what you're suggesting, all those people that use an IMAP client to receive their gmail, and send via their ISP wouldn't be able to send to other gmail users

  7. McAfee by contrapunctus · · Score: 4, Interesting

    So by using gmail, am I indirectly making money for McAfee?

  8. Re: I do care how it works by npwa · · Score: 3, Interesting

    ...because it's actually not working - Gmail spam filter recently became very ineffective - i have to classify about 5-10 Viagra spams daily. (Google, have you heard of it? geez!) then it occurred to me that a while ago Gmail captcha was cracked, so I imagine spammers send themselves hundreds of spams only to classify them as "non-spam". - as a consequence, spams are now slipping through the crowd-sourced filter because the crowd is infiltrated. c'mon google this can't possibly that hard to fix!

  9. Re:now am worried !! by Ron+Bennett · · Score: 4, Insightful

    150 milliseconds sounds fast, but equates to only 7 messages per second.

    Sure that may be faster, presuming it's a deep intensive scan, than what one can do on their home PC, and yes Google has zillions of boxes ... but anyways, my point is that 7 messages per second illustrates the very real, high cost of dealing with spam; scanning of just a million messages, which is a fraction of the spam volume, at 7 messages per second, takes well over a day of computer time.

    Ron