Slashdot Mirror
A Look At Google's Email Spam Prevention
CNet has a story about the security measures Google employs to protect their email systems and fight the never-ending war on spam. Their Postini team, acquired two years ago, has a variety of monitoring tools and automated response systems to find and block undesirable messages. Quoting:
"The system scores each message on numerous combinations of criteria, assigning a weight to each and then comparing the score to those in a database of several hundred thousand message types that have been flagged as good or bad from Postini honey pots and customer spam reports. ... To block fresh spam attacks not covered by existing heuristic technologies and viruses not covered by existing signature databases Postini relies on proprietary Zero-Hour technology to identify new outbreaks that show up in the traffic patterns and quarantine them for later rescanning. Customers can also create and build out their own white lists of message senders they trust and blacklist others they don't trust. It takes an average of 150 milliseconds for a message to be scanned by the antivirus engines that Postini licenses from McAfee and Authentium.
This is great for business mail too... small company where I work was literally BURIED with spam until we moved to gmail. Since their mail addresses were "in the open" on our website for years, some of them get 200+ spams a day. Now, if 1 in 1000 passes, it's a bad day. Also, in my private inbox, I had an VERY old mail address still redirected to gmail address... turned out that was the source of 1/2 spams (100+ / day). But those were filtered too without problem. So far so good... not a single false detection for ham. Nothing but praise so far. Disclaimer: I do not work for gmail. I am the genuine satisfied customer with smile on my face, from "after" picture, as seen on TV!
At least 75% of my spam is addressed as though it was sent from *my* gmail account. Of course, it's easy to set up a filter to reject all such spam, but then I lose the ability to send reminder messages to myself. Seems like it would be extraordinarily simple for google to outright reject messages that claim to be sent from their servers that in fact were not. I sure wish they would!
I have had a similar experience with Postini, but from a different point of view. I usually use my own mailserver to send emails, and in the beginning I was greylisted and occasionally blocked by a few servers here and there, but after just a few quick emails here and there to ask why I was blocked, I was always promptly unblocked. I just use it for personal email so I'm not sending commercial or bulk emails. And before someone asks, no it's not on a dynamic IP or anything, it's in a fairly large colocation facility.
/., after all.)
Google is the only mail service that I know of who still just won't accept my emails. They make it very difficult to contact them. There is a form buried somewhere in their help system, but it says that they won't respond unless they need additional info from you, which leads me to believe that they never actually read anything submitted through that form. (I have tried a few times.) They also specifically say they don't take whitelist requests. I have SPF records, I have correct reverse DNS, I'm not on any blacklists, etc.
This means when I send emails to my friends who use Gmail, or comparies who use Postini, I get blocked without cause. Then I have to use a different server. It's kind of annoying.
(Why do I use my own email server? Because I can. This is
Have you noticed? GMail gives one no way at all to sort the captured spam. Since I still endure false positives from the system and there is NO way to disable or bypass it, having means to sort all of it by From:, To:, and other criteria would make it easier to identify the false positives and rescue them from the trash bin.
Well, I'll take that back, in part: that applies to the Webmail interface, but if ones uses IMAP with a local IMAP client, then the spam folder could be subscribed and sorted within the client. God only knows how GMail's system interprets the dragging of a message from Spam to Inbox via IMAP: does that automatically whitelist that sender in the future, or do I have to still log into the Web site and identify it as Not Spam manually?
So by using gmail, am I indirectly making money for McAfee?
...because it's actually not working - Gmail spam filter recently became very ineffective - i have to classify about 5-10 Viagra spams daily. (Google, have you heard of it? geez!) then it occurred to me that a while ago Gmail captcha was cracked, so I imagine spammers send themselves hundreds of spams only to classify them as "non-spam". - as a consequence, spams are now slipping through the crowd-sourced filter because the crowd is infiltrated. c'mon google this can't possibly that hard to fix!
I had a similar experience; I run my own mail server, send no bulk mail whatsoever, and both Postini and GMail independently decided I was a spammer. No DNSBLs had me listed, ReturnPath was happy, etc. Meanwhile, I was blocked from sending mail to my lawyer, my financial advisor, my chiropractor, etc., all of whom turned out to be downstream from Google. Despite Google's claims that the customer is in full control of filtering, none of them were able to get at my e-mail without getting their sysadmins involved - which often required discovering that they had sysadmins at all.
Worse, Postini's spam filtering takes its own output as input. Once it's scored a message of yours as spam, future messages will be more likely to score as spam - which of course makes any subsequent messages even more likely to score as spam. Brilliant. At one point, my spam score from a triple-signed (SPF/DK/DKIM) server was 98 out of a possible 100.
Google's philosophy of "we don't do it unless we can automate it" works horribly when it comes to customer service. There's no feedback loop, no whitelisting, no channels, no nothing. It's SPEWS all over again, or perhaps the Kafka International Airport.
But Google has no reason to worry about false positives; the more messages they call spam, the more spam they can say they blocked. Perverse incentives.
I've set up GMail to filter my email and by comparison I'd say one or two spams get through. So I'm very happy with GMail's level of coverage. It's not perfect but it makes things tolerable. I'm not at all happy with Yahoo's level of coverage. Yahoo allegedly also has spam filters, but I've yet to see they actually work. It's not uncommon to find my email box filled with Nigerian and other scams.
For what it's worth, Gmail has been just the opposite for me. It's Yahoo and AOL which randomly decide to block me -- sometimes with some cause, sometimes just because it's on a residential connection.
Yet Gmail never so much as greylists me -- everything goes straight through, every time.
Don't thank God, thank a doctor!