A Look At Google's Email Spam Prevention

CNet has a story about the security measures Google employs to protect their email systems and fight the never-ending war on spam. Their Postini team, acquired two years ago, has a variety of monitoring tools and automated response systems to find and block undesirable messages. Quoting: "The system scores each message on numerous combinations of criteria, assigning a weight to each and then comparing the score to those in a database of several hundred thousand message types that have been flagged as good or bad from Postini honey pots and customer spam reports. ... To block fresh spam attacks not covered by existing heuristic technologies and viruses not covered by existing signature databases Postini relies on proprietary Zero-Hour technology to identify new outbreaks that show up in the traffic patterns and quarantine them for later rescanning. Customers can also create and build out their own white lists of message senders they trust and blacklist others they don't trust. It takes an average of 150 milliseconds for a message to be scanned by the antivirus engines that Postini licenses from McAfee and Authentium.

Comments

Don't care how they do it..

[Finallyjoined!!!]

I now get a couple of shed loads less spam. I used to check the apam directory for false positives. Don't bother doing that either.

Go gmail :-)

Re:Don't care how they do it..

[hansraj]

Pfft.. the internet became sentient sometime ago and used to babble like a baby. Since whatever it said was pretty much garbage, it was impossible for anyone to correctly figure out whether the noise was the baby's (spam) or from the tv (non-spam?). Now that the internet speaks more coherently it is far more easier for Google to figure out stuff that is coming from the internet - spam that is. It is rather obvious actually.

I wonder why yahoo has a miserable spam filter though; maybe Yahoo is like the careless parent who never gave a shit to figure out when the baby stopped babbling. And judging by the kind of spam I get in my hotmail box (it is all from microsoft), probably MS would be like those parents who insist on babbling themselves when the baby is around.

There, mystery solved! Now no one has to RTFA. Now if only someone made this into a car analogy for the greater good.

Postini may or may not work,

[Anonymous Coward]

but what I really want to tell you is that I've inherited a great deal of money and I need someone to help me transfer it to the US. I live in Nigeria. You all seem to be great gentleman, so I will pay appropiately.

Contact me.

Toughest spam

[Pessimist Cynic]

They can filter out the obvious spam mail, but some spammers are so clever and so well hung - because they've taken some DrMaxMan to acquire an enlarged sexual wand with which you can perform better and be bigger for f.r.e.e - that they can actually embed their spam offers inside real messages in such a way as to be completely undetectable by filters.

Re:"Postini"?

[Jay L]

Google is the only mail service that I know of who still just won't accept my emails.

I had a similar experience; I run my own mail server, send no bulk mail whatsoever, and both Postini and GMail independently decided I was a spammer. No DNSBLs had me listed, ReturnPath was happy, etc. Meanwhile, I was blocked from sending mail to my lawyer, my financial advisor, my chiropractor, etc., all of whom turned out to be downstream from Google. Despite Google's claims that the customer is in full control of filtering, none of them were able to get at my e-mail without getting their sysadmins involved - which often required discovering that they had sysadmins at all.

Worse, Postini's spam filtering takes its own output as input. Once it's scored a message of yours as spam, future messages will be more likely to score as spam - which of course makes any subsequent messages even more likely to score as spam. Brilliant. At one point, my spam score from a triple-signed (SPF/DK/DKIM) server was 98 out of a possible 100.

Google's philosophy of "we don't do it unless we can automate it" works horribly when it comes to customer service. There's no feedback loop, no whitelisting, no channels, no nothing. It's SPEWS all over again, or perhaps the Kafka International Airport.

But Google has no reason to worry about false positives; the more messages they call spam, the more spam they can say they blocked. Perverse incentives.