Slashdot Mirror
Researchers Claim "Effectively Perfect" Spam Blocking Discovery
A team of computer scientists from the International Computer Science Institute in Berkeley, CA are claiming to have found an "effectively perfect" method for blocking spam. The new system deciphers the templates a botnet is using to create spam and then teaches filters what to look for. "The system ... works by exploiting a trick that spammers use to defeat email filters. As spam is churned out, subtle changes are typically incorporated into the messages to confound spam filters. Each message is generated from a template that specifies the message content and how it should be varied. The team reasoned that analyzing such messages could reveal the template that created them. And since the spam template describes the entire range of the emails a bot will send, possessing it might provide a watertight method of blocking spam from that bot."
As a co-author of this work, I should be clear that we never suggested that we have a perfect spam filter per se, simply a new tool that has the benefit of being orthogonal to existing techniques. For _existing_ botnets, our filters are extremely good, but the paper is also quite clear about the variety of ways that spammers might try to evade the approach.
Spam filtering isn't very hard, if you see the email for a large number of accounts, as Gmail does. The one characteristic that spam must have is that it's sent in bulk. The commonality across receiving email accounts gives it away. The only hard part is recognizing the commonality, which is already working rather well. This is just a new technique for recognizing commonality.
Recognizing spam for a single account is tougher, because you don't get to see the "bulk" property.
I originally posted it here in 2002. Note how dated it is (e.g. no smartass comment about CAPTCHA).
Some mathematician (I forget who) had his graduate students send back cards with forms like these to people who sent in attempted proofs of Fermat's Last Theorem.