The Spam Problem: Moving Beyond RBLs

whirlycott writes "I just published a paper called The Spam Problem: Moving Beyond RBLs on my site. I comprehensively describe RBLs and list eight specific problems with them. I also get into ideas that next generation antispam system creators should read. I hope that this will be useful to anybody who is attending the Spam Conference at MIT on Jan 17th."

Comments

EFF said it better

[Lumpish Scholar]

whirlycott's article points to the Electronic Freedom Foundation's Public Interest Position on Junk Email (Google cache), which begins:

Executive Summary: Any measure for stopping spam must ensure that all non-spam messages reach their intended recipients.

For the past several years, the Electronic Frontier Foundation (EFF) has watched with great interest the debate regarding what to do about unsolicited bulk email from strangers, or spam. We have been asked to lend our support to bills that have been introduced in Congress, and we have been approached in various other ways to help lead the fight against this annoying intrusion into people's email mailboxes.

While members of the EFF staff and board find this unsolicited email to be as annoying as everyone else, we believe that the two most popular strategies for combatting it so far--legislation and anti-spam blacklists--have failed in their fundamental design. Anti-spam bills have been badly written, are unconstitutionally overbroad, and frequently wander into areas where legislators have no expertise, such as the establishment of Internet standards. And anti-spam blacklists, such as the MAPS RBL (Mail Abuse Prevention System Realtime Blackhole List, the most popular), result in a large number of Internet service providers (ISPs) surreptitiously blocking large amounts of non-spam from innocent people. This is because they block all email from entire IP address blocks--even from entire nations. This is done with no notice to the users, who do not even know that their mail is not being delivered.

The focus of efforts to stop spam should include protecting end users and should not only consider stopping spammers at all costs. Specifically, any measure for stopping spam must ensure that all non-spam messages reach their intended recipients. Proposed solutions that do not fulfill these minimal goals are themselves a form of Internet abuse and are a direct assault on the health, growth, openness and liberty of the Internet.

Email is protected speech. There is a fundamental free speech right to be able to send and receive messages, regardless of medium. Unless that right is being abused by a particular individual, that individual must not be restricted. It is unacceptable, then, for anti-spam policies to limit legitimate rights to send or receive email. To the extent that an anti-spam proposal, whether legal or technical, results in such casualties, that proposal is unacceptable.

Re:EFF said it better

[Zeinfeld]

Executive Summary: Any measure for stopping spam must ensure that all non-spam messages reach their intended recipients.

The problem with the vast majority of psuedo-solutions to spam is that the promoters simply will not listen to any ideas other than the one they first thought of and they simply won't listen to people who point out that blocking good mail is a serious problem.

The 'cry me a river' response is as idiotic as it is arrogant. SPAM is a problem, failure to deliver email is a bigger problem.

That does not mean that we don't address the problem of SPAM, it just means that we have to approach the problem from both ends, identifying the good signal as well as eliminating the bad.

The MIT conference is likely to be a failure because the organizers are only presenting the tried and failed filtering approaches of the past. Those approaches are now well understood, they can mitigate the problem but can never do more than that. Filters suffer from reverse network effects, the more widely used they are the greater the incentive to program arround them.

Blacklists fail for many reasons, not least complete lack of accountability. As the paper reports the operator of one blacklist that claimed to only list open relays actually listed sites for other reasons. Ultimately a blacklist that does not have some robust accountability structure is simply a vigilante operation. Vigilantes are frequently popular with people who think they are victims of crime regardless of whether they create more problems than they solve.

The tools we need to start applying are digital signatures and email authentication in combination with whitelists. This follows sound business process, if you want to talk to someone well known their secretary will use a two step process, first ask who you are and check to see if you match the access criteria (e.g. to set up a cold call meeting with a Fortune 100 CEO you had better be a Fortune 500 CEO), then check to see if you really are who you claim to be.

Authentication and Authorization requires no heuristics and there is no feasible counter-strategy for the spammers.

I believe that the way to stop spam in the long term is to deploy signed email ubiquitously. Self signed certificates are sufficient for this purpose if we can provide a lightweight authentication via a DNS-linked PKI.

For example consider the problem of stopping spam to email lists. These are a prime target for spammers as the email server does most of the work. As a result most email lists are now filtered so that only subscribed readers can post. This has in turn been gamed by the spammers who use automated tools to scan the archives of an email list and send emails with forged headers purporting to come from another subscriber. Authentication and authorization prevents this mode of attack.

The counter-argument to using authentication is that the spammers can get their own credentials. If you spend some time analysing SPAM however you will find out that this is unlikely. Almost every spam has forged or obscured headers. While this does not prove that this is a requirement it is certainly indicative of the fact that the spamers do not want this type of visibility.

Even if a spammer can get a credential they are most unlikely to get a credential that would match my personal whitelist which would consist of the signing keys of the email lists I subscribe to and the domain names of the member companies of W3C and OASIS.