The Spam Problem: Moving Beyond RBLs

whirlycott writes "I just published a paper called The Spam Problem: Moving Beyond RBLs on my site. I comprehensively describe RBLs and list eight specific problems with them. I also get into ideas that next generation antispam system creators should read. I hope that this will be useful to anybody who is attending the Spam Conference at MIT on Jan 17th."

Comments

Preemptive methods

[LunarOne]

Simple, preemptive methods of fighting spam are often the best:

1. Don't let a spammer verify your email address
2. Don't post your email address on the internet
3. Secure your email client
4. Avoid common email traps
5. Fight back

Let me know if these can be improved.

Re:Preemptive methods

[DeadSea]

You have no control of your email address. I only give my address that I use for personal correspondence to my family and closest friends. My father gave me a DVD rental for my Birthday, and on of my friends invited me to a party and used one of those web sites that do invitations. Between the two leaks, my address is now in the hands of spammers and I am getting 2 to 3 a day at that address. Short of beating my friends and family around the head, I don't think I can stop that sort of thing.

Not posting your address is important. If you you post your address on the internet, expect more than 10 spam a day. Similarly if you use it to post on usenet, expect more than that. It seems to be hardly sufficient, however.

I have decided that my only recourse is to change my address every time it starts getting spam. People that email me at an old address get a note saying why the address has been disabled and a url on my website where they can fill out a form to contact me. (btw, if you are interested, you can get the contact form that I use on my website, it is designed to thwart spammers, unlike formmail and other cgi to email gateways.)

Re:Preemptive methods

[artemis67]

I can't read your links because of a good slashdotting, but from what I see, your arguments are flawed.

1. Don't let a spammer verify your email address

This isn't a huge problem for spammers. If they send you an HTML email, then just opening the email (or previewing it in Outlook) can provide the verification that they need.

Additionally, the extremely low cost of spamming means that bogus addresses are a marginal problem at best. The spammer would rather take a chance that the email account is active and send the spam than not send it.

2. Don't post your email address on the internet

I learned this lesson too late. A Google search pulled up a dozen newsgroup messages with my email address in them. Nine were posted by me, and I asked Google to remove them. Unfortunately, 3 are by other people quoting me, and I have no recourse to remove them. Spammers will therefore have permanent access to my main email address.

Additionally, I have no control over emails that other people send that include my address. I hate "pass along" emails that certain people get and feel the need to send to everyone in their address book, but I can't help that a) my email address is included in a batch of 50 others, and b) it's a very convenient way for spammers to collect verified email addresses.

3. Secure your email client

By this I assume you mean using client-level filtering. I do. Alot. I typically get about 60-80 pieces of spam a day, and have set up 30 or so filters. But that only catches about 2/3's.

Simply put, there is no client-level filtering solution that is going to work 100% of the time.

4. Avoid common email traps

I assume here that you mean things like "posting to newsgroups". You can only avoid traps that you already know about, and most people don't know about them.

Besides, why should we live in fear of the spammers? They are encroaching on our free expression. I certainly think that the structure of email needs to be revisited to put the prohibitions on the spammers, not the recipients.

Bollocks!

[odaiwai]

Having briefly looked at the paper, it seems like the usual complaining about RBLs as being too broad you see all the time in NANAE (news:news.admin.net-abuse.email).

Summary: someone tries to send email and finds that they're listed on SPEWS. They complain because "we're not an open relay", without figuring out just why they're on that list. Almost invariably, they're on the list because their ISP persistently ignores spam complaints and prefers spammer money to honest customer money. I think there's been about two or three actual mistakes in the SPEWS listings in the year or so I've been following NANAE. Otherwise, it's all been a legitimate extension of the block because the ISP knowingly ignores complaints and supports spammers.

Spam is theft. Theft of Bandwidth, theft of service and theft of time. It's that simple. Spammers are thieves. ISPs which support spammers are thieves. Soon, they'll be blocked from the public internet for anti-social behaviour. After all, if your local bargain supermarket ignored the thieves stealing 20% from every transaction you make with them, will you go back?

Many South American and Asian ISPs are blacklisted because they were quite happy to spam everyone when they could steal bandwidth and service from other ISPs. Now that they're blacklisted, they're whinging and moaning about 'freadom of speach', interference with interstate commerce, and other such bullshit.

It's about none of these things. Blacklists are about protecting your network from a Denial of Service attack by spammers.

People who complaing about RBLs (OR DNSBLs, to be more accurate) are missing the point. They should be complaining about spammers who think it's acceptable to steal my bandwidth and your bandwidth to advertise their product..

dave "the only good spammer is a rotting corpse, dangling from the noose"

Re:Easiest way to stop spam...

[Zeinfeld]

Tell EVERYONE you know never to click on any spam links, or buy spamvertised products. People spam because it WORKS. The only real way to stop it is to STOP BUYING SPAMMED PRODUCTS.

The problem is that you are in a global network. It is like the problem of eating whale meat, you can persuade 99.999% of the world population that eating whale meat is a bad idea but the other 0.0001% that is left can eat the endangered species to extinction within a matter of months.

It only takes a vanishingly small number of businesses out there to SPAM and you have a massive problem.

SPAM does not have to even be profitable for people to do it. If I wanted to launder a lot of drug cash I would set up a spam house and bombard people with ads for herbal viagra..

There was a time not so long ago when the majority of the SPAM being sent out was adverts for spam software. SPAM does not have to work as a marketing method for creeps to get rich charging others to spam. The pitch line they use to haul in suckers is 'it must work or why would people do it', well no, it does not have to get one single end customer for it to work for the spammer.

The two problems (which impact more than e-mail)

[Lumpish Scholar]

(1) You (and I) get too much spam.

(2) Your e-mail system administrator (and mine) need to keep beefing up the servers because the sheer volume of e-mail is growing so quickly.

To a first approximations, filters solve (1) but not (2), and black hole lists solve (2).

whirlycott summarizes the problem with (2) in two words: "collateral damage." How much of the e-mail network do we need to destroy in order to save it?

We need to move past first approximations. We need systems that work at the server level, but that somehow address the problems of collateral damage and false positives.

This is only the tip of the iceberg. Any network messaging medium is vulnerable to abuse by spammers. The problem started with Netnews, it continued with e-mail, it's happening now with instant messaging. We need at least high level solution that helps solve the problem regardless of prototcol.

I wish I had one.

Moving Beyond SMTP is the Answer

[zentec]

The problem, as I've said here before, is SMTP itself.

The RFC pretty much states that to be compliant, you have to accept the mail as it is presented. Can't achieve accurate or trusted reverse name lookup information on the sending system? Well, that's tough, take the mail (read this for yourself).

This problem stems from when systems on the Internet were inherrently trusted. That's not the case any longer, and it's time for a new mail transmission standard.

For starters, it should allow system administrators the ability to give priority to systems that can present some form of credentials. SSL or keyed encryption, whatever the standard is, it will permit systems to give totally trusted access to systems that meet the specific security and trust guidelines of the receiving system, not the RFC (times have changed, tough).

Those systems that do not meet minimum trust levels will either have to clean up their act or take the time to contact the remote system to figure out the issue.

It won't stop spam, but it will go a long way to slowing it down and possibly providing some secure method of mail transport in the process.

If he's annoyed, then it's working.

[?erosion]

Isn't this how a blacklist is supposed to work? I thought the idea was precisely to annoy the honest users, such that they complain to the ISP. If the users know that they are blacklisted because of a spammer, they are likely to either leave the ISP or pressure it to turn the spammer off. It's not nice, but the intent is to get results.

Re:Whiner...

[minas-beede]

In your case it worked out. If you had simply been asked to persuade your ISP to boot the spammer would you have ignored the request? Are you actually so dense that it takes blocking your email to get you to act?

Note that I'm not trying to claim you are dense or prove it - my point is that you could have been reached in a way that led to the same result but that DID NOT block your valid email. Is there any reason why the brutal method should be the one chosen first? Uh, any good reason - surely there are thugs who enjoy using their power to abuse others.

Not to mention that there's been more than one case in NANAE where the collateral damage was suffered by someone related to an ISP that had long ago booted the spammer but had not removed all traces. No spam flowed because of the omission, the listing was long after the spammer was removed, no risk to anyone existed. Still, the IP of an innocent party was wrongly listed, wrongly blocked, much time and energy was spent discussing it in NANAE, a person and organization that could perhaps have become spam opponents were given reason to hate the guts of spam fighters. No win of any kind I can see in that.

And, of course, the brutal blocking actions haven't ended spam, other than the occasional anecdotal victory. I ran an open relay honeypot, I saw how modern bulk spammers operate. The DNSBLs are a weak tool to deal with that. Don't take my word for it: run your own open relay honeypot. You'll quickly learn a lot about how spammers operate. All the while you'll be stopping their spam, too. Open proxy honeypot? Bless you - you'll also do wonders.

(Any of you sendmail experts able to figure out my pseudonym?)

Re:Whiner...

[melonman]

RBL's are like a fever. They tell you when something it wrong and only a dork blames the fever when the problem is the disease.

It's not like any fever I've come across. For the analogy to hold, when I'm ill my entire village would get a fever, and some of the population might die, in the hope that the sound of the ambulances and funerals might alert me to the fact that I have a problem.

I'm glad you are so happy about having your reputation threatened when you have done nothing wrong. Our business is hosting websites on our own machines in a server park. Server parks are always going to be a good place for spammers to rent cheap machines, and if our clients start getting their mails bounced, they don't write to the server park owners, they cancel their contracts with us. And, no, we can't just take our servers elsewhere at 3 minutes' notice, so the RBL puts zero economic pressure on our server park (which seems to act fairly promptly on abuse compaints anyway).

RBLs punish the innocent to get at the guilty. This is wrong. The next time my business is hit by SPEWS or any other such system, I'm going to start writing pithy articles for the general press, with the aim of scaring customers away from ISPs that use RBLs, eg "Do you want your ISP to tell you what email you can read?. And I shall certainly take legal advice on whether I can sue companies who bounce my mail with any rejection message containing the word 'spam' for libel or something similar.

Re:Whiner...

[melonman]

Ok hotshot, I've just added cyberporte.co.uk to our local RBL list and taken the liberty of posting a link (with a C&C warning) to your post on NANAE. Would you like the address of our attorney now....

This is great, you've just demonstrated that RBLs are not neutral, and are driven more by a desire to punish than to solve the problem. If I ever need to send an email from that domain, I'll use one of our other smtp servers, or that of one of my ISPs, or rent a clean one, or... the problem last time was that I didn't know how ineffective RBLs are. The one thing I'm not going to do is change my server park because someone on the other side of the world is on a quixotic crusade. It's not my battle, and I object to people trying to enlist me.

Why your netblock or address range has been rejected.

In our case, it is because one machine in our 16-bit IP range had been used for spam, so SPEWS blocked 65,000 machines, each of which is administered by a different person/company. How does jeopardising the existence of my company, whose smtp server is clean, help to fight against spam? Like I said, we can't just pick up a fairly full server and take it somewhere else, so there is no real economic pressure on the server park.

Joe Internet user is tired of spam

See n previous /. discussions about this, but the (statistically) average email address gets about 3 a day. Quite a lot of /.ers say they get very few spams, and many of those who do say that the annoyance value is pretty low. On the other hand, if you are trying to buy a skyscraper (real example) and you can't get emails from the estate agent, who happens to be in a different continent, that is extremely annoying, especially if there is absolutely no reason for blocking that particular server.

Any decent way to block spam

Err, if netblock is such a greeeeat system, how come spam is increasing? Am I missing something? If there is a consensus that spam is a major problem, legislate against it. I don't have a problem with that. I do have a problem with what mrneutron calls 'collateral damage', ie people damaging my reputation to get at someone else, especially when the system obviously isn't reducing the amount of spam sent globally.

Re:Incomplete!

[Frater 219]

He's pointing out that current blacklisting systems are stupid. He's pointing out that the people who run the blacklisting systems are generally unaccountable (most lists are secret), that they do impose arbitrary blacklist entries against groups they disagree with, well outside of their advertised remits (such as MAPS blocking an ISP that had a handful of customers that sell spamming software), that ordinary bystanders are frequently the victims of over zealous blocking and that, per se, anyone relying on a third-party RBL based solution is making a huge mistake.

But, you see, those things he's "pointing out" are wrong. They just aren't so. They aren't the way the world works, and they aren't the way DNSBLs work.

• DNSBLs are not secret or unaccountable. They can't be! They are accountable to those who use them (mail server operators), who are respectively accountable to their users. Individual DNSBLs have force solely because sites use them; a DNSBL nobody uses is a no-op. I use certain DNSBLs because I trust them to accurately do what they say they will. If a DNSBL that I use starts going haywire and listing things that it said it would not, then nobody will continue to use it -- and it will therefore be without force in the world. (Incidentally, anonymity or pseudonymity does not equal unaccountability -- but if you don't know that, get the fuck off the Internet, since we fought that one almost a decade ago, and St. Julf of Penet was right.)
• MAPS screwed up, and was held accountable for it. That is why nobody who is serious about spam-fighting takes MAPS seriously any more. They fucked up, they fucked up bad -- and so today they are naught but a minor player. SPEWS, SBL, and ORDB are the big players in the world of DNSBLs, because they do what they say they will do, and they don't fuck around. (Note: That they do what they say they will do doesn't mean they do what you want them to do. You don't get to decide that except for your own mail server.)
• There is no "overzealous blocking" problem. There just isn't. If you are thinking about SPEWS, keep in mind that sites which use SPEWS know what it does and want it to be doing that -- otherwise, they would quit using it. SPEWS doesn't force itself upon unwitting mail servers -- rather, operators have turned to it because it works, it works well, and because they and their users are sick and tired of putting up with ISPs which don't boot off their spammers. It isn't "overzealous" -- it is doing precisely what we want.
• Using DNSBLs isn't a "huge mistake"; it's effective collaboration. Right now, DNSBLs represent the best means for sites to share information with one another about which IP addresses emit spam, or are open proxies, or belong to spam supporters. They are used not only by mail server operators, but also by IRC operators tired of proxy-borne abuse. They are effective -- and if they were not effective nobody would use them. If a better means comes along to do what DNSBLs do, then we will happily use it -- but it ain't here yet.

It is not mail users who want us to consider DNSBLs passe' or something to "move beyond". It is spammers who want us to give up our current most effective tool for collaborating to impede their crimes.