A New Type Of Realtime Blocklist: The SURBL

Glamdrlng writes "The SURBL, or "Spam URI Realtime Blocklist", represents a nexus of RBL's and content filtering that may bring us one step closer to a spam magic bullet. While traditional RBL's perform a DNS lookup on the connecting mail server, SURBL's take this a step further by parsing the text of the email looking for URI's and doing a lookup on those web servers. They also prevent "joe jobs" by maintaining a whitelist of legitimate web servers whose domain names may show up in spam messages, e.g. EBay, Paypal, Microsoft, etc. The only requirement to implement the SURBL is a plugin on your MTA such as spamassassin that can parse the body of each email. While there is no MTA that directly supports SURBL's without a plugin, the author hints at one being in development."

Comments

Re:Is this really a GOOD idea?

[beh]

(one minor thing I missed before:

The advent of bayesian spamming brought spams that included whole paragraphs of random words - just so that your list would get more and more bloated...

How long do you think it will take spammers to add dozens of valid - but in the context of the spam nonsensical - URLs just to fill up the black-list and make it useless?

Re:It's a great idea

[beh]

...unless I would send out a spam with TONS of valid links on various sites that haven't got anything to do with the rest of the spam...

Boy - that list will be f***ed up pretty soon...

Re:Time to dig out this old post.

[interiot]

• (x) Users of email will not put up with it

We'll see.

• (x) Eternal arms race involved in all filtering approaches

One of the few constants is that there will be way for money to get from the target back to the original spammer or seller. (well, it's possible something more complex is going on and that's not the real goal of spam, but at the least, it's something that's remained constant for years, which is notable in the world of spam). So "following the money" is really based on an acceptance of the above criticism, and a realization that the arms race can never get around the money stream.

Filters may be lead to arms races, but does anyone NOT use them right now? There are few alternatives, namely things like making email non-anonymous / PKI, enacting large legal penalties along with huge international support, rejecting email from anyone you don't know, ....

• (x) Whitelists suck

Actually, it's a blacklist. Blacklists may suck, but it's possible they suck less than spam, and the proliferation of RBLs kind of implies that.

Sure, there might be a way to stop spam once and for all and then blacklists would be hated, but the very presence of a antispam-rejection-template implies that there won't be a magic bullet for a long time to come.

• (x) Sorry dude, but I don't think it would work.

The only way it CAN'T work is if money isn't the real goal of spammers, or if they make it hard enough to "follow the money" that other methods are easier/nicer.

Re:Spam is unavoidable

[rw2]

We can't ever have a workable spam filter because of the adaptability of spam.

This is because the solutions of the day focus on content instead of anonymity.

I've said it before, I'll probably say it again, get rid of unauthenticated email and the spam problem becomes a thousand times easier to fight. SPF and various RMX solutions exist in design today. If people want the spam problem to go away, that can be done today. Unfortunately people would rather piss and moan and call for legislation or perfect solutions than deal with these good ones today.

In the case of spam the perfect is the enemy of the good enough. We should stop spam today.

Re:Present problem.

[Phroggy]

Presently the only problem with this is that there are no plug-ins for the MTAs themselves yet. The plug-in is for spamassassin. That means that the message has to be transfered and passed onto Spamassassin before it can be dropped or tagged whereas, the other RBLs allow you to drop the connection before the message is transfered. This problem will be solved once there are plug-ins for the MTAs themselves.

Sorry, but that's not because it's a SpamAssassin plugin vs an MTA plugin. That's because the SMTP protocol doesn't allow for what you describe.

Let's say I'm an MTA. When you connect to me, the first thing you do is introduce yourself, then tell me the envelope sender and envelope recipient of the message you're about to send, then give me the full message including headers and body. My options for blocking the message are:

1. Before you even connect, your IP could be blocked at the firewall level, so I'd never see you.
   
2. After you connect, before you introduce yourself, I have your IP address, and can check it against a blacklist and/or whitelist, and give you an error and disconnect if I don't like what I find. I can also do reverse and forward DNS queries on your IP to make sure they agree.
   
3. After you introduce yourself, I can compare your greeting against your reverse DNS, since that's how you should be introducing yourself. I can give you an error if I don't like it.
   
4. After you give me the envelope recipient, I can make sure that domain exists, etc. (Side note: Verisign wants to break this; ICANN is currently not letting them.)
   
5. After you give me the envelope recipient, I can make sure that e-mail address is OK - if it's my domain name and the username is somebody I know I'll accept it, or if it's a valid domain name somewhere else and your IP is on my LAN I'll relay it. Otherwise I can give you an error.
   
6. If we've gotten this far, I must now accept the entire message, including headers and body. If there's something in the headers I don't like, too bad! If there's something in the body I don't like, too bad! I have to let you send the whole message.
   
7. After I've accepted the message, if there's a problem, I can generate a bounce message to send back to you, assuming the e-mail address you gave me actually works. If that fails, I'll send an e-mail to my postmaster explaining what happened. Or if that's too annoying, I could just delete your message and not tell anyone.
   

Existing RBLs work at step 2. Filtering based on message content can't happen until step 7. You could build it into the MTA, but MTAs are complex enough as it is; using something else (SpamAssassin, Procmail, whatever) is a better idea.

Could be good, could be bad.

[autopr0n]

I see one major problem with this, which is that Spammers might now be able to cause problems for legitimate websites simply by including their URL in the a Spam.

I'm a little sensitive to this since a spammer is actually Jo-jobbing one of my domains (not autopr0n), and I get hundreds of "user unknown" messages every day, along with a handful of messages telling me "my" email was blocked. It's really irritating.

But, if it's done right, it could work out pretty well. In fact, this would actually be effective against a lot of the current Spam out there, and kill Spam with off-site images.

Anyway, let me throw one countermeasure out there. Suppose spammers start including commonly mailed URLs (such as those on hotornot, yahoo, etc) in their spams in order to decrease the usefulness of these things. If this thing gets popular, expect to see a lot of Spam include a lot of random URLs the way they now include lots of random words. You'll also start to see things like "Javascript decryption" and other techniques to prevent machines from figuring out which, exactly, URL it is that is being advertised, rather then random noise.