Choosing a Good DNSBL

stry_cat submitted a story about selecting a good DNSBL. It talks about some of the problems with DNS blacklists and the sorts of things that you should be looking for. Things like Speed, Selection Criteria, and Goals make the list. And of course not requiring payment to be removed from the blacklist.

Comments

Al Iverson is your FRIEND.

[seebs]

http://stats.dnsbl.com/

Or, for commentary:

http://www.dnsbl.com/

Absolutely the best resource on the topic.

These work well for me

[SCHecklerX]

@rbl = relay_is_blacklisted_multi_list($ip, 8, 0,
                [
                'zen.spamhaus.org',
                'combined-HIB.dnsiplists.completewhois.com',
                'list.dsbl.org'
                ]
        );

I reject on these in mimedefang's filter_sender routine, since they provide straightforward methods for removal. For other lists, spamassassin will raise score accordingly, and will raise score based on any blacklisted stuff in the headers (not just the server handing off to you) which is nice.

Greylisting kills a lot of stuff too.

DNSBL for comment spammers?

[_xeno_]

This seems like as good a place to ask as any. Can mostly email-based DNSBLs be used to try and block comment spammers? I'd love to reduce the load I get from comment spammers trying to spam my website.

I've been contemplating using an existing DNSBL, but all the well-known ones are focused on email spam. I expect that comment spambots and email spambots mostly overlap, but I'm not sure how effective such a measure would be.

Re:DNSBL for comment spammers?

[wytcld]

Had a bunch of robot spam going through a home-grown PHP comment form - all of it from Russia. So I got the the Russia CIDR list from here and added this:

$testip = $_SERVER['REMOTE_ADDR'];
function ipCheck ($IP, $CIDR) {
  list ($net, $mask) = split ("/", $CIDR);
  $ip_net = ip2long ($net);
  $ip_mask = ~((1 << (32 - $mask)) - 1);
  $ip_ip = ip2long ($IP);
  $ip_ip_net = $ip_ip & $ip_mask;
  return ($ip_ip_net == $ip_net);
}
$CIDRs = file ("/path/to/ru.zone.file");
foreach ($CIDRs as $CIDR) {
  if (ipCheck ($testip, $CIDR)) {
    $act = "view"; // switches to viewing old comments rather than posting new one
    break;
  }
}

It's fast, and when comment spam shows up from other countries I don't care about, I'll block them too.

Requiring payment for delisting

[dbolger]

I used to work in the abuse department of an ISP which had been blacklisted by SORBS. SORBS require a "donation" to get your IP range off their list, and since we refused to hand over extortion money to these gangsters, there was no way for us to deal with them. Despite our best efforts, we also found that there was no way to get in contact with them, and as such no way to help our customers.

Doing a Google search for information about this lot brought up so many horror stories that I can't fathom how so many people ended up using their "service". It got to the stage where if we had a customer having trouble with SORBS blocking their mail, the only advice we could give was to contact their recipient via other means and ask them to stop using these thugs to filter mail.

Re:Requiring payment for delisting

[CopaceticOpus]

Amen! I have run up against SORBS blocking as well, and we refused to pay them. Unfortunately, their blacklisting service is used by a major U.S. supplier of email addresses. (I can't remember which one at the moment.)

Just say NO to SORBS!

Re:Requiring payment for delisting

[ciscoguy01]

SORBS require a "donation" to get your IP range off their list, and since we refused to hand over extortion money to these gangsters, there was no way for us to deal with them.
Which stinks to high heaven. I wish Matthew Sullivan wouldn't do that.

There are many reasons someone who is not an actual wrongdoer could become listed as a spam source. I have little doubt the parent's organization was such a spam source and did not properly address the issue. They deserved it.
It's not what problems you have, it's how you handle those problems is what matters.
As long as a site addresses the spam problem and gets results, reads their abuse mail and acts like a good net neighbor I have no problems with them. They should be delisted as soon as possible.
There have been times when certain cable modem operators were the major source of spam in the world and they essentially ignored abuse mail. They should have been disciplined until they clean up their act. Anyone who is not addressing the problem promptly deserves to be blackholed until they solve their problems.
There are plenty of clueless sysadmins in the world, people who are in over their head, or dominated by the company sales department so they cannot disable a circuit with deliberate spammers on it.
That's what DNSBLs are supposed to work to change.

Re:Requiring payment for delisting

[Akatosh]

I guess whatever provider that was stopped, because I havn't heard a thing out of my users about Sorbs for a long time. They're irrelevant now, moreso since Sorbs shut their spam list down a few weeks ago after the founder had a breakdown. Did anyone even notice? That's how irrelevant they are.

Re:Requiring payment for delisting

[Zedrick]

I have little doubt the parent's organization was such a spam source and did not properly address the issue. They deserved it.

And what are you basing this belief on?

As long as a site addresses the spam problem and gets results, reads their abuse mail and acts like a good net neighbor I have no problems with them. They should be delisted as soon as possible.

Right. I work for a big webhost, which is blacklisted by SORBS from time to time. The problem is that they do not send abuse reports. (I handle abuse@mycompany and I do not miss or ignore one any mails). They blacklist, and expect you to pay. ...Which makes me think they're interested in the money, not preventing spam.

Contrast that to, for example, Spamcop who sends mails that clearly states what it's about, a copy of the mail headers and a nice link where you can let them know what's been done (such as shutting down the spammers account).

Re:Requiring payment for delisting

[ciscoguy01]

If the true goal is to go after the spammers, how does a DNSBL help this?

ISPs have customers, customers who want their mail to go through. Customers like you. If an ISP has lax abuse policies (or no abuse policies, or is a willing spam host) and you are a legitimate customer of that ISP, your mail may be blocked with the other legitimate customers of the ISP.
You are not being listed, your ISP is.

The DNSBL hopes you will call your ISP, and as a valuable customer demand they cure their spam problem so you will be able to send mail.

If an ISP's customer is spamming me all I can do is complain, and they can ignore me. You are their customer, you are influential and you want your mail to go through, so you are completely within your rights to demand they get rid of their spammers that are causing you problems. Your ISP can make a choice, either deal with spammers and all their legitimate customers go elsewhere or sue them, or get rid of the spammers and have you, legitimate customers.

It makes perfect sense, doesn't it?

If we ever get blacklisted by SORBS or any other extortionist and they ask for money, we'll probably sue and/or file a criminal complaint.
Criminal complaint? Nobody has to accept your email!
If you are a spammer that's what you might do, which is why most of the DNSBLs are in countries other than the US where they are protected by the local laws from lawsuits like that.
What you should do is sue your ISP for getting you listed along with them, or demand they cure their spam problem.
Unless it's you that are the spammers, that is.

Local Whitelisting!

[HitekHobo]

Choosing a good DNSBL (or three!) is definitely important, but IMHO, you should NEVER run DNSBL's without building a local override into the system. We run our own DNSWL (dns whitelist) which is consulted before hitting on BLs... if a customer has had problems with one of their contacts being blacklisted, we can selectively add their IP to the list.

Unrelated to the above, I would also recommend looking at ironport systems if this is a commercial project with a decent sized budget. (I am not affiliated, just a happy customer).

There is no such thing as a good DNSBL

[deviator]

They all have issues; all of them create headaches for administrators of legitimate e-mail servers at one time or another.

Re:There is no such thing as a good DNSBL

[seebs]

Of course they do. That doesn't mean they're not good; it means they're not perfect.

The fact is, without DNSBLs, the headaches would be worse. LOTS worse. Centralized blocking gives you some kind of theoretical hope of getting unblocked once you've fixed the problem. Decentralized blocking leaves you no chance at all. Furthermore, without tools like DNSBLs, administrators would be far too busy to even get to the point where they could have these headaches.

I'd rather live in a world with a number of reasonably good DNSBLs than not have any.

NEVER use a DNSBL as an absolute block

[ebunga]

DNSBLs are subject to the whims of some of the most unreliable and whiny schmucks on the face of the planet. NEVER under any circumstances use a single DNSBL as an absolute block. Use it to increment a score along the lines of Spam Assassin that will eventually hit a threshold, preferably with a minimal content-based component. Don't even think about using multiple hits on multiple lists as a gauge of spam-worthiness. The amount of inbreeding and sharing among lists is disgustingly high. Not even the Spamhaus aggregate is trustworthy these days.

Spammers can get around blacklists anyways. They're about as effective as locking a door made of tissue paper. The number of false positives is high. The amount of spam blocked is negligible. My suggestion is to abandon the idea altogether.

Re:NEVER use a DNSBL as an absolute block

[Shaman]

Sounds good, except it's not true. I was just on one of our spam systems (Barracuda 400) and the stats look something like this:

20,000,000 blocked e-mails
480,000 tagged e-mails
90,000 viruses found
135,000 quarantined messages (user choice to quarantine or not)
610,000 delivered/approved mail

To nobody's surprise, some spam is still getting through. This is in less than two weeks, and there are two servers to handle the load, the other one is more or less as bad.

So what were you saying about not using blacklists?

Re:NEVER use a DNSBL as an absolute block

[ion]

how many of your 20,000,000 blocked emails are false positives? aka legit email.

I would so much agree that using a DNSBL as a absolute block is a bad idea. I have experienced being caught up in them, and that is annoying. Even if the mailserver is removed some days later. Later is not soon enough, i want my email to arrive now.

I would much rather suggest running some sort of spamassassin while the SMTP connection is still open, and if it looks like spam i would reject it. This can be parallized if needed.

I would also consider to reject any email that came with an attachment if you have not already received legit email from the same address. This tries to use that spammers seldom send from the same email address and that they started sending attachments. Legit email does not usually start with an attachment in the first email (at least mine does not). So, if you previously received emails and that email address has a negative spam score, aka not being a spammer, then i would accept attachments, else i would not.

This might be pr. domain, but hotmail and others are often used by spammers. This could lead to a domain spamscore, aka if you received emails before from this domain and none was spam, then accept attachments, even if it is the first time someone sends you something from the domain. This is for company uses where John sends you something and then later Jane sends you something with an attachment.

You might want to allow certain kinds of attachments even if they are not listed before. These attachments could be .vcf files, and possible .html, but not .pdf or .jpg

This has a score of two?!

[Anonymous Coward]

Greetings, sir,

Allow me to introduce myself. I'm a representative of the Consortium of Common Sense. I've noticed you recently posted to an Internet-based conversation, complaining about the reduction of a nine-letter word to two letters via acronym. Your post referenced such things as numbers of syllables.

Please look at your desk now, and slam your head down as hard as you can on it. Do you feel those weird little indentations in your forehead?

THEY'RE CALLED KEYS - DID YOU NOT REALIZE THAT THINGS ARE TYPED, NOT SPOKEN, ON THE INTERNET?

Thank you. Please let us know if you have any other ridiculous complaints.

- Consortium for Common Sense

NEVER use a DNSBL as an absolute block...

[HitekHobo]

...unless you have to.

There is a lot of truth to the OP's statements. However, unless you have the budget for a commercial spam filtering application, there are not a lot of good solutions.

Spamassassin is great for what it does, but in high volume environments, you will be throwing so much hardware, bandwidth and electricity at the problem that you'll either give up on filtering at all or break down and buy a commercial solution.

DNSBL's give you a bit of breathing room between the two extremes. Our environment has about a 98% spam catch rate currently with commercial solutions. We have about 150 connections per second AVERAGE.

Our infrastructure could just barely keep up with this load when we were using DNSBL's only. Had we tried to use a spamassassin style tool, we'd have needed quite a bit more infrastructure to handle all of the increased filtering. DNS lookups are pretty cheap compared to the amount of CPU required for context / content filtering.

DNSBL's definitely generate too many false positives, but when the alternative is buying 10x the hardware or having mail take 1-2 hours to be delivered during peak times, I'll take the false positives.

Re:Dynamic IPs / Zombies

[Jeffrey Baker]

Except the blacklists which are supposedly dynamic IPs contain tons of other shit. There is one which contains any IP which reverses to a name containing the letters "dsl". This is pretty stupid since a lot of business DSL lines have static IPs and because Speakeasy business T1 lines also reverse to whatever.city.dsl.speakeasy.net. Other ISPs have the same scheme, and they don't all delegate reverse DNS. I have a business MX hosted on a T1 line that's blocked by some blacklist that Earthlink uses. So I can't send mail from that business to anyone at Earthlink. It's a really stupid policy.

No Blocking

[rawg]

I have found that my customers don't want me to block spam. I would get complaints from customers not getting their mail from hosts that are being blocked. So I use Spamassassin to tag the spam and filters on my clients to delete it. Yep, I have to process all that spam and yep the customers have to download all that spam, but I don't get any phone calls anymore.

Re:No Blocking

[The Cisco Kid]

And doing so is entirely your choice, and no one other than your boss (unless you are the boss) has any business telling you to do otherwise.

I am curious though, if you (or your boss) are happy with the loss of profits involved due to increased bandwidth and server resource costs that go with that choice (Or, if you've raised your prices to offset that, if your customers are happy with that).

DNSBLs to feed other tools

[billstewart]

Most DNSBLs have problems, and there are few that I'd trust absolutely, though Spamhaus runs a tight enough shop that I'd trust it. But DNSBLs can be used effectively to augment other tools:

• SpamAssassin weights - most of the DNSBLs are worth a couple of points of SpamAssassin weight; even rabid ones like SORBS can give you some information, and the country-specific ones are also useful here (e.g. mail from China had better not look spammy at all.)
  
• Greylist Augmentation - The big value of DNSBLs is that you can reject mail from the SMTP headers without needing to receive the message body and grind it through CPU-instensive content filtering. But Greylists also do this, and some people have been using DNSBLs to tune their greylists (e.g. if it's on the DNSBL, then tell the sender to call back in an hour instead of 5 minutes.) Among other things, that gives you a way to use the lists of Dynamic-Address broadband users - the home Linux servers will call you back, the zombies won't, so the list gives you information which you might otherwise have to ignore. And country-code DNSBLs can also get forced to wait an extra hour for spammy places that you don't get much mail from.
  
• 
  TMDA Autoresponders - One of the most annoying and effective anti-spam tools is autoresponders that say "I don't recognize your address - respond to this mail and prove you're a human". You could integrate this with a DNSBL - if the mail's not whitelisted, and it's on some DNSBLs, then maybe it gets a TMDA test instead of bit-bucket. It's lower CPU than SpamAssassin.
  
• DNSBL integration with DNS Servers? - One of my pet projects for when I get some copious spare time is to munge a DNS server to check blacklists/whitelists. Trusted or non-blacklisted sites get the MX record for the good mailserver, non-blacklisted sites get the MX record for the heavily-filtered mailserver that occasionally overloads the CPU, blacklisted sites get the MX record for the teergrube or 127.0.0.1. It's certainly not foolproof - many systems are likely to check their ISP's DNS cache before hitting your DNS directly, and if spammers want to do a set of DNS queries from a clean server they could - but at least at the corporate-email level (i.e. where you can afford multiple mail servers) it gives you a way to avoid having your mail server lose mail from legitimate sources because it's overloaded with SpamAssassin CPU load.
  

  
  I originally thought of this back when Open Relays were the popular spam threat - if you get a DNS MX request from an open relay, tell them that the IP address for spambait.yourdomain.com is some other open relay's address. That would let them spend their time sending mail to each other. But spammers moved on to open proxies and then zombies, so that opportunity went away.
  

• You can think of other things.
  

Re:Project Honey Pot's Http:BL

[porneL]

Project Honeypot's http:BL isn't handling dynamic IPs in any special way, so you have to be careful about these (combine with SORBS DUL and take into account age/threat that http:BL reports).

To truly make blacklists useful...

[IGnatius T Foobar]

To truly make blacklists useful, you've got to filter not only mail coming from IP addresses listed within them, but also mail containing URL's that resolve to IP addresses listed within them. Once you implement this, you will see a *dramatic* drop in spam. Spammers can move their delivery systems from place to place, but at some point they've got to advertise a web site. Yes, the stock spam will still get through, as well as some others, but over the years I've spent administering (and developing) email systems, this was the single most effective thing I've ever seen.

Happily, these tests are already present in SpamAssassin; they're just not scored highly enough. Here's a nice easy way to fix that. Edit your /etc/mail/spamassassin/local.cf and add these lines:

# High score for URL's whose IP addresses are in rbl
score URIBL_AB_SURBL 10
score URIBL_JP_SURBL 10
score URIBL_OB_SURBL 10
score URIBL_PH_SURBL 10
score URIBL_SBL 10
score URIBL_SC_SURBL 10
score URIBL_WS_SURBL 10

Restart spamd, and you will immediately see a large drop in spam.