Choosing a Good DNSBL

stry_cat submitted a story about selecting a good DNSBL. It talks about some of the problems with DNS blacklists and the sorts of things that you should be looking for. Things like Speed, Selection Criteria, and Goals make the list. And of course not requiring payment to be removed from the blacklist.

Comments

Al Iverson is your FRIEND.

[seebs]

http://stats.dnsbl.com/

Or, for commentary:

http://www.dnsbl.com/

Absolutely the best resource on the topic.

These work well for me

[SCHecklerX]

@rbl = relay_is_blacklisted_multi_list($ip, 8, 0,
                [
                'zen.spamhaus.org',
                'combined-HIB.dnsiplists.completewhois.com',
                'list.dsbl.org'
                ]
        );

I reject on these in mimedefang's filter_sender routine, since they provide straightforward methods for removal. For other lists, spamassassin will raise score accordingly, and will raise score based on any blacklisted stuff in the headers (not just the server handing off to you) which is nice.

Greylisting kills a lot of stuff too.

Re:These work well for me

[SCHecklerX]

People on dynamic ranges should probably smart-relay through their ISP's mail server anyway. The last 2 I'ved been able to host my own mail server with didn't allow outbound SMTP forcing me to do that anyway. I eventually just spent the money on a business class line to avoid any problems.

Re:These work well for me

[koning_robot]

The thought that someone considers blocking dynamic ranges a good rule of thumb absolutely baffles me. Alas, even Hotmail seems to be doing this these days, so I've pretty much given up on e-mail.

Using my ISP's mail server is not really an option, since it will only relay for hosts within their address range, which is really nice in a world of laptops where you're in a different network almost daily. If only they used proper authentication and some kind of encryption (like _my_ _blocked_ server is set up for), the world would be a better place, for me at least.

Re:These work well for me

[dodobh]

And you want to run a outbound SMTP client off a dynamic IP exactly why? How does the recipient ISP distinguish between your host and the swarms of neighbouring botted Windows hosts?

Just pay for a smarthost instead.

Re:These work well for me

[dodobh]

Colocate a host, get a static IP, lease a box, or a virtual server ...

Re:These work well for me

[dodobh]

Smarthost through a provider hosted outside the US.

Keep in mind that recipients are under no obligation to accept your email. If I can't distinguish between your email and stuff sent by zombies at the SMTP envelope, you fall into the same category. The only trustworthy factor in that decision is the IP address of the SMTP client.

If you aren't willing to pay for the privilege of communicating with my mailfarm, feel free to send mail by registered post.

Re:These work well for me

[SCHecklerX]

So what is so hard about, uh, using your own mail server configured securely with auth and TLS...that server then forwards to your ISP's server? If you are smart enough to host your own server, and you understand the problem of your 'roaming laptop', it's not too hard to solve. Not trivial, but not incredibly difficult. Or you could just set up a VPN to your home network. There are many ways to solve this particular problem. Businesses do it every day.

I use IMAPS too so that I can easily wander around with Sylpheed on my laptop, but in a bind I could simply just tunnel everything over SSH if I had to.

Spamhaus and Spamcop

[flyingfsck]

Those are the only two that work for me (located in North America).

DNSBL for comment spammers?

[_xeno_]

This seems like as good a place to ask as any. Can mostly email-based DNSBLs be used to try and block comment spammers? I'd love to reduce the load I get from comment spammers trying to spam my website.

I've been contemplating using an existing DNSBL, but all the well-known ones are focused on email spam. I expect that comment spambots and email spambots mostly overlap, but I'm not sure how effective such a measure would be.

Re:DNSBL for comment spammers?

[multipartmixed]

Try stuffing something into your website that relies on a calculation (or other action) done in dynamically-generated javascript. Then validate that on the back end.

I have yet to have anybody add code to their spam engine to incorporate a javascript interpreter. They just move on to the next target. Even clicking a checkbox with javascript has been enough.

Re:DNSBL for comment spammers?

[_xeno_]

I've added code that is essentially my own version of the "lameness filter." This has been enough to stop almost all the spammers. It may annoy some legitimate posters, but it works, and legit posters can still post (unless they want to post about Levitra, Cialis, or Viagra). It doesn't require JavaScript, which is a plus, since as a NoScript user I would be kind of annoyed if it did. (And, yes, I'm pretty sure I never whitelisted my own domain.)

But I'd still like to block spammers before I get to the point of doing various magic like a lameness filter, preventing access altogether, so that if they ever do code to work around the filter they're still blocked from posting. It would also potentially reduce the load on the servers.

Re:DNSBL for comment spammers?

[wytcld]

Had a bunch of robot spam going through a home-grown PHP comment form - all of it from Russia. So I got the the Russia CIDR list from here and added this:

$testip = $_SERVER['REMOTE_ADDR'];
function ipCheck ($IP, $CIDR) {
  list ($net, $mask) = split ("/", $CIDR);
  $ip_net = ip2long ($net);
  $ip_mask = ~((1 << (32 - $mask)) - 1);
  $ip_ip = ip2long ($IP);
  $ip_ip_net = $ip_ip & $ip_mask;
  return ($ip_ip_net == $ip_net);
}
$CIDRs = file ("/path/to/ru.zone.file");
foreach ($CIDRs as $CIDR) {
  if (ipCheck ($testip, $CIDR)) {
    $act = "view"; // switches to viewing old comments rather than posting new one
    break;
  }
}

It's fast, and when comment spam shows up from other countries I don't care about, I'll block them too.

Re:DNSBL for comment spammers?

[Saac]

I havn't come across any centralized resource for this, but I was thinking of building one. Wanna help? I was thinking a simple page for people to submit new ip's to, and the ability to dl the list. So you could setup a cron job to update your list every night. What do you think? I'm willing to host.

Re:DNSBL for comment spammers?

[CFrankBernard]

http://akismet.com/

Re:DNSBL for comment spammers?

[Intron]

Many blog sites would be interested in a solution to comment spam. You might try letting them know that you're working on it. In fact, you should post a note on a few thousand blog sites letting them know about your website!

Re:DNSBL for comment spammers?

[joost]

Yes they can and they work great. My rails plugin uses them to great success.

Re:DNSBL for comment spammers?

[markjl]

I came across this but haven't tried it yet: http://www.bad-behavior.ioerror.us/ and of course, there are other interesting ways to prevent submissions: http://recaptcha.net/learnmore.html

Re:DNSBL for comment spammers?

[G-funk]

Add a field named "address" or whatever to your form, and give it an id="notshown" or whatever. Add in your stylesheet "#notshown { display:none }" and if you get a submission with that field filled in, it's spam.

Re:DNSBL for comment spammers?

[sg_oneill]

I'm starting to see even that one used.

And yeah, the javascript trick works well. I call it the 'browser turing test'. Its like a captcha for your browser to fill in, metaphorically speaking.

But I've even seen that damn thing work well.

Of course Akismet is the web spam filter that always works the best for me.

Requiring payment for delisting

[dbolger]

I used to work in the abuse department of an ISP which had been blacklisted by SORBS. SORBS require a "donation" to get your IP range off their list, and since we refused to hand over extortion money to these gangsters, there was no way for us to deal with them. Despite our best efforts, we also found that there was no way to get in contact with them, and as such no way to help our customers.

Doing a Google search for information about this lot brought up so many horror stories that I can't fathom how so many people ended up using their "service". It got to the stage where if we had a customer having trouble with SORBS blocking their mail, the only advice we could give was to contact their recipient via other means and ask them to stop using these thugs to filter mail.

Re:Requiring payment for delisting

[CopaceticOpus]

Amen! I have run up against SORBS blocking as well, and we refused to pay them. Unfortunately, their blacklisting service is used by a major U.S. supplier of email addresses. (I can't remember which one at the moment.)

Just say NO to SORBS!

Re:Requiring payment for delisting

[ciscoguy01]

SORBS require a "donation" to get your IP range off their list, and since we refused to hand over extortion money to these gangsters, there was no way for us to deal with them.
Which stinks to high heaven. I wish Matthew Sullivan wouldn't do that.

There are many reasons someone who is not an actual wrongdoer could become listed as a spam source. I have little doubt the parent's organization was such a spam source and did not properly address the issue. They deserved it.
It's not what problems you have, it's how you handle those problems is what matters.
As long as a site addresses the spam problem and gets results, reads their abuse mail and acts like a good net neighbor I have no problems with them. They should be delisted as soon as possible.
There have been times when certain cable modem operators were the major source of spam in the world and they essentially ignored abuse mail. They should have been disciplined until they clean up their act. Anyone who is not addressing the problem promptly deserves to be blackholed until they solve their problems.
There are plenty of clueless sysadmins in the world, people who are in over their head, or dominated by the company sales department so they cannot disable a circuit with deliberate spammers on it.
That's what DNSBLs are supposed to work to change.

Re:Requiring payment for delisting

[keeboo]

Well, we do use SORBS, but not the default filter they provide (which is way too agressive).
You may choose one more to your liking, as described here
I believe the best is to pick "safe" things like open relays, ADSL IPs and only the recently added hosts.

Yeah, I'm aware of all the horror histories on SORBS, but you know what? We maintain a public university mail server, the e-mail addresses are readily available everywhere (also, the users don't help either) AND we have a severe lack of technical personel (working on the perfect spam blocking system is not an option). And we're constantly being flooded with spam and attacks.
So, yeah, in our case SORBS is the lesser evil.

Re:Requiring payment for delisting

[Vellmont]

and since we refused to hand over extortion money to these gangsters, there was no way for us to deal with them

Extortion is a good word for it, but I'd say protection racketeering is a better one.

Re:Requiring payment for delisting

[Akatosh]

I guess whatever provider that was stopped, because I havn't heard a thing out of my users about Sorbs for a long time. They're irrelevant now, moreso since Sorbs shut their spam list down a few weeks ago after the founder had a breakdown. Did anyone even notice? That's how irrelevant they are.

Re:Requiring payment for delisting

[Zedrick]

I have little doubt the parent's organization was such a spam source and did not properly address the issue. They deserved it.

And what are you basing this belief on?

As long as a site addresses the spam problem and gets results, reads their abuse mail and acts like a good net neighbor I have no problems with them. They should be delisted as soon as possible.

Right. I work for a big webhost, which is blacklisted by SORBS from time to time. The problem is that they do not send abuse reports. (I handle abuse@mycompany and I do not miss or ignore one any mails). They blacklist, and expect you to pay. ...Which makes me think they're interested in the money, not preventing spam.

Contrast that to, for example, Spamcop who sends mails that clearly states what it's about, a copy of the mail headers and a nice link where you can let them know what's been done (such as shutting down the spammers account).

Re:Requiring payment for delisting

[ciscoguy01]

Uh, based on the belief that hardly anyone is actually rabid. I don't think SORBS is rabid. I doubt Matthew Sullivan is listing innocent people out of spite, though it was established that there habe been some DNSBL compilers who have actually done that.
It's pretty much counterproductive, listing people who are innocent, out of spite.
You just get false positives that way. The way a DNSBL becomes "influential" is by getting people to use it. If no one uses a DNSBL because it interferes with legitimate mail who cares what it lists? No one. So no DNSBL would have the goal of interfering with legitimate mail, unless the underlying goal is to change behavior.

CBL.abuseat.org is an example of a mostly (or maybe completely) automated DNSBL which should have NO false positives at all. It works by lots of spamtraps, if botnets hit the spamtraps in a certain "bot-ish" way as I understand it the IP will become listed, for a short time. After the TTL expires the listing falls off. So if someone cleans the trojan off their computer being used by a botnet to send spam that listing will fall off fairly quickly.
It's a pretty much perfect anti bot-based spam solution. Good for blocking spam, not for changing behavior.

Some lists like SPEWS and now APEWS and SBL list based on different criteria, criteria that is obviously designed to change behavior, not to block spam though people use it for that, hopefully in non business critical environments. I wouldn't use that for business. If you are an ISP and you don't resolve your spam problem by terminating your spammers and disabling trojaned machines you get listed, if you don't like that you change your policies to prevent it. Some allow you to request delisting which takes place promptly if your spam has actually stopped. No problems.
The first two I mentioned are anonymous, no one actually knows who is behind them.
This makes perfect sense in light of how MAPS was sued into oblivion by spammers and forced to delist known spam sources because of their legal expenses.
SBL is in UK and the UK loser pays legal system largely prevents foreign entities from affecting their operations, at least legally.

I said SORBS money request is improper. I wish he wouldn't do that. But understand he has been sued and he does likely have some legal expenses so that may be why he is doing it.

As far as I can remember SORBS started out as publishing other's DNSBL zones in a queryable way, for free, he took over when Osorusoft went away. He may have some he maintains himself now.
Remember Osorusoft? They were sued into turning off their zone, they had been providing a free lookup service that SPEWS users could query. That kind of nonsense costs money, and spammers have money. They sue DNSBLS.

A free DNSBL query service like SORBS is a target, has been a target and may be targeted again. Sad, since nobody has to query them on inbound mail. Clearly they block no mail. Their users do, as a conscious decision. But it's too hard for spammers to sue giant ISPs for using SORBS.

Re:Requiring payment for delisting

[ciscoguy01]

Right. I work for a big webhost, which is blacklisted by SORBS from time to time. The problem is that they do not send abuse reports. (I handle abuse@mycompany and I do not miss or ignore one any mails). They blacklist, and expect you to pay. ...Which makes me think they're interested in the money, not preventing spam.

That is wrong, then. They should send abuse reports.
Are you sure there are no abuse reports?
It's unlikely they would be *From: SORBS*.
They might be anonymous, like the ones SPEWS reportedly used to send before listing a block. Several reports before they listed you, as I recall.
If you want to avoid being listed on DNSBLs you cannot just act on the *important* spam abuse reports, like the ones from spamcop and ignore one from someone you don't know, since that might be from the APEWS, SORBS or SPEWS operator. They test your veracity by not telling you they are influential.
Compltely proper, IMHO.
Given the true goal is stopping spammers, not blocking spam.

Re:Requiring payment for delisting

[dmpyron]

If the true goal is to go after the spammers, how does a DNSBL help this? You aren't going after the spammers, you're blocking the spam. I send out emails to about 300 members of my professional association. I get bounces saying some have been blocked by blacklist, but don't say who. How am I supposed to get off the blacklist? We never get an email from any blacklister. If we ever get blacklisted by SORBS or any other extortionist and they ask for money, we'll probably sue and/or file a criminal complaint.

Re:Requiring payment for delisting

[nuzak]

> But understand he has been sued and he does likely have some legal expenses

I bet he incurs even more liability by having an apparent financial interest. By the way, has anyone ever told you that overuse of boldface is really annoying?

Re:Requiring payment for delisting

[ciscoguy01]

If the true goal is to go after the spammers, how does a DNSBL help this?

ISPs have customers, customers who want their mail to go through. Customers like you. If an ISP has lax abuse policies (or no abuse policies, or is a willing spam host) and you are a legitimate customer of that ISP, your mail may be blocked with the other legitimate customers of the ISP.
You are not being listed, your ISP is.

The DNSBL hopes you will call your ISP, and as a valuable customer demand they cure their spam problem so you will be able to send mail.

If an ISP's customer is spamming me all I can do is complain, and they can ignore me. You are their customer, you are influential and you want your mail to go through, so you are completely within your rights to demand they get rid of their spammers that are causing you problems. Your ISP can make a choice, either deal with spammers and all their legitimate customers go elsewhere or sue them, or get rid of the spammers and have you, legitimate customers.

It makes perfect sense, doesn't it?

If we ever get blacklisted by SORBS or any other extortionist and they ask for money, we'll probably sue and/or file a criminal complaint.
Criminal complaint? Nobody has to accept your email!
If you are a spammer that's what you might do, which is why most of the DNSBLs are in countries other than the US where they are protected by the local laws from lawsuits like that.
What you should do is sue your ISP for getting you listed along with them, or demand they cure their spam problem.
Unless it's you that are the spammers, that is.

Re:Requiring payment for delisting

[D'Arque Bishop]

If an ISP's customer is spamming me all I can do is complain, and they can ignore me. You are their customer, you are influential and you want your mail to go through, so you are completely within your rights to demand they get rid of their spammers that are causing you problems. Your ISP can make a choice, either deal with spammers and all their legitimate customers go elsewhere or sue them, or get rid of the spammers and have you, legitimate customers.

It makes perfect sense, doesn't it?

The problem I have with your argument is the fact that it assumes that 100% of the people who are on the list deserve to be on there. It does not take into account human error in placing the address into the blocklist, or the fact that maybe the ISP caught the spammers themselves without being notified first and still got blacklisted.

That's the big reason I have such an issue with SORBS. If you cannot be removed without paying a "donation" no matter what, then that is tantamount to blackmail. The fact that no one else has to accept your email doesn't make it any less blackmail. Unless you're a believer that the ends justify the means, then there's no reason for you to be justifying unethical behavior to fight unethical behavior. You just end up being dirty as well.

Re:Requiring payment for delisting

[91degrees]

Criminal complaint? Nobody has to accept your email!

I always love comments like this. It's nice to believe that every lawyer in the country might think, "Oh, golly. They have a point, and out of literally thousands of laws on the books, there isn't one that might be appropriate"

Many countries use an adversarial system. That means that they will be making a really strong argument that the DNSBL owner is guilty of extortion. And by making demands from a US company they are trading in the US. That means US law applies.

Other countries have a different legal system, but they will consider whether listing an organisation as a spammer and demanding money to remove them from the list whether they are still sending spam or not is extortion.

Re:Requiring payment for delisting

[naChoZ]

We noticed. We were using their dynamic ip range list so we started noticing it pretty quickly.

As for the extortion fee for getting off their list, we never had to pay it. We would explain the problem, note that we took steps to correct the issue, and they'd remove us with no fee. On one occasion where it really was our fault for fat fingering something and they really wanted to charge us the extortion fee, we just whimpered and cried and bowed and scraped a little bit and they took us off anyway with no fee. But, like you mentioned, they're irrelevant now anyway, without that useful dynamic range list.

Re:Requiring payment for delisting

[ciscoguy01]

The problem I have with your argument is the fact that it assumes that 100% of the people who are on the list deserve to be on there. It does not take into account human error in placing the address into the blocklist, or the fact that maybe the ISP caught the spammers themselves without being notified first and still got blacklisted.

I never said *100%* of the people that might find themselves blacklisted deserved to be there, but different blacklists have different goals. Some, like CBL are purely for blocking spam, list no innocent parties, and are very safe for corporate use.

Some, like SPEWS, and some SORBS zones are not safe for corporate use. Some might have significant intentional false positives. They are there to help put pressure on lax isps and force them to deal with their spammers. If they blocked too much legit mail they would lose their influence if people find them unsafe to use. So they have to walk a fine line. But they are there to enlist your help when you find yourself blocked, to get you to call your ISP.

That's the big reason I have such an issue with SORBS. If you cannot be removed without paying a "donation" no matter what, then that is tantamount to blackmail.

And I said that was wrong. SORBS shouldn't charge for removal, it makes them have unclean hands. Apparently they don't agree with me, or you on that.

Re:Requiring payment for delisting

[a_nonamiss]

I agree with the goals of SORBS, but I had an experience recently which proved to me that they are utterly and completely worthless.

A girl who works for a company that I support has never engaged in spamming in any way. Their corporate network is secure. Their mail is hosted by some company. They don't know the details, but their email usually "just works"
She tries to send an email to a perspective client, and it gets bounced due to SORBS
She calls her boss
Her boss calls me, the company consultant.
Charging $100 per hour, I contact SORBS. After a day and a half, they tell me that I am not the owner of the IP address, and thus I can do absolutely nothing. I need to contact my webhost
I call their webhost, who also hosts their email
The site host contacts SORBS
After another day and a half, the webhost is informed that the owner of the larger IP address block is responsible. They will do absolutely nothing for the webhost.
The webhost contacts his provider. They contact SORBS. After 18 hours, SORBS contacts them back with the IP address of the offending spammer that caused the blacklist in the first place.
The IP address does not directly belong to them, but is part of a large block managed by THEIR upstream provider, which willingly allows what they don't consider to be spam, but SORBS does consider to be spam. (Something to do with specific requirements for Australia's spam prevention act. I don't know the details because by this time, it's too far upstream for me to have any direct contact with anyone.)
Here are the choices I am left with:
1) Switch to a new host. This is hardly trivial, and will cost hundreds in billable hours to my client.
2) Persuade the webhost to switch ISP's. Not likely.
3) The webhost's upstream provider must convince the people who manage the large block to cancel the contract of a paying customer who is NOT violating the TOC.

It's worth noting here, that this ISP does not appear on ANY other blacklists. Only SORBS. The webhost is a responsible netizen, who happens to have an upstream provider who allowed some activity that SORBS classifies as spam. (and NO other blacklist.) In the end: I billed my customer for hours of time spent investigating the issue. He lost the contract because he could not submit his bid on time. Sure, I could switch him to another webhost now, but that would cost him quite a bit of money. He has done nothing wrong, I have done nothing wrong, the webhost did nothing wrong, and his datacenter did nothing wrong. However, to address the issue properly, it will cost thousands of dollars in lost productivity, consultation fees and potentially lost business. There is still no guarantee that a new webhost/ISP/datacenter won't have the same problem again. Moreover, it took 2 1/2 weeks to determine that we were at an impasse with SORBS and could do nothing to remedy the situation.

I hate spam with a passion, but how are they doing any favors for the Internet? I know how it's supposed to work in theory, but clearly, it's broken.

Re:Requiring payment for delisting

[dodobh]

*shrug*. If you have a good working relationship with SORBS, you get unlisted very fast. That implies terminating spammers really quickly. If you are doing that, and consistently doing it, you might want to give SORBS a heads up on unlisting you (yes, they do it).

Re:Requiring payment for delisting

[tokul]

I used to work in the abuse department of an ISP which had been blacklisted by SORBS. SORBS require a "donation" to get your IP range off their list, and since we refused to hand over extortion money to these gangsters, there was no way for us to deal with them. Despite our best efforts, we also found that there was no way to get in contact with them, and as such no way to help our customers.

Checked delisting rules

Fee is required only if you are listed as spammer.

Re:Requiring payment for delisting

[fm6]

Persuade the webhost to switch ISP's. Not likely.

Especially when the ISP is also the colo provider. Which is usually the case. Nobody is going to move their servers to a new data center without a lot of motivation. Like a meteor headed towards the old data center.

I'm amazed that we're still talking about black lists at this late date. On top of all the nonsense with punishing innocent folks, screwing up legitimate email, increasing user costs, and accusations of extortion, there's one little detail everyone keeps forgetting to mention: it doesn't f***king work! Most spam now comes from botnets, and stopping those by blacklisting IP blocks is like fighting off locusts with a flyswatter.

Re:Requiring payment for delisting

[a_nonamiss]

So there's a reasonable chance that you're just a troll, but in case you really are that ignorant, and since you didn't post AC, allow me to swallow your flamebait.

Nowhere in my post did I say anything about not knowing the owner of the address block. I knew who it was in the first 5 minutes. Of course, since SORBS runs the blacklist, they were my first point of contact. At the time, I didn't know how unreasonable their policies were. From there, I assume you are actually suggesting that I call up Qwest and say "Hey, I'm a consultant for a customer of a customer of a customer of yours, and I demand that you terminate the contract of your other customer, who, by the way, isn't violating your terms of service, just so my customer can send email now." That's not how it works. Generally, in business, you can only work with your vendor on these problems. You can't go three levels up the chain just to save time.

Apparently you're job at Burger King hasn't exposed you to the complexities of the IT field, so let me explain in terms you can understand:

Spam bad. Customer upset. SORBS policy makes no sense. Makes things hard to fix.

Understand now?

Re:Requiring payment for delisting

[a_nonamiss]

Like a meteor headed towards the old data center.
Webhost: "How big did you say this meteor is?... Oh, that big. Hmmm. I see... Is it going to take out the whole datacenter, or just part of it?... Ah. Ok... What kind of downtime are we talking about here?...

Re:Requiring payment for delisting

[mvdwege]

So why the cock-and-bull story about how it took so long to track down the netblock owner responsible? You were just racking up billable hours, weren't you?

If your customer was listed on one of the actual spam blacklists, the problem should have been lower in the hierarchy. I'm guessing your customer got on the DUHL. Well, guess what, that could have been prevented. I have two words for you: due diligence. It's not as if Qwest has a spotless reputation when it comes to spam.

Now, it may not have been you that set up their domain, but it is still not SORBS' problem that your customer is easily fleeced by a bunch of con men. It's guys like you and the ones that set up their domain that give consultants a bad name.

And as for my job, I wasn't bragging that I was a one-hunderd-dollar-an-hour consultant that couldn't find his arse with two hands. Even if I did work at Burger King, my arguments as to why you should remove yourself from the gene pool are relevant. You are just trying to blame others for your own failures.

Have a nice day, luser.

Re:Requiring payment for delisting

[a_nonamiss]

My story had nothing to do with finding the block owner. Read it again. It had to do with trying to get the issue resolved. If you order a cheeseburger, and the cheese on it makes you sick, you don't just immediately run over to the local dairy farm and start throwing a tantrum. You start with the people that you purchased the cheeseburger from.

It sounds like you're real problem here is that you don't feel my services are worth the money by boss charges for them. I, by the way, don't make anywhere near $100 per hour. I put that dollar amount in there to put in perspective how much this fiasco was costing my customer, although they did absolutely nothing wrong. Also, in my city, the Geek Squad charges (much) more than $100 per hour, so it's not like I'm putting myself on a pedestal. My company charges a very competitive rate, I work very hard, and I know what I'm doing, despite your best efforts to "prove" otherwise.

Yes, it was the DUHL that they were on. You say "due diligence." What would your recommended course of action have been? You offer vague, nebulous terms with big words that you heard on Law & Order, but I think you have no real idea how this could have been prevented. (Oh, I know, you should have never used a company that is located in a facility that has an IP block owned by Qwest. Guess what. There are a LOT of reasons much too complicated for you to understand as to why they haven't switched hosts. All you need to understand is that it's not an option.) You have two posts on here in response to me, and in neither of them did you offer any suggestions as to how a real IT person should have handled them. That suggests to me that you don't know.

Re:Requiring payment for delisting

[mvdwege]

You know, with every post, your gaffes and contradictions mount. The idea is to stop digging if you find yourself at the bottom of a hole. The fact that you apparently only know the meaning of 'due diligence' in the context of a TV series is but the least of your errors.

Mart

Re:Requiring payment for delisting

[a_nonamiss]

Marty, I've put enough effort into this futile flame-war. Unfortunately, in order to have a conversation with someone, you need to be on similar intellectual levels, and clearly, we are not. You fail to include any meaningful information in your replies. You avoid answering points directly, and try to refute points I don't make. I'll let this thread stand on its own, and the people reading it can draw their own conclusions. I wish you the best of luck in life.

Local Whitelisting!

[HitekHobo]

Choosing a good DNSBL (or three!) is definitely important, but IMHO, you should NEVER run DNSBL's without building a local override into the system. We run our own DNSWL (dns whitelist) which is consulted before hitting on BLs... if a customer has had problems with one of their contacts being blacklisted, we can selectively add their IP to the list.

Unrelated to the above, I would also recommend looking at ironport systems if this is a commercial project with a decent sized budget. (I am not affiliated, just a happy customer).

Re:Local Whitelisting!

[More Trouble]

Choosing a good DNSBL (or three!) is definitely important, but IMHO, you should NEVER run DNSBL's without building a local override into the system. Having a whitelist is definitely important, but how you use the blacklist is even more important. The staff time required to simply maintain a whitelist of all the mistakes found in various blacklists is simply a waste. Better to use the blacklists in a mostly advisory capacity.

:wes

There is no such thing as a good DNSBL

[deviator]

They all have issues; all of them create headaches for administrators of legitimate e-mail servers at one time or another.

Re:There is no such thing as a good DNSBL

[seebs]

Of course they do. That doesn't mean they're not good; it means they're not perfect.

The fact is, without DNSBLs, the headaches would be worse. LOTS worse. Centralized blocking gives you some kind of theoretical hope of getting unblocked once you've fixed the problem. Decentralized blocking leaves you no chance at all. Furthermore, without tools like DNSBLs, administrators would be far too busy to even get to the point where they could have these headaches.

I'd rather live in a world with a number of reasonably good DNSBLs than not have any.

Re:There is no such thing as a good DNSBL

[MadMidnightBomber]

Bollocks. I used to run email for a university with around 50,000 students (and around 500,000 deliverable email addresses - don't ask). We had one issue during 2 years which was a local college had got itself listed in one of the spamcop zones, because it had turned into a spam relay. So the DNSBL was working as desired. We whitelisted them as they had fixed the problem, but the listing expired around the same time anyway.

During that period we were dumping about 50% of inbound mail thanks to DNSBLs, with no complaints. Our users would have thrown away more legitimate mail than the DNSBL did, had we not been filtering.

Re:There is no such thing as a good DNSBL

[deviator]

Good spam filtering software can accomplish the same thing.

We work with lots of customers who absolutely rely on e-mail for business correspondence: occasionally they are unwittingly listed in some RBL and removing them is a pain in the ass. Who made Joe of JOESCRAPPYDNSBL God, telling our customers' customers not to receive e-mail from them?

Unfortunately there are alot of very bad examples of DNSBLs, and there are a lot of very bad examples of e-mail admins out there - putting the two together just causes headaches for us. Software-based filters seem to be a better option.

Re:There is no such thing as a good DNSBL

[deviator]

I think server-side and user-side spam filtering software would get a lot better more quickly if there were no DNSBLs. :)

Re:There is no such thing as a good DNSBL

[seebs]

I don't think so. The thing you need is the ability to determine whether or not other people are getting substantively identical messages, and frankly, just blocking the bad networks is an order of magnitude more efficient.

No, that's not right.

It's at least three or four orders of magnitude more efficient.

NEVER use a DNSBL as an absolute block

[ebunga]

DNSBLs are subject to the whims of some of the most unreliable and whiny schmucks on the face of the planet. NEVER under any circumstances use a single DNSBL as an absolute block. Use it to increment a score along the lines of Spam Assassin that will eventually hit a threshold, preferably with a minimal content-based component. Don't even think about using multiple hits on multiple lists as a gauge of spam-worthiness. The amount of inbreeding and sharing among lists is disgustingly high. Not even the Spamhaus aggregate is trustworthy these days.

Spammers can get around blacklists anyways. They're about as effective as locking a door made of tissue paper. The number of false positives is high. The amount of spam blocked is negligible. My suggestion is to abandon the idea altogether.

Re:NEVER use a DNSBL as an absolute block

[Shaman]

Sounds good, except it's not true. I was just on one of our spam systems (Barracuda 400) and the stats look something like this:

20,000,000 blocked e-mails
480,000 tagged e-mails
90,000 viruses found
135,000 quarantined messages (user choice to quarantine or not)
610,000 delivered/approved mail

To nobody's surprise, some spam is still getting through. This is in less than two weeks, and there are two servers to handle the load, the other one is more or less as bad.

So what were you saying about not using blacklists?

Re:NEVER use a DNSBL as an absolute block

[ion]

how many of your 20,000,000 blocked emails are false positives? aka legit email.

I would so much agree that using a DNSBL as a absolute block is a bad idea. I have experienced being caught up in them, and that is annoying. Even if the mailserver is removed some days later. Later is not soon enough, i want my email to arrive now.

I would much rather suggest running some sort of spamassassin while the SMTP connection is still open, and if it looks like spam i would reject it. This can be parallized if needed.

I would also consider to reject any email that came with an attachment if you have not already received legit email from the same address. This tries to use that spammers seldom send from the same email address and that they started sending attachments. Legit email does not usually start with an attachment in the first email (at least mine does not). So, if you previously received emails and that email address has a negative spam score, aka not being a spammer, then i would accept attachments, else i would not.

This might be pr. domain, but hotmail and others are often used by spammers. This could lead to a domain spamscore, aka if you received emails before from this domain and none was spam, then accept attachments, even if it is the first time someone sends you something from the domain. This is for company uses where John sends you something and then later Jane sends you something with an attachment.

You might want to allow certain kinds of attachments even if they are not listed before. These attachments could be .vcf files, and possible .html, but not .pdf or .jpg

Re:NEVER use a DNSBL as an absolute block

[GReaper]

Couldn't agree more with this.

I use several DNSBLs at SMTP level, however instead of blocking any blacklisted IP they get greylisted. The majority of zombie machines never bother trying to resend the mail, so it cuts out a large amount of spam. Any blacklisted IP address which does successfully resend gets added to the whitelist so they don't have to bother with the greylist.

For our users it works perfectly. Users from non-blacklisted IPs get their mail sent immediately, those who are blacklisted get a short delay.

Re:NEVER use a DNSBL as an absolute block

[ion]

yes i know. But arent those usually called .vcf files? which i already said i would allow though.
If not you could recognize the vCards by extracting the attachment and run some tool to recognize a vCard. And then allow it all through.

Re:NEVER use a DNSBL as an absolute block

[ion]

and if this blocking of attachments forces spammers to send small text emails first and then send the .pdf or .jpg attachment 5 minutes later, you could further require that you have reply to this or other messages comming from the same email address or the same domain (carefull with hotmail). Hopefully this approach could render attachment spam more useless.

Re:NEVER use a DNSBL as an absolute block

[Pontiac]

Sounds like your block numbers are about the same as ours..

We run a 2 layer system with a cluster of St Bernard Eprism 2000 appliances and a software filter on the mailbox servers.

Right now we reject 10 million a week on RBL
Another 4.5 million in spam filters
We pass about 400,000 as legitimate mail..
Our virus rate is only 400 but the firewall is also doing AV filtering so I don't see what it's catching.

The false positive rate is very low with the Borderware RBL list the Eprism devices use..

I have more false positives on the software based filtering using List.dsbl.org and spamhaus.

The really interesting this is the jump in the RBL blocks in the last 2 weeks.. It jumped from 8 million to 10 million in 4 days.

If anyone is looking to implement RBL's with this kind of volume be sure your DNS servers are up to the job and think about setting up your own RBL servers in house. Slow DNS resolution can cause some big mailfow problems.

Re:NEVER use a DNSBL as an absolute block

[Pontiac]

I forgot a few points..

Every new Block list you add adds another DNS lookup for each message you receive.

You can configure a linux box running RBLDNSD to sync multiple block lists and perform a single lookup against all those lists on a singe Query..

The down side to that is you won't know what list did the blocking but it's great for taking some load off the DNS server.

Re:NEVER use a DNSBL as an absolute block

[Samhain]

>Legit email does not usually start with an attachment in the first email

I guess you are not counting the many many corporate users who send a vcf card with all of their e-mails by default using outlook. This appears as an attachment. Or that have a jpeg/gif of their company logo in their signature (not that I like either of these things -- but they happen a lot!)

Generalizing, that 'well this never happens to me' usually does not work -- or is that a generalization.

Re:NEVER use a DNSBL as an absolute block

[KC7GR]

"Spammers can get around blacklists anyways. They're about as effective as locking a door made of tissue paper. The number of false positives is high. The amount of spam blocked is negligible. My suggestion is to abandon the idea altogether..."

Thank you for your suggestion. It will be duly ignored, laughed at, or similarly ridiculed by those of us who actually run our own mail systems, or are responsible for such at work.

In my case, I'm self-hosted. Authoritative DNS for my domains, mail, web, Usenet, the works. I can say, from five-plus years of direct experience, that your statements above are just plain wrong.

I use a combination of Spamhaus and my own home-grown blacklist to keep spam in check. The few false positives I've gotten over the years have been ENTIRELY due to overly-broad entries in my LOCAL list, and have been easily and quickly corrected by white-listing.

I have NEVER received a false positive from any Spamhaus entry. Not once. The old SPEWS list, yes, but I haven't used them in years (and I'm leery of their successor, APEWS).

The fact remains that those who own mail systems have absolute and total authority over who they choose to communicate (or not) with. If a mail server operator decides to block a single address, a /24 IP address subnet, or even an entire country, that is their privilege. There is no legal recourse I know of that can force anyone to open their server(s) to traffic that they do not wish to carry (in short: private property rights).

Keep the peace(es).

Re:NEVER use a DNSBL as an absolute block

[Anonymous Coward]

how many of your 20,000,000 blocked emails are false positives? aka legit email.

We publish our help desk number in the bounced message. In the 9 months we've been using them, we've been called once. So lets imagine the real rate is 100x worse than that. A false positive rate of .0005%. Which is far higher than many of our Spam Assissin rules, and far higher than the miss rate caused by spam floods tying up servers.

Pick you DNSBL carefully and they work wonders, cutting server loads by orders of magnitudes (We block 98% of incoming connections these days based on DNSBL's, so that effectively makes my server 50x more powerful) Your position on such a powerful anti-spam tech makes me think you yourself are a spammer.

Re:NEVER use a DNSBL as an absolute block

[DavidTC]

Well, to be fair to SPEWS, you probably didn't get an actual false positive by their definition. They don't block spammers, they block spam-supporting ISPs, which quite deliberately includes many IPs that never send spam and are extremely unlike to start. I know you weren't attacking SPEWs, but everyone seems to misuse SPEWS and then complain it's blocking exactly what it said it would block, include legit email. Well, yeah, they said it would do that.

It's sorta like complaining that superglue is defective because it won't come off your hands and can even glue you to yourself, unlike 'other wood glue'. That makes it inappropriate to use in many or even most circumstances, not 'defective', because it's not supposed to be 'wood glue', and it quite clearly says it will stick to people. :)

Anyway, I wouldn't use SPEWS or APEWS to block (I wouldn't use any DNSBL to block, but certainly not them.), but I found SPEWS handy for scoring on, in that there is a demonstratable increase in likelihood that if an email is coming from a SPEWS level-1 listed IP that it is spam. It's nowhere near 100%, but it's still useful. (Although at this point I'm not using any DNSBLs to filter except for locating dynamic IPs, and then only to greylist. It's scored in spamassassin, but that's for end users to filter on if they want.)

I haven't looked into APEWS. Am I right in assuming there's a story there about that and SPEWS? I need to start reading NANAE again.

And I, too, love Spamhaus, I've been tempted to actually block entirely on that, despite me generally disliking doing that.

As for the GP, in any discussion about spam-fighting, on slashdot, there will be dozens of people with fucking stupid ideas because they don't actually run a mail server, or run a tiny Linux box on DSL that is their 'mail server' that gets maybe 10 pieces of email a month. Because they are fools who want to 'be in control of their email', they sit around and whine that their email isn't accepted by everyone, which they could fix by simply using their ISP's mail server. Meanwhile, they aren't required by anyone to actually run a functioning mail server that can receive email from unknown people and filter out a large proportion of spam, so have no idea how that actually works. Just ignore them.

As for all the people I was just talking about: Look, I don't walk up to people and tell them how to optimize an heavily-loaded Oracle server, how to write a optimal bytecode interpeter, or how to build a world-wide network, because I don't have any experience doing those fucking things. I'm think I'm smart enough, I could learn how and have useful opinions, I just have no practical knowledge at all right now, so I don't make random suggestions. If you haven't run a mail server, in the past five years, where other people expect to be able to receive random legit email, and complain if they get too much spam, STFU on this discussion, please.

Re:NEVER use a DNSBL as an absolute block

[ion]

I already wrote in my original statement that i would allow .vcf files. The small logo's might become a problem because alot of spammers also send small .jpg's, but then you could look at the rest of the email, the headers, lookup the MX records for the domain, ask if the MX servers wants to accept email to the sender, and possibly check if there is a website for the domain. But ... maybe companies should try to avoid sending the same kind of attachments that spammers does to distinguish themselves from spammers.

Re:NEVER use a DNSBL as an absolute block

[ion]

No i am not a spammer, but i have been burned by DNSBL more than once.

I have tried to send legit email to someone i had sent to before. But just because the local ISP that i sent through was in a DNSBL then i could not get my email through and i had to wait days before i could continue the emailing. That was annoying. Other times one of my customers called and complained because one of their customers ISP was in a DNSBL.

I didnt particular setup the usage of DNSBL, i just used default postfix configuration for Debian. I still had to turn it off.

So i dont like this black or white usage of a DNSBL, i prefer the scoring system of a system like spamassassin and looking at previous history of email partners to whitelist email address and possibly domains.

I also tried looking at all my attachment spam and see if i can work out a general rule. It seems to me that noone sends email to me with an attachment upfront, they sent text emails first. Later they might send attachments.

Unnecessary acronym

[nightsweat]

DNS BL? DNS blacklist. Same number of syllables. DNS makes sense since it is only three syllables instead of "do-main name ser-vice (or Sys-tem)" which is 5 syllables.
But BL for Blacklist? Nah.

Speed, Selection Criteria and Goals make the list?

[jsse]

No...

It's how quick the maintainers of this particular DNSBL responding to your request to remove your ass from the list when they choose to blacklist you.

We've multiple MTAs for a single mail domain, because when an attacker found some way to relay or bounce-back one of our MTA and cause it to be backlisted by major DNSBL on earth, we still have other MTAs take up the job.

Then we could spend the rest of the week to ask for removing that MTA from their DNSBL, by email, or worse, by forum.

Trust me, it's painful.

This has a score of two?!

[Anonymous Coward]

Greetings, sir,

Allow me to introduce myself. I'm a representative of the Consortium of Common Sense. I've noticed you recently posted to an Internet-based conversation, complaining about the reduction of a nine-letter word to two letters via acronym. Your post referenced such things as numbers of syllables.

Please look at your desk now, and slam your head down as hard as you can on it. Do you feel those weird little indentations in your forehead?

THEY'RE CALLED KEYS - DID YOU NOT REALIZE THAT THINGS ARE TYPED, NOT SPOKEN, ON THE INTERNET?

Thank you. Please let us know if you have any other ridiculous complaints.

- Consortium for Common Sense

Use as many as you can get your hands on.

[syntax]

I seldom trust the results of a single RBL. The best technique, and what SpamAssassin does, is to check against a ton of them. I myself have gotten my own server listed on a handful of blacklists, but not from sending out email. I just happened to be in the same Class C block as another server that had been a relay over a year ago. This became a problem with mail servers that would block your mail off of a single BL hit. I gave up trying to negotiate with the BL and my SP (Rackspace) and just changed the default outgoing IP on my load balancer -- probably not an option many people stumble across.

Just make sure that despite using them, you don't trust them as absolute.

But back on topic, I've always been a big fan of SpamCop.

Whole countries

[DogDude]

I don't bother with blacklists. It's easier to just eliminate all traffic from whole countries. I get a spam from China. I look up the ISP. I block all traffic to/from that entire ISP's block. Done. Same thing for former Soviet states, and other such places. It works amazingly well. Of course, this doesn't help with zombified PC's, but neither does a DNS black list.

NEVER use a DNSBL as an absolute block...

[HitekHobo]

...unless you have to.

There is a lot of truth to the OP's statements. However, unless you have the budget for a commercial spam filtering application, there are not a lot of good solutions.

Spamassassin is great for what it does, but in high volume environments, you will be throwing so much hardware, bandwidth and electricity at the problem that you'll either give up on filtering at all or break down and buy a commercial solution.

DNSBL's give you a bit of breathing room between the two extremes. Our environment has about a 98% spam catch rate currently with commercial solutions. We have about 150 connections per second AVERAGE.

Our infrastructure could just barely keep up with this load when we were using DNSBL's only. Had we tried to use a spamassassin style tool, we'd have needed quite a bit more infrastructure to handle all of the increased filtering. DNS lookups are pretty cheap compared to the amount of CPU required for context / content filtering.

DNSBL's definitely generate too many false positives, but when the alternative is buying 10x the hardware or having mail take 1-2 hours to be delivered during peak times, I'll take the false positives.

Re:NEVER use a DNSBL as an absolute block...

[ebunga]

At work, we don't use spamassassin. There are better, albeit more expensive tools that provide everything spamassassin does and more at a fraction of the cpu usage. You're doing about four times what my mail system handles on average. Our filtering system alone uses seven machines. Out of our entire load, less than 100,000 legitimate messages are received each day. The amount of legitimate mail is statistically insignificant. It's easier to say we receive 100% spam.

Rather than subjecting ourselves to the whims of a DNSBL operator, we run our own internal blacklist. Without it, the mail system would not work at all. Sure, we have just as many false positives as typical DNSBLs, but at least the blame rests squarely on my shoulders. Since I'm only a phone call or email away and actually care about paying customers, there's plenty of incentive to fix any problems. Additionally, the list isn't really a DNSBL. Rather, the blocks happen on a load balancer/firewall. Why waste resources with a DNS query?

Re:NEVER use a DNSBL as an absolute block...

[HitekHobo]

After much bludgeoning of our accounting folks, we were finally able to get some ironport systems in house. I wouldn't say they have the magic bullet to spam filtering, but they do take a pretty unique approach. It is based on a particular IP's previous mailing habits and it's current mailing habits, the number of complaints received etc. Also, you can choose to simply throttle suspected spammers rather than block them. After quite a bit more bludgeoning, we were able to get content based filtering added into the cluster. At the end of the day, it cost quite a bit, but our false positives are pretty low and we only allow about 2% of all inbound connections to actually send mail. We run 4 inbound servers (of varying ages... 2 newer models would probably handle the load) and 2 outbound servers (pretty old hardware outbound). The outbound scanning has really kept us off a ton of DNSBL's and I don't think we've actually had a customer complaint that they were unable to send out their 'mailing list' yet. We set things up so our tier 2 CSRs can manage the override lists via a DNS whitelist. Consequently, I have not taken a spam / blocking related call in over a year. Don't get me wrong, people will still call and bitch that they received spam or their inbound mail was blocked, but our complaints are in the single digit per week range these days. Unrelated: There is nothing like having a 'dead' domain that receives several million spam messages per day when it comes time to evaluate commercial solutions!

Re:NEVER use a DNSBL as an absolute block...

[TheRaven64]

The nice thing about a DNSBL is that it lets you reject with an error while the SMTP connection is still open. This means that even if you drop some emails, the sender is guaranteed to find out about it, and if it's important they can try again from (or to) another address or contact you some other way. This is preferable, in my mind, to having an email shuffled off into a spam folder by something like Spamassassin, where neither the sender nor recipient will find out about it for a long time.

I prefer to use OpenBSD's spamd as the first filter; it uses a static IP list, is very low overhead, and tar-pits the spammer, preventing a lot of zombie-types sending you email after email for different users. The next pass is a DNSBL. Both of these give an error message to the sender. The next step is Spamassassin. By this stage there aren't many spam emails getting through, and so the overhead is quite low.

Re:NEVER use a DNSBL as an absolute block...

[DavidTC]

Don't use a non-local blacklist to absolutely block anyone. It's just too dangerous.

Many solutions let you use weighted DNSBLs. maRBL is a good one for postfix. Not only can you do weighted tests based on lists, you can do tests based on OS using p0f. And have multiple tests.

So I have a profile that checks some trusted lists, and you have to be on three or four to get blocked. Then I check the OS. Then I check some dialup regexps and some dialup DNSBLs. If you've either on Windows or apparently a dynamic connection, you get greylisted.

Even if spammers come back, they're blacklisted by then.

This doesn't require any more bandwidth or CPU than the tests by themselves.

Dynamic IPs / Zombies

[HitekHobo]

Actually, using a blacklist that is purely dynamic IP's works quite well for zombies. I won't recommend one in particular, but there are several lists with just this purpose.

Re:Dynamic IPs / Zombies

[Jeffrey Baker]

Except the blacklists which are supposedly dynamic IPs contain tons of other shit. There is one which contains any IP which reverses to a name containing the letters "dsl". This is pretty stupid since a lot of business DSL lines have static IPs and because Speakeasy business T1 lines also reverse to whatever.city.dsl.speakeasy.net. Other ISPs have the same scheme, and they don't all delegate reverse DNS. I have a business MX hosted on a T1 line that's blocked by some blacklist that Earthlink uses. So I can't send mail from that business to anyone at Earthlink. It's a really stupid policy.

missing rDNS?

[fl!ptop]

no one has (yet) mentioned using the missing rDNS sendmail hack. i block about 100,000 messages and servers per week using a combination of send_pause, blacklists, spamcop, iptables and the rDNS hack. rDNS routinely accounts for more than 50% of the spam that never makes it to my server.

any mail server that doesn't have an rDNS lookup, in this day and age, is imho not worth accepting messages from.

Re:missing rDNS?

[macdaddy]

Lovely. So you're one of "those" people that interpret the standards as you see fit. Nice. So what you're saying here is that you're actually blocking all mail from people who's DNS is broken when you make your query. Or for people who's have complete in-addr.arpa zones but the single UDP reply packet got lost along the way. That's rather expected considering UDP is of course "best effort" and "unreliable". I have no doubt that you're blocking a large amount of spam. That's a given. I also have no doubt that you're blocking a large amount of ham as well. The RFCs are written for a reason and by very smart people I might add. Strict adherence to the standards is the only way to ensure accurate communication for this medium. We can't simply interpret the standards any damn way we please. Reverse DNS is not required by RFC 2821. It's recommended but not required. I would recommend scoring based on rev-DNS but never blocking outright. That's plain ignorant. I say that as a mail admin and avid spam-fighter.

Re:missing rDNS?

[fl!ptop]

I would recommend scoring based on rev-DNS but never blocking outright.

i used to, but it got to the point where i was saying to myself, "all right, now what? report the offending messages to spamcop?"

over the several years i've been actively fighting spam, i have yet to receive one complaint from someone or an admin who's been blocked based on the rDNS hack. most of the complaints i receive are because of the blacklists. those are handled on a case-by-case basis, and are usually whitelisted for 2 weeks to give the admin time to get delisted.

i realize the rfc doesn't require rDNS, but if you want me to accept mail from you, i do. my customers like not receiving spam at all, and understand (and accept) the risks i take in my efforts to fight it. if i didn't have their blessing, i wouldn't do it.

Re:missing rDNS?

[Mr. Roadkill]

any mail server that doesn't have an rDNS lookup, in this day and age, is imho not worth accepting messages from.

Oh, I agree wholeheartedly. So do AOL. So do a whole lot of other places.

The problem is, I work for a University. I can't do it. There are too many stupid admins out there, and they're ALL working for sites that are considered important "offshore partners". Hell, I have enough trouble dealing with offshore agents whose outbound mailservers are in dynamic dial-up ranges - I kid you not. I'd dearly love to tempfail any message from a server that doesn't have a PTR until it times out and bounces to the sender... unfortunately, it's not gonna happen because I like working where I do.

I block between one and two hundred thousand messages per day with a half-dozen RBLs for hard-rejects (with some local mitigating whitelistings, of course), greet_pause, SA blacklist and content checks and customised scores (we reject at 15, because Chinese webmail often accumulates 9 to 13 points just by being sent...) and the SANE Security custom ClamAV signatures. We typically accept between 20 and 30 thousand messages per day, and typically get between four and a dozen reports of blocked messages PER MONTH that the senders thought should have got through. Lately, around half of those "false-postives" are due to SPF, so they don't really count. This keeps spam down to a minor distraction for most users, and costs nothing apart from some bandwidth and the electicity for the servers and a little bit of admin time. We'd probably be looking at a quarter million to a million dollars to begin with and a quarter million per year thereafter if we had to get a commercial solution at least as effective as what we've got right now, and I just don't see that happening.

Ouch.

[Deagol]

I used to use this under postfix at a department of a large university, and, later, at a small software company. I was constantly *amazed* at the lack of "correct" DNS configuration out there. Rejecting SMTP connections based on the lack of rDNS does indeed block a TON of spam, but also results in much gnashing of teeth and pulling of hair for the admin who uses it.

For a site with low, static email traffic, this is a great method. Otherwise, I wouldn't wish the resulting pain on anyone.

Now... if I could selectively gray-list such hosts, then that may help a lot.

Re:Ouch.

[fl!ptop]

I was constantly *amazed* at the lack of "correct" DNS configuration out there

i guess i should clarify - any rejected email is not simply sent to /dev/null, but is returned with an explanation that's unique to the tool used to reject it. the rDNS hack has 3 standard return error statements. wouldn't any reputable sysadmin would *want* to know that his/her mail server does not have a properly configured zone file?

Re:Ouch.

[Akatosh]

Checking for reverse dns is alright, checking to see if the forward dns works for said reverse dns makes admins cry because sooooo many people have 'reverse dns' that resolves to a hostname that points somewhere else, or doesn't exist, etc. In the sendmail hack given, you have three conditions near the bottom, FAIL, TEMP and FORGED. It's FORGED that causes the headaches (this verifies matching reverse/forward dns). Just remove that line and it will only verify that reverse dns exists.

Re:Ouch.

[Deagol]

You would think people would act rationally and try to figure out things, but that's not usually the case. Usually the sender would simply see an error message, often repeated after a few attempts to email the same person, then contact the recipient by other means to tell them there was a problem on our end. They rarely (if ever) actually read the error message itself, which, though terse, should have been enough to clue them in that it was on their end. The sender rarely consulted w/ their own sysadmin/isp/whatever before bugging *us* first.

No, I didn't just route these to the bit bucket. However, the overhead of handling complaints was unbearable so we just gave it up.

Spamhaus and abuseat.org

[AaronW]

I have had very good luck using Spamhaus and cbl.abuseat.org. I use it to outright block spam and have never had a problem with legitimate email. I go one step further, however, and block several countries. I don't know anybody in those countries, like China, Russia and Nigeria, so I just block them entirely. That has also made a huge difference.

-Aaron

Re:Spamhaus and abuseat.org

[Bandman]

Seconded. I use cbl.abuseat.org, and it dropped my spam-in-inbox a huge amount. It's hard to quantify exactly, since people have their own spam filters as well, but everyone in my organization has remarked on its success.

I don't block, but I still use DNSBL

[TechwoIf]

I use the list to score how long e-mail is greylisted and scoring in spamassassam. DNSBL are notorious for being political and having false positives. So a scoring system works better. Low scores for the worst offenders and higher spam scores for the better DNSBL.

Fake MX Records any good?

[rufo]

Anyone have any experience with fake MX records?

I find the idea sort of intriguing, but I have doubts that it'll work for long in the ever-escalating arms race of spam...

Re:Fake MX Records any good?

[DavidTC]

I've done it. There were no real problems with it. I eventually removed the first record because there were servers out there waiting five minutes before moving on, and some people wanted their email instantly. (Some people are stupid.) I still have the last completely pointless MX record, but I have no idea how much spam it's deflecting.

Something I've considered doing is having two MX records that both attach to the same graylist. So that legit senders will connect to the first one, get a temp error, move on the next one, and get through there. Most mail server go through the entire MX list in order with a very small or no delay, and then pause for 30 minutes or more before starting over, so doing that would reduce the delay quite a lot.

(I only run graylisting on dynamic IPs or connections from Windows boxes, if you're trying to reconcile that with 'people wanted their email instantly'.)

This, of course, could be combined with fake MX records. A fake one at each end, two real ones in the middle.

However, there are MX-preference compliant spammers out there (Otherwise I'd have gotten no spam during the experiment.) and that would let them completely short-circuit my graylisting. With a 30 minute delay, often they don't come back, and, when they are, they are now in blacklists.

Fun tip: Simply moving the mail server to another IP, results in a 10% decrease or so for a time, because when spammers hijack a box, they don't give it a list of email addresses. They give it a list of email addresses and IPs to connect to, the hijacked box doesn't do the lookups. They do, often long in advance.

And some of this lookup software doesn't believe in MX records, and always gives said lists the IP of the A record of the domain. So simply moving the mail server off the same IP as the A record will reduce spam by, in my estimates, 5% permanently, and putting an invalid MX record first and last will do about the same thing, because a lot of software only looks up one to pass on.

I've always wondered if being clever enough with DNS records and enough IPs, and leaping around randomly with a short enough TTL on the MX domain, you could break all spamming software. I'm sure there's some reason this is a very bad idea, but it's still funny to think about. It's not like it would add a lot of overhead...you just need to change the one A record the MX is pointing at, like currentsmtp.example.com, all the other A and MX stuff could get cached like normal. It wouldn't be hard, you'd just need some sort of tiny DNS server (They have SQL-based one right?), a cron job, some clever firewall rules that 'moved' a listening port from IP to IP, leaving old IPs working long enough to catch all legit people, and a class-C network. You wouldn't even have to touch the mail server.

(Let's see if this last paragraph ever ends up being used as prior art in a patent dispute. If so, please drop me a line.)

Re:Fake MX Records any good?

[totally bogus dude]

I only run graylisting on dynamic IPs or connections from Windows boxes

What do you use to determine if the sender is a Windows box?

Re:Fake MX Records any good?

[DavidTC]

maRBL is a postfix policy server that hooks to, of all things, an Amavisd daemon that runs p0f where it talks to a socket. A policy server is basically something that postfix stays connected to and passes information about connections to, running before it gets the email.

You have to go get the stupid amavis daemon even if you don't run amavisd. Why p0f has absolutely no daemon support, and no one's apparently ever written a frontend to daemonize it except for the amavis people, is completely unknown to me, considering that 99% of the use by other applications would be 'I have this port open, please constantly keep track of each connecting IP on that port and inform me what OS it is'. You'd think that would be builtin in some manner, or at least some third party would have a tool to make that easy, but, no, I have to track down a perl script from a amavis distribution that simply runs p0f and hooks it to a unix socket. (Having to do that somewhat annoyed me, if you can't tell.)

I've set maRBL to, instead of returning a DUNNO or REJECT, it returns is_windows or is_not_windows, which are added restrictions I've set up that postfix then jumps to. (That sentence will make no sense if you don't use postfix.) The is_not_windows restriction then does various dynamic IP checks, oddly enough using maRBL again, although via a different port, which is a different 'policy server'. (Although it's the same process.)

Sendmail can also do it, I've seen .cf files that set it up. (Sendmail can, and will, do anything, even if you don't particularly need or want it to. Even if you desperately want it to stop.)

I don't know about any other servers.

Oh, and amavisd can apparently do it. Or amavisd-new, or one of those things. Whichever one maRBL says you need the script from.

Re:Fake MX Records any good?

[totally bogus dude]

Thanks, very useful info. I might look into setting that up. I do use Postfix by the way, so it all makes perfect sense. We currently use greylisting across the board, which is pretty effective at stopping the zombies but the delays in receiving emails are sometimes irritating. It would be interesting to see if only greylisting Windows senders is still effective.

Re:APEWS/SPEWS

[91degrees]

Actually, I don't think APEWS really know what they are. The proponents seem to be unable to decide whether its punative or simply a blocking list, but remember, most of the children on NANAE have nothing to do with it. The official statement is just that it's a list to use as you wish.

I like a multi-layer approach myself

[bconway]

Checking the logs from my domain last night...

Spam blocking by site:
zen.spamhaus.org: 314
dnsbl.sorbs.net: 28
bl.spamcop.net: 40
psbl.surriel.com: 24

Not bad a for a single-user domain.

Re:I like a multi-layer approach myself

[gad_zuki!]

>dnsbl.sorbs.net: 28

Yeah, and those 28 will never be removed ,regardless of false positives. SORBS wants 50 dollars per delisting. How this is different from a criminal protection racket is beyond me.

Project Honey Pot's Http:BL

[InvisiBill]

http://www.projecthoneypot.org/httpbl.php

What Is http:BL?

Http:BL is a system that allows website administrators to take advantage of the data generated by Project Honey Pot in order to keep suspicious and malicious web robots off their sites. Project Honey Pot tracks harvesters, comment spammers, and other suspicious visitors to websites. Http:BL makes this data available to any member of Project Honey Pot in an easy and efficient way.

There are plugins for WordPress, phpBB, and many others. Use http://www.projecthoneypot.org?rf=32167 if you want to give me some credit when you register. Or not, whatever.

Re:Project Honey Pot's Http:BL

[porneL]

Project Honeypot's http:BL isn't handling dynamic IPs in any special way, so you have to be careful about these (combine with SORBS DUL and take into account age/threat that http:BL reports).

DNSBLs as Greylist, TMDA Parameters

[billstewart]

DNSBLs are a really good combination with greylisting - some of the sites you don't want to hear from are running real SMTP servers, but many of them are running zombieware or tuned-for-speed spamware, and setting your greylists to discourage them for a couple of hours instead of just five minutes can help. Also, while greylisting can block legitimate mail from dialup users, it's no problem for DSL/cable users, so you can use those DNSBLs to keep longer greylist times on those, which will also discourage zombies but work fine for home Linux users.

Another class of anti-spam tool that can benefit from greylist info is things like TMDA, those annoying autoresponders that say "I don't know who you are, so click this link/captcha/etc. to prove you're not a spammer". Humans don't like the things, but if you occasionally get mail from spam-heavy places like China, it gives them a way to get through to you that's better than just blocking, and it can be pretty low-CPU, unlike running SpamAssassin.

Re:DNSBLs as Greylist, TMDA Parameters

[cpghost]

I'm a huge fan of TMDA, but I've dropped it a few months ago, because greylisting, DNSBLs and very stringent checks at SMTP level managed to drop the amount of spam to less than 0.5% of all legit mail, while keeping the amount of false positives to a bare minimum. Almost all mails that TMDA autoresponded to were legitimate anyway after all the previous combing. Basically, there was no need to use TMDA anymore.

Actually, I was lucky, because shortly after I've stopped TMDA, my domain was hit by a huge tidal wave of spam. Amongst the few spams that still managed to sneak in, there were at least 3 fake addresses whom I know as being spam traps for DNSBLs. Had TMDA auto-responded to those, my subnet would have been immediately listed there. Clearing it wouldn't have been a problem, just a (well-deserved) hassle.

Use the right type

[91degrees]

There are two reasons for a blacklist. Reason 1 is simply to identify probable spam sources. Reason 2 is political. It's a boycott of certain organisations whose policies the maintainer decides are reprehensible. Make sure you use the right sort. If you agree with the political motivations of the maintainer, use the second type by all means but make sure you know the reason things are being blocked.

The problem with several DNSBLs is that they are the second type masquerading as the first type. Since most probable spam sources correspond well to those organisations with reprehensible policies, they tend to be difficult to distinguish. You will often find that some otherwise legitimate emails are blocked because the ISP is also hosting a phishing website, or hosting a company involved in some sort of mail fraud. This is all well and good unless you're under the impression that the BL will block spam.

The list I currently use

[Phroggy]

FEATURE(`dnsbl', `sbl-xbl.spamhaus.org', `"550 Rejected: Your IP address has been used to send spam.  " $&{client_addr} " listed at sbl-xbl.spamhaus.org"')
FEATURE(`dnsbl', `list.dsbl.org', `"550 Rejected: Your IP address has been used to send spam.  " $&{client_addr} " listed at list.dsbl.org"')
FEATURE(`dnsbl', `cn.ascc.dnsbl.bit.nl', `"550 Rejected: Due to a high volume of spam we do not accept mail from China.  " $&{client_addr} " listed at cn.ascc.dnsbl.bit.nl"')
FEATURE(`dnsbl', `korea.services.net', `"550 Rejected: Due to a high volume of spam we do not accept mail from Korea.  " $&{client_addr} " listed at korea.services.net"')
FEATURE(`dnsbl', `web.dnsbl.sorbs.net', `"550 Rejected: Your IP address is known to host a web site containing security holes which can be used to send spam.  " $&{client_addr} " listed at web.dnsbl.sorbs.net"')
FEATURE(`dnsbl', `spam.dnsrbl.net', `"550 Rejected: Your IP address has been used to send spam.  "$&{client_addr} " listed at spam.dnsrbl.net"')

Re:The list I currently use

[Phroggy]

You may want to replace:

sbl-xbl.spamhaus.org

with:

zen.spamhaus.org Nope, I got too many false positives with that.

Re:The perfect anti-spam tool

[dmpyron]

I misconfigured my catchall on one of my domains and received about 500 bounces in a matter of 5 or 6 minutes. All from fake names in my domain. I'm pissed enough when one of my real names gets forged, so these fake names steamed me. I checked all of my domains after that.

No Blocking

[rawg]

I have found that my customers don't want me to block spam. I would get complaints from customers not getting their mail from hosts that are being blocked. So I use Spamassassin to tag the spam and filters on my clients to delete it. Yep, I have to process all that spam and yep the customers have to download all that spam, but I don't get any phone calls anymore.

Re:No Blocking

[The Cisco Kid]

And doing so is entirely your choice, and no one other than your boss (unless you are the boss) has any business telling you to do otherwise.

I am curious though, if you (or your boss) are happy with the loss of profits involved due to increased bandwidth and server resource costs that go with that choice (Or, if you've raised your prices to offset that, if your customers are happy with that).

DNSBLs to feed other tools

[billstewart]

Most DNSBLs have problems, and there are few that I'd trust absolutely, though Spamhaus runs a tight enough shop that I'd trust it. But DNSBLs can be used effectively to augment other tools:

• SpamAssassin weights - most of the DNSBLs are worth a couple of points of SpamAssassin weight; even rabid ones like SORBS can give you some information, and the country-specific ones are also useful here (e.g. mail from China had better not look spammy at all.)
  
• Greylist Augmentation - The big value of DNSBLs is that you can reject mail from the SMTP headers without needing to receive the message body and grind it through CPU-instensive content filtering. But Greylists also do this, and some people have been using DNSBLs to tune their greylists (e.g. if it's on the DNSBL, then tell the sender to call back in an hour instead of 5 minutes.) Among other things, that gives you a way to use the lists of Dynamic-Address broadband users - the home Linux servers will call you back, the zombies won't, so the list gives you information which you might otherwise have to ignore. And country-code DNSBLs can also get forced to wait an extra hour for spammy places that you don't get much mail from.
  
• 
  TMDA Autoresponders - One of the most annoying and effective anti-spam tools is autoresponders that say "I don't recognize your address - respond to this mail and prove you're a human". You could integrate this with a DNSBL - if the mail's not whitelisted, and it's on some DNSBLs, then maybe it gets a TMDA test instead of bit-bucket. It's lower CPU than SpamAssassin.
  
• DNSBL integration with DNS Servers? - One of my pet projects for when I get some copious spare time is to munge a DNS server to check blacklists/whitelists. Trusted or non-blacklisted sites get the MX record for the good mailserver, non-blacklisted sites get the MX record for the heavily-filtered mailserver that occasionally overloads the CPU, blacklisted sites get the MX record for the teergrube or 127.0.0.1. It's certainly not foolproof - many systems are likely to check their ISP's DNS cache before hitting your DNS directly, and if spammers want to do a set of DNS queries from a clean server they could - but at least at the corporate-email level (i.e. where you can afford multiple mail servers) it gives you a way to avoid having your mail server lose mail from legitimate sources because it's overloaded with SpamAssassin CPU load.
  

  
  I originally thought of this back when Open Relays were the popular spam threat - if you get a DNS MX request from an open relay, tell them that the IP address for spambait.yourdomain.com is some other open relay's address. That would let them spend their time sending mail to each other. But spammers moved on to open proxies and then zombies, so that opportunity went away.
  

• You can think of other things.
  

Re:DNSBLs to feed other tools

[pw201]

TMDA and the like are spammers themselves: they send lots of identical messages to people who haven't asked for them, namely the people whose addresses are used fraudulently in the From lines of spam emails.

Greylist to save CPU before using DNSBLs, SpamAss

[billstewart]

SpamAssassin and commercial solutions use a lot of CPU, but there are other low-CPU ways to defend yourself besides DNSBLs. Greylists are a popular one - mail from unknown mailserver addresses gets told to go away and try again in 5 minutes or an hour or whatever. They're currently effective against most zombies, and inherently effective against some of the stolen-address-space attacks, and because they're not permanently blocking mail, false positives aren't a problem. It works at the SMTP-header level, so you don't need to accept the message body or process it. It's not a perfect defense, but I've seen people reporting 80-90% of the spam goes away for minimal CPU cost.

There are also people combining greylists with DNSBLs - senders from blacklisted addresses get told to wait much longer than non-blacklisted addresses, or they get told to wait and non-blacklisted addresses don't.

Even if you're going to also reject on DNSBLs, this'll let you be less aggressive about it, e.g. use SpamHaus's list of known big spammers, then greylist, and use the other DNSBLs only as SpamAssassin weight, or greylist first, then use Spamhaus on the people who called back; you could also do some analysis to see how many of the greylist rejects are covered by people from which RBLs.

To truly make blacklists useful...

[IGnatius T Foobar]

To truly make blacklists useful, you've got to filter not only mail coming from IP addresses listed within them, but also mail containing URL's that resolve to IP addresses listed within them. Once you implement this, you will see a *dramatic* drop in spam. Spammers can move their delivery systems from place to place, but at some point they've got to advertise a web site. Yes, the stock spam will still get through, as well as some others, but over the years I've spent administering (and developing) email systems, this was the single most effective thing I've ever seen.

Happily, these tests are already present in SpamAssassin; they're just not scored highly enough. Here's a nice easy way to fix that. Edit your /etc/mail/spamassassin/local.cf and add these lines:

# High score for URL's whose IP addresses are in rbl
score URIBL_AB_SURBL 10
score URIBL_JP_SURBL 10
score URIBL_OB_SURBL 10
score URIBL_PH_SURBL 10
score URIBL_SBL 10
score URIBL_SC_SURBL 10
score URIBL_WS_SURBL 10

Restart spamd, and you will immediately see a large drop in spam.

APEWS/SPEWS

[valloned]

I actually sent some feedback to SANS when I saw the problems they were having with these idiots and told them about my experiences with them (I'm sure I wasn't the only one)

My company has been getting some bounceback emails from certain clients who rely too heavily on this blacklist. I go to their site and find out that not only are listing my companies network, but a large portion of MCI's commercial data circuits as well. It appears they simply gather these entries from other sources and then increase the scope of the listing to include a stupidly large number of IPs (mainly the entire upstream provider). As SANS noted, they were blocking just about the entire AT&T network. They don't identify who they are, they have no method of being contacted, and they are incredibly careless. Anyone who relies on an APEWS or SPEWS blocklist for anything will be very sorry they did. They are beyond useless. There are some reputable blocklists that, when used correctly and with a combinatino of other filtering methods, can provide positive results. These idiots are not among that group.

dnsbl's + other means for spam abatement to use

[a2gsg]

here's the bl's that i am using with sendmail that would go into your siteconfig.mc file -- that through trial and error -- i have found have zero false positive hit rate... n.b. that the XXX.r.mail-abuse.com (RBL) & XXX.q.mail-abuse.com (QIL) bl's require that you to have a subscription to Trend Micro Advanced Email Reputation Services at http://us.trendmicro.com/us/products/enterprise/ne twork-reputation-services/index.html -- you can get a free trial at https://nssg.trendmicro.com/download/trial/trial-s ervices.php?id=66 --
make sure you select "Email Reputation Services, Advanced". you would then replace the "XXX" in the below with the activation code they would send you:

FEATURE(dnsbl, `XXX.r.mail-abuse.com.', `"550 Mail from " $&{client_addr} " BLOCKED/RBL; see http://www.mail-abuse.com/cgi-bin/lookup?ip_addres s=" $&{client_addr}')

FEATURE(dnsbl, `zen.spamhaus.org.', `"550 Mail from " $&{client_addr} " BLOCKED/ZEN; see http://www.spamhaus.org/query/bl?ip=" $&{client_addr}')

FEATURE(dnsbl, `bhnc.njabl.org.', `"550 Mail from " $&{client_addr} " BLOCKED/BHNC; see http://www.njabl.org/lookup?" $&{client_addr}')

FEATURE(dnsbl, `bl.spamcop.net.', `"550 Mail from " $&{client_addr} " BLOCKED/COP; see http://www.spamcop.net/w3m?action=checkblock&ip=" $&{client_addr}')

FEATURE(dnsbl, `list.dsbl.org.', `"550 Mail from " $&{client_addr} " BLOCKED/DSBL; see http://www.dsbl.org/listing?" $&{client_addr}')

FEATURE(rhsbl, `dsn.rfc-ignorant.org.',`"550 Mail from domain " $`'&{RHS} " BLOCKED/DSN; MX of domain dose not accept bounces in violation of RFC 821/2505/2821, see http://www.rfc-ignorant.org/tools/lookup.php?domai n=" $`'&{RHS}')

FEATURE(rhsbl, `bogusmx.rfc-ignorant.org.',`"550 Mail from domain " $`'&{RHS} " BLOCKED/BMX; MX of domain contains bogus address information in violation of RFC 1035/3330, see http://www.rfc-ignorant.org/tools/lookup.php?domai n=" $`'&{RHS}')

FEATURE(dnsbl, `XXX.q.mail-abuse.com.', `"450 Mail from " $&{client_addr} " BLOCKED/QIL; see http://www.mail-abuse.com/cgi-bin/lookup?ip_addres s=" $&{client_addr}')

FEATURE(dnsbl, `safe.dnsbl.sorbs.net.', `"450 Mail from " $&{client_addr} " BLOCKED/SAFE; see http://www.dnsbl.sorbs.net/lookup.shtml?" $&{client_addr}')

i also use the http://hcpnet.free.fr/milter-greylist greylisting package as well as spamassassin with some custom score tweaks available at http://iconia.com/user_prefs. all this keeps my mailbox as well as other users at a college radio station and a commercial asp with lots of public email addresses on their respective websites relatively spam free.

respectfully submitted,
geoff goodfellow

Re:APEWS/SPEWS

[DavidTC]

You're not being blocked. MCI's being blocked.

APEWS does not block spam, or even spammers. APEWS blocks ISPs that allow spammers, and MCI is (was?) one of those.

And APEWS doesn't collect data from anywhere. They run spamtraps. Mail get sent to them to a spamtrap, they start blocking. As mail continues to come in, they continue blocking and add more and more IPs, until they encompass entire companies, and then companies that prove those companies connectivity, and then companies that provide those companies connectivity, until the spammer, or the company hosting them, or someone in the chain, is removed from the internet. (An interesting open question is where this stops, upsteams. No one knows if it would cross the huge peering agreements among the big boys, but luckily it's not gotten to that point.)

And there's absolutely no reason for them to want contact from you. You are not the problem, you cannot solve the problem, you are unrelated to the problem. MCI is (was?) the problem, for continuing to host someone. They are the only ones who can solve it, so it's recommended you contact them.

is spamhaus a dnsrbl?

[sholdowa]

I'd always thought that it was an IP based rbl - blocking the ip address, and not the domain name. As such, it's identifying servers ( or bots, whatever ) that are behaving badly. Which sounds good to me.

Re:APEWS/SPEWS

[valloned]

I'm assuming you are part of APEWS since you pretty much recited verbatim what's in their rather useless FAQ? While they may run spamtraps, it has already been proven by SANS that they DO COLLECT DATA from other sources and use that to construct their blocklists. I would go read the SANS diary post. You can not defend them by claiming they use only spam traps when it has been proven they do not. If (and that's a big if) someone on an MCI data circuit (or any hot for that matter) was the problem, there is nothing stopping APEWS from listing only the IP block registered to the offender, as that info is readily available from ARIN. To list the entire MCI block, comprised of thousands of companies, is beyond stupid. As SANS pointed out, they are also listing the entire AT&T network. They took a /32 listings SANS had at http://isc.incidents.org/ipsascii.html and rolled it up to /17s. You think blocking 32,000 hosts in responce to a single IP address being listed somewhere else is helping anything? MCI is not the problem, as you say. APEWS and people who rely on it are the problem. The list is not the least bit accurate and whoever runs it doesn't seem to want to take responsibility for their bad practices. It's garbage, period.

-1, Bullshit

[mvdwege]

I see the spammers are out in force to day, to see this modded up to +5.

SORBS does not ask for donations for a mere delisting. All you have to do is submit a request to their automated request system, and you will be delisted. I have actually done this for a customer of ours who got a false positive listing. 48 hours later, listing gone, and most of that was propagation delay.

Mart

Re:-1, Bullshit

[Lost Race]

Even when ISPs do pay to get off the list, SORBS doesn't get any of the money. What a racket!

Wrong: A good RBL is worth its weight in gold.

[chathamhouse]

I would suggest that you are uninformed, and do not run a high volume mail system.

I'm responsible for a mid-sized mail system that receives an average of 10,000,000 connection requests per day. A good RBL is worth a lot to my employer.

We use Spamhaus xbl-sbl, and Trend Micro's Network Reputation Service - which is a combination of the more static RBL+ (of MAPS fame) and the highly dynamic QIL list.

Together, they drop approximately 92% of inbound connections to the SMTP server farm. This is a lot cheaper, computationally and financially, than using the lists later on in a content filtering stage. Without these RBLs, we would require ten times the CPU power to move and filter the messages that the dropped connections would undoubtedly attempt to deliver.

The RBLs allow us to provide customers with good to excellent filtering, at a tenth of the infrastructure cost that would be required without them, subscription cost to the two lists included. When we use a standard server build that runs approximately $15k/system, plus another $10k/system to rack, power, and cool it over it's lifetime (~3yrs) that's almost $450k saved over 3 years! And I'm not counting the bandwidth saved here, which is a substantial savings when buying international transit in Australia.

But the best part about the whole thing is the recorded number of complaints. I'm up to 10 in the past year. Even if the reported to unported ratio is 100:1, that's pretty excellent given the size of our customer base, and it's makeup - a lot of businesses that will complain if mail is blocked. Most problems were due to the QIL being a bit trigger-happy with listing other major Australian ISPs. No worries - it can be configured to whitelist by country, ISP, and arbitrary IP ranges. Fantastic.

Only a couple of complaints from people running mail servers behind DSL, in a residential (marked as dynamic) range. To these people I have one message: pay up to get a static (aka. business) grade service, co-locate your mail server, or get a real provider to be your mail host. Most spam comes from zombies sitting behind dynamic IP blocks - this is why they get dropped.

The final nicety from subscribing to these lists is while their support is good if you're a non-customer trying to be delisted (6-12hrs, tested prior to subscribing), their support is excellent when you're a customer. Quick to get spam evidence, quick to fix problematic listings of our systems _if the work has been done to clear the source_!

In summary:
1. spam still gets through the system. Now seeing 3-4% of connection attempts resulting in a delivery to a customer mailbox. Without the two RBLs on the front end, much more spam is seen because content filters are far from perfect.
2. Contrary to your assertion, list sharing is quite low: about 50% of the addresses are common between the two lists. In other words, we get about 60% connections dropped per list, for an aggregate of that 92% figure. If you assume that some spam sources are prolific, it indicates quite a bit of novel collection on the part of each.
3. A well run list isn't run by a schmuck. It's run by a company, with customers who pay it to do a good job and err towards reducing false positives. If you want schmuck, use SORBS.

Re:Wrong: A good RBL is worth its weight in gold.

[Bronster]

I was going to see who you are (since I'm a sysadmin for an Australian email provider as well) but your website links to a photo which links to a redirection to an HTTPS SVN repository that tells me to get fucked in no uncertain terms. Hmm.

Re:APEWS/SPEWS

[The Cisco Kid]

Why is MCI not the problem? If MCI continues to provide Internet connectivity to known spammers, then yes, MCI *is* part of the problem. The object of a list like APEWS (and before them, SPEWS) is to remove the spammers from the Internet, not just to play whack-a-mole with individual IP blocks. I'll even take it one step further - if you knowingly and intentionally continue to pay an ISP that knowingly and intentionally providers spammers Internet connectivity, then you too are part of the problem, and I would be quite happy to refuse mail from you.

If you dont like how a particular DNSBL works, then dont use it, no one is forcing you to. Others that do like how it works may choose to use it, and don't have to listen to your arguments about it. The *senders* of mail dont get to choose what lists apply to them, the *recipient* does. (and by recipient, I mean the owner of the server that receives the email, or whatever admin they might delegate that authority to, not necessarily any individual mailbox user - however that would be a matter of the contract between the individual mailbox users and the owner of the server and would be of no concern to some random sender of email)

Other lists do work the way you describe (only listing the actual spammer IP's), although since ISP's move spammers around (and once an ISP is know to be 'friendly' to spammers more spammers sign up with them), they arent terribly effective.

Re:APEWS/SPEWS

[DavidTC]

You mean I should look at the diary post...the one in the article...that doesn't mention APEWS at all? Why would I do that?

As for the SANS diary that you did not provide any link to, but I nevertheless managed to find, that claims that APEWS is taking data from SANS...there is no 'proof' there whatsoever, just an assertation. (And it would be damn easy to prove it, by putting, for example, some invalid IPs on the list and see if they get listed by APEWS.) There's absolutely no evidence at all.

The fact that APEWS apparently listed an incident report from SANS as a reason for blocking is due to the fact that APEWS does not expose its spamtraps, so all incident reports are taken from other people. No, you can't claim copyright on posting someone else's spam, and SANS is an ass for even attempting it, because the whole damn point of publicly posting spam is for other people to cite it as evidence of spamming behavior. But there are, yet again, absolutely no links or evidence provided that even what I just said actually happened, much less that something unethical happened. (Nor can you, to cover all bases, claim copyright on factual lists of IP addresses.)

I've decided to sum the discussion so far:

You: APEWS blocks too much, including address that aren't spamming that are hooked to the same ISP as spammers.

Me: APEWS doesn't say it blocks addresses that are spamming, it says it blocks ISPs that allow spammers.

You: Ha, you're obviously working for APEWS because that's in the FAQ and everyone knows that! Plus, I'm going to continue to whine they don't behave the way that I want them to behave instead of the way they clearly state they behave.

My response:

The list is near 100% accurate for a list of spam-allowing ISPs. It is an incredibly bad idea to actually block on such a list. When I used their predecessor, SPEWS, I used it as part of a scoring system, like any responsible person would, to assign a specific negative weight to email from spam-allowing ISPs, which, combined with other indicators, allowed me to detect spam. And I resent your idea that merely because morons use APEWS incorrectly that APEWS is being 'irresponsible'.

You'll notice that APEWS doesn't recommend blocking based on it or provide any directions whatsoever to do so.

You know, there are DNSBLs out there that list all IPs in a country, one for each country. Is that 'irresponsible', or is it, as I call it, 'useful information that allows me to build an anti-spam system', despite the whining of people who tried to email other fools that decided to block all of Mexico?

RBL == censorship

[mike_sucks]

Using an RBL lets an untrustworthy third party censor email being sent to your users.

Do not use one. /Mike

Re:SpamCop

[Alan Doherty]

good for them

you obviously don't realize spammers don't spam through thier own isp's
{unless total morons[who arn't the souce of the problem]}

they use their internet connections to remotely operate their bot-nets and have them send the spam/harvest the addresses etc

so yes by not disconnecting the criminal they ARE enabling the crime

same as by not cancelling spamvertised websites / e-mail addresses / domains used in spam/phishes/419 fraud etc they also are enabling the spammer to profit from illegal activity

My preferred DNSBLs

[mespinola]

This is a timely article for me. I have been infrequently researching DNSBLs for a few years now, and I am almost finished documenting my findings here: http://www.asspsmtp.org/wiki/DNSBL Its a wiki page for an anti-spam filter that I help develop and maintain this web site for - but the article is completely neutral POV and devoid of any product references. Its my assessment of the DNSBLs that I use and recommend. I would appreciate any thoughtful feed back on the article or recommendations for anything I may have missed. My username on the site is "ME2". Thanks!