Slashdot Mirror


Are SPAM Blacklists Unreasonable?

rlsnyder asks: "I'm the inadvertant co-administrator of e-mail a for company that relies pretty heavily on it for daily business (e.g. sending confirmations of financial transactions). At one point in the not-too-distant past, our server was an open relay. I admit I'm a sinner for letting it happen, and I'm ready to do my pennance. Given the relatively low volume of mail our server moved that did not originate from inside, I doubt I was a major contributor to the world of SPAM. In any event, we've been blacklisted on a number of sites. Some lists have reasonable policies, and we've since been removed. Other places are a little more arbitrary as to removal policies, and although I can prove we're not a relay, we're still listed." While I approve of the basic concept of SPAM Blacklists, there are dozens of SPAM blacklists out there who are real keen on adding open relays to the list, but not so keen on taking rehabilitated hosts out. I would posit that SPAM blacklists that are not properly maintained are a part of the problem, not the solution. What are your thoughts on the subject?

rlsynder continues: "Am I way off base here, or is this self-appointed mail police thing going in the wrong direction? Given that I can't reliably deliver e-mail to a number of places due to being blocked, I've got a big exposure. Is this making spam less of a problem, or are we trading one problem (SPAM) for another (the reliablility of proper maintenance of SPAM Blacklists)?

I could draw a bunch of analogies here, but isn't the bottom line that no one owns the internet e-mail system? I realize no one makes ISP's subscribe to the blacklists, but basically, I'm trying to move data from one point to another, and some machines in the middle are discriminating against my data because a corrected, perfectly legal system configuration error. How is this helping? Has SPAM really decreased universally thanks to these lists?"

25 of 619 comments (clear)

  1. Real Pain by Tadrith · · Score: 5, Insightful

    The company I work for had the same problem. As a result, we ended up having trouble getting e-mail to some of our customers. Thankfully, it was easy to get ourselves removed, but I think if people are going to use blacklists, they should also take the responsibility of keeping them maintained, both in additions and removals.

  2. Subscribing to blacklists did not help me. by Dick+Click · · Score: 5, Interesting

    When I used to manage a mail server, I was asked to filer based on orbs. Not did this in no significant way limit the amount of spam entering the system, it became a huge administrative headache. Eventually, we stopped using the lists. I am sure there are likely better lists, but I simply prefer creating my own list, based on investigation into what's coming in.

    1. Re:Subscribing to blacklists did not help me. by diamondc · · Score: 4, Insightful

      We use ordb and orbz here at work. Over a day or so it rejected about 500 emails.

      Then we blocked all mail from mail servers who's IP numbers don't resolve. Now we have cut down on spam dramatically.. our root@ email account has gone from 200 spam emails a day to about 10

      --
      "I keep looking in the want-ads under 'revolutionary' but there don't seem to be any listings.. "
  3. Its more of a pain in the neck by tkrotchko · · Score: 4, Informative

    I like the idea of something like MAPS-RBL, but I think many of them are bad hacks put together by guys who take the spam thing as a holy crusade. I don't really have a problem with that, its a free country, you do what you want.

    However I fault ISPs for using them without understanding their policies. Many ISPs use these small-time black-holes because they don't want to use MAPRBL (I assume its a money thing at this point). And if you get listed, how do you know that you're listed? You don't until somebody calls somebody and says "I can't get mail through to you". There needs to be a better way.

    And some sites, its not worth getting delisted. "www.joes.antispam.site.com" isn't worth the effort one way or the other.

    --
    You were mistaken. Which is odd, since memory shouldn't be a problem for you
    1. Re:Its more of a pain in the neck by crucini · · Score: 4, Informative
      Many ISPs use these small-time black-holes because they
      don't want to use MAPRBL (I assume its a money thing at this
      point).

      I don't think it's only a money thing. MAPS is almost useless - they don't list spammers until they've tried to "educate" them. I've noticed that servers sending me spam are never on MAPS. But the fact that they're charging doesn't help.
      And if you get listed, how do you know that you're listed? You don't until somebody calls somebody and says "I can't get mail through to you". There needs to be a better way.

      You generally know that you're listed because some of your outbound mail bounces with a message explaining that you are listed and giving a URL for further info. Are you saying that you've had outbound mail bounced due to a spam list and there was no indication of the reason? I realize this is theoretically possible, but I don't understand why someone would set up a mail server that way.
  4. I've been e-mailing the admins of those lists,... by 5.25"+Floppy · · Score: 4, Funny

    ... but dammit, they just don't seem to be getting my e-mail! I'm going to start having all my friends send them a few mails as well... *sigh*

  5. Mail servers are private property by Tyrall · · Score: 5, Insightful
    From the article: I could draw a bunch of analogies here, but isn't the bottom line that no one owns the internet e-mail system?
    This is a fallacy that continues to be propagated. I own my own mail server. The company I work for owns its mail servers. We can both decide who we want to allow to send mail to our users.

    At work, we use two open relay lists; ORDB and ORBZ. Nobody forces us to use them; it's our server cluster, and our choice.
    The reason we use those two systems, however, is due to the reasons pointed out in the article. Some blacklists are far too easy to get onto, or hosts are arbitrarily added by humans. The only way to get onto either of those lists is to be an open relay. The only way off is to be automatically retested and found to not be an open relay.

  6. ORDB.org by paranoidia · · Score: 4, Informative

    ordb.org is a great site for this. They are very professional with both addition of servers, and subtraction of them. My mail server was an open relay for a time till I got an email from them saying that I was blacklisted. I quickly fixed the server, and submitted that my site be checked again, the next day I was taken off their lists, very easy. They run about 20 tests connecting to your server and sending e-mails for the most common way of sending spam. Also, as they say in their faq that they reload their lists every hour to get servers off it quickly. Well done!

  7. Blacklist sites by schon · · Score: 5, Interesting

    OK, you've fixed your mail relay(s)..

    This is a good thing - and what every blacklist's ultimate goal is.

    Speaking as a mail server admin, I'd be interested to know which lists are not removing you - so that I can make sure I'm not using them.

    Seriously - letting people know about this is the best way to get what you want. If your site is not a relay, any blacklist maintainer is doing their users a disservice by listing you.

    As a mail admin, I'd want to know.

    Alternatively, you could do the American thing and threaten a lawsuit - most blacklist operators are immune from libel charges because they're just listing people who operate open relays (truth is defense against libel) - if you're not an open relay, then you've got a good case for libel: they're deliberately publishing false information to hurt your business.

  8. That's a self-solving problem (mostly) by devphil · · Score: 5, Insightful


    Yep, that's the root of the problem: there are a number of for-free blacklists out there which are professionally managed. Those are the ones that should be used.

    And as long as we publicly point out the blacklists that are being poorly run, people will stop using them, and switch to the good ones (like RBL, RSS, DUL, ORDB). The solution is not to ban or otherwise stop using blacklists, the solution is simply to (vocally) promote the ones which stay on top of the problem.

    --
    You cannot apply a technological solution to a sociological problem. (Edwards' Law)
  9. Shout out for SpamAssassin by dietz · · Score: 5, Informative

    I'd just like to give some props for SpamAssassin.

    If you haven't heard of it, it's an elegant system that assigns a weight to each email message based on hundreds of different tests, and if the email scores over 5 (configurable), it is marked as spam.

    One of the nice things about it that is it uses most of the email blacklists, but they're only worth ~2 points, so being in a blacklist alone isn't enough to kill a message. That's good for those blacklists that throw far too many people in that don't belong (osirusoft). It also uses razor, but that is only worth three points, so if someone is piping bugtraq to razor-report (that happened for a while) you won't lose all that email.

    There's a really interesting set of tests (it's fun to read them) each with an obscure set of points including:
    HTML with a non-white bgcolor (1.2)
    Claims conformance to obscure spam law (1.0)
    HTML mail with no text portion (3.33)
    Various spam phrases (various points depending on how many "hits" there are)
    Subject ends in an exclamation point (0.5)

    The points have apparently been calculated using some program to give the best accuracy.

    Anyway, SpamAssassin is the best of the spam removal programs I've seen. Give it a shot!

  10. Protecting my server, thank you very much by alansz · · Score: 5, Informative
    DNS-based blacklists are not your problem. There are no more than a dozen that are really widely used (some orbs spinoffs like http://www.ordb.org and http://www.orbz.org, the MAPS ones if you're willing to pay (or can get a hobby contract) at http://www.mail-abuse.org, and the collection at http://relays.osirusoft.com that includes open relays, spamhaus, and SPEWS. All of these systems have clearly-published listing policies and are actively maintained and if you're blocked by one of them, you'll likely get out sooner or later once you're clean. (In some cases, you can have them automatically retest you). Plenty of mail admins find that using the information on these sites to protect their mail servers from spam is highly effective.

    Your problem is twofold. First, while you've cleaned up your open relay, plenty of spammers and spam-friendly hosts make the same claim and lie (Rule #1: Spammers lie). So you may have to be patient.

    More importantly, your server ip may now be sitting in hundreds of private blacklists of mail servers whose admins don't like to use the centralized lists, and just reject/blackhole spammers on their own. It is the presence of well-trusted centralized blacklist services that gives you even the hope of ever having decent communication, because without them, you'd get into a thousand tiny blacklists and never get out.

    (P.S. Note that if you're checking your status using the rblcheck tool at http://relays.osirusoft.com, it will tell you about a lot of blacklists that are not intended to be publicly used and not part of the usual osirusoft dnsbl, as well...)

  11. Re:No. Deal with it. by Anonymous Coward · · Score: 4, Insightful

    Crack house? A bit harsh considering the guy simply had an open relay which he then fixed.

    You really think this is a valid analogy? Go spend a night in one, then go back to our cushy world of sysadmin stuff.

    Didn't think so.

    I'm betting he was asked to install a server - prolly a turnkey type - did so, and watched it chug along for a good long time before someone found out it was open and started using it.

    More like finding a crackhead in your garage, eh?

    Gee, ya think maybe he missed the giant neon sticker that came with the mailserver manual that said "your box is an open relay by default. fix that. tag - you're it!" Oh, right - that's because there is no such sticker.

    If they maintain the lists, they should *maintain* them, not just treat them like a brick wall and simply pile up the addresses and leave it at that. My experience with orbz is that they don't pay attention to the people in the middle - I've been there.

    Just takes a little bit of hard work, and this guy's apparently willing to do his part.

    Lighten up and tackle the appropriate problem.

    --Jake

  12. Going to get far worse before it gets better. by Thagg · · Score: 5, Insightful

    rlsnyder asks Has SPAM really decreased universally thanks to these lists? Well, it is hard to say. Spam has increased monotonically since its inception, and it continues to grow. It is possible that blacklists have helped lower the rate of growth.

    What blacklists really do is get the attention of sysadmins, and get them to take the problem seriously. I, like rlsnyder, was victimized in the same way -- our mail server was an open relay, we forwarded some spam, and got blacklisted. It took me a week or so to get it straightened out, and in the process I learned quite a bit about the UCE problem. rlsnyder similarly has been enriched by the experience, whether he agrees to that at this point or not.

    One always has the option of sending mail from one of the many free mail systems. If your mail is blocked while your case is being reviewed, then send it from hotmail or someplace like that. That's what we did. In took about a week for the last of the spam reporting services to delist our site, and while it was inconvenient, it wasn't devastating. It won't be for rlsnyder, either, I trust.

    The big problem is that there is nothing to stop the spammers. People who relay mail through unsuspecting companies are already criminals, they will not be dissuaded by laws. The only thing that the anti-spam community can do is to try to put a finger in all 2^32 holes in the dike, and the only way to do that is to educate people. The blacklists are that education program

    thad

    --
    I love Mondays. On a Monday, anything is possible.
  13. Re:Automate the maintenance by Tyrall · · Score: 4, Insightful

    Nope, the usual way to do it is:
    1. Filter the open relay checker's IP.
    2. Click 'check me now'.
    3. Spam as usual.

    This is a retarded, but effective way of avoiding the automatic blacklist generators.
    You'll still get on a lot of the automatic+human checkers like MAPS' open relay list.

  14. Re:No. Deal with it. by Tackhead · · Score: 4, Insightful
    > What if it used to be a crack house, but the neighborhood cleaned up and was safe?

    A good point. That's why I'd buy SPEWS a beer.

    The system appears to be automated -- if the blocked host stops sending spam for a long enough period of time, SPEWS appears to unblock it.

    If, on the other hand, the spam continues to issue from the blocked host, SPEWS appears not to unblock it.

    From what I've read in news.admin.net-abuse.email, the length of time for which a provider remains in SPEWS appears to be proportional to the length of time the provider ignored abuse complaints.

    Contrast this with a privately-run blocklist (e.g. my "fsck it, block the /24".) I can't be bothered to check if the /24 has cleaned up. There are IP address ranges all the way back to the days of Cyberpromo that I haven't been bothered to unblock.

    The advantage of SPEWS and its ilk is that 1000 systems can be unblocked. The problem with the blocklist on my own system is that I can rarely be bothered to unblock it.

    (In crackhouse terms, SPEWS reads police blotters, and if it stops seeing crime in a certain area, allows pizza delivery. I'm the crusty old Italian guy who says "No, you can't deliver to 48th street, it's a war zone, at least, it was the last time I tried to deliver a pie there sometime in 1996!")

  15. some of thee guys are nuts by ellem · · Score: 4, Funny

    A little while ago a site I worked at was blacklisted.

    We fixed the problem that day and when we contacted the SPAM COP he wrote back to say, basically:

    All Lotus Notes Mail Servers are insecure so we're leaving you on the list. Get another mail server.

    I made achange in the Notes.INI file that made it look like I was using SendMail. And he fixed us.

    Ridiculous policy. Notes is pretty secure anyway! I wonder what this guy read...

    --
    This .sig is fake but accurate.
  16. some companies deserve it by not_anne · · Score: 4, Interesting

    My employer's corporate office email system is an open relay, so that outlying offices (like ours) can send email, and so the company can track what we're doing.

    Recently, spammers have discovered our open system and have been relaying at a furious rate (read: thousands of emails a day.) This caused *our* email to get reflected back to us most of the time, and it also got my employer's domain on several spammer blacklists. This is such a problem, that the corporate office recently switched ISPs over it.

    Now, with the new ISP, the IT guys have "cracked down on security" by banning relaying...for 1/2 the day. In the mornings we can send all the email we want (and so can the spammers), but after we all get back from lunch, no more email can be sent out. My employer is baffled why we can't get off of the blacklists, even after the move to the new ISP. I just laugh and goof off for the rest of the afternoon.

    I'm all for an appeals process of some sort in order to get off of spam blacklists, but some companies do deserve to stay there, as long as their habits and policies don't radically change.

    not_anne

    --
    My comments here are my own; I do not speak for my employer.
  17. simple solution.. by Lumpy · · Score: 4, Interesting

    a self maintaining blacklist. if you get blacklisted and then fix it, you go to a webpage that you submit that you're fixed. then the system simply uses a seperate computer that is NOT on the webpages domain and tries to relay email. if the relay happened then the blacklisted site is still blacklisted, otherwise it is automatically removed.

    Maybe 100 lines in perl to accomplish this. no real effort required.

    --
    Do not look at laser with remaining good eye.
  18. Re:It's anti-democratic ! There are other (better) by hpa · · Score: 4, Insightful

    The problem 3 is quite grave : What can you do if your mail server doesn't support anti-relay ?
    Or if you must buy another licence, or it it's opensource, but needs a new version of the OS, or things like that. OK, now all email servers support anti-relay. But this was not the case at this time.


    If so, they're right in blocking you. You're saying "oh, we're not willing to go through the trouble of cleaning up our server, to hell with anyone who gets spammed." It's exactly those sites that they're supposed to be blocked


    2. They should leave you 10-15 days to fix the problem before blocking you


    That's insane. Once you end up on a spamrelay list, you'll be the conduit for tons of spam within hours of even minutes. 10-15 days is an eternity in that respect.

  19. Re:No. Deal with it. by JordoCrouse · · Score: 4, Insightful

    Don't like living in a crack house? Move.

    What about the people living next door to the crack house? Should they not be able to get a pizza as well? How about the good houses that get anonymously accused of being crack houses?

    The fact of the matter is, for every legimiate spammer on the list (even the well administrated ones), there is another placed there unfairly.

    In the three weeks preceding the much awaited dumping of ORBS, we started dropping mail from 4 different valid mailing lists and 1 valid business (it was a brick and mortar business - no web presence, just an e-mail server). One of the lists was LKML (and I have no idea why it was on the list), and the other three had the misfortune of being on the same web hosting service as a spammer.

    The brick and mortar was on the list because of an open relay (which was a good reason to be listed), however once it was closed, they were not allowed to be removed, though their level of e-mail is about 20 - 30 message a day, and they have never send a spam in their existance.

    The problem is that we are all living in close proximity here - legit businesses are only a few digits away from spammers (just like the real world). And the knee jerk reaction that most sysadmins take in dealing with the situation is similar in nature to burning half your mail daily because the postmark is similar to a known junk mailer. And burning is a reasonable analogy, because blocked emails don't get archived or analyzed, they get tossed, lock stock and barrel.

    Its so easy for a sysadmin to install a blacklist and never worry about it again (unless of course, *he* starts losing messages).

    The price for having a spam free existance is to constantly monitor and evaluate the system, not to light a match and walk away.

    --
    Do you have Linux and a DotPal? Click here now!
  20. It's democracy and freedom in action. by fmaxwell · · Score: 4, Insightful

    I don't accept ORBS having decided what's permitted and what's not !

    ORBS does not decide what is "permitted" nor do any of these other databases. They have a set of criteria for deciding whether and when your mail server ends up in their database. If their criteria matches mine, then I can choose to use them as part of my mail filtering.

    1. These list should inform you have been added
    2. They should leave you 10-15 days to fix the problem before blocking you
    3. They should help you. I was *very* shocked by ORBS attitude "we block you, and we don't care if you cannot correct it"


    I'm sick of the attitude that ORBS owes you something when your mail server is an open relay. If your system is an open relay, your fuck-up will cost them time and effort as they add your system to the database. Now you think that they owe it to you provide you an absurd amount of warning (10-15 days), notification that you were added, and then you want them to provide free consulting services (see item 3). If you don't know how to run a mail server, then stop trying to.

    It's like being ticketed for driving your car down the wrong side of the road at 90 miles per hour and then being pissed off that the cop did not provide you with free driving lessons and give you 10-15 days to stop driving like that.

    If your system is an open relay, unplug the Ethernet cable immediately and leave it unplugged until the system is fixed. If you don't know how to fix it, then pay professionals to provide your SMTP & POP services. A spammer could spew tens of thousands of messages per hour through an open relay and you owe it to everyone else on the net do whatever it takes, including pulling the plug, to make sure that your system is not an open relay.

    I think that ORBS should charge a processing fee for "expedited removal" from their database and, otherwise, just remove systems once a week.

    1. Re:It's democracy and freedom in action. by fmaxwell · · Score: 4, Interesting

      Not removing now closed relays from the list is like not releasing prisoners from jail. Something which might or might not be a good idea...

      That's assuming that you consider the list to be a punishment. I believe that they are information sources -- IP X was, and may still be, an open relay.

      Also, I think the usefulness of DBs like ORBD lies in them staying current, as I think it might cost more losing one important mail than wading through tons of spam.

      I agree. But keeping the open-relay databases current is not a responsibility the database providers have to those listed in the databases. It may affect the popularity and usefulness of their service, but that's another matter altogether.

      If some person/group decides to create such a database, they have only the following two responsibilities:

      1. Do not defame/slander by listing a system incorrectly. That said, they make up the rules and if they say their databases are "IP addresses that were open relays within the last six months", they have up to six months after a relay is closed to remove the record from the database.

      2. Provide services paid for. If they accept payments to remove entries within, say, 24 hours (rather than the normal cycle), they have to remove those entries within 24 hours. Otherwise, they can remove them in conformance with the criteria that they set (see item 1).

      Again, you are viewing this as punishment and I'm viewing it as information. Since ORDB does not block e-mail, harass ISPs listed in the database, etc., they aren't punishing. They are just providing information Now if bobco.com rejects your e-mail because your IP is listed in the ORDB, then maybe bobco.com is punishing you, but ORDB is not.

  21. What helped us and our users the most by shadie · · Score: 5, Insightful

    We (dds, a dutch isp) had a spam problem, and being a free email provider for such a long time did contribute to that. When we went out to solve this problem we did it in three steps:

    - Implement RBL+ on our mailservers (got the load down a bit though)

    - Created a global "spam filter" (weight system a la junkfilter) wich was opt-in for our users..

    - We installed procmail, gave each user it's own .procmailrc and made a web interface to create procmail recipes in an "outlook" style.

    This recipe maker could then be accessed by each user on their own user pages, or they could just make receipts through their shell access

    Our end users didn't really notice much about our use of RBL. And most of them don't know what rbl is annyway.

    But giving them the possibility of filtering email on the serverside _themseve_ did make a difference! It gave them a feeling we are fighting spam, and that THEY are also in control !

    And last but not least... Giving your users info on how to _avoid_ spam is important!. We did this by writing clear faqs on avoiding spam, and pointing each new user to these faqs

    (b.t.w... this was my first post on /. , lurking time is over i guess :-)

    --
    -- Hi! I'm a signature virus. Copy me into your sig file and help me spread
  22. Bad analogy. by achurch · · Score: 4, Informative

    [Running an open relay is] like being ticketed for driving your car down the wrong side of the road at 90 miles per hour and then being pissed off that the cop did not provide you with free driving lessons and give you 10-15 days to stop driving like that.

    Nice analogy, except that it doesn't work. If you're driving at 90 miles an hour on the wrong side of the road, then (1) your speedometer will tell you that you're driving at 90 miles an hour and (2) looking ahead will show you which side of the street you're on, which you can tell is the wrong side because of what you had to know to pass the test to get your driver's license.

    With mail servers, however, there isn't, at least yet, any widespread tool that will tell you if you have an open relay (and given how such tools work, they'll probably be banned as "hacker tools" at the rate things are going these days). In fact, I found out recently that I'd been placed on a blacklist for having an open relay, which took me by surprise because I'd been careful to avoid having anything like that happen; it turned out that I had missed one of the potential avenues of abuse (specifically, using error bounces to spam people).

    So until running a (secure!) mail server becomes as simple as driving a car and people need licenses to run servers, your analogy is inappropriate.