Slashdot Mirror


Paul Graham Describes Dangers of Spam Blacklists

CRoby writes "Paul Graham posted an essay describing the danger and corruption of the main spammer blacklists today. It discusses MAPS and the SBL, the blacklist created to try to alleviate the abuses of MAPS, and suggests (maybe) another blacklist's creation."

14 of 611 comments (clear)

  1. Definitely a bad idea... by nev4 · · Score: 3, Informative

    We've been blacklisted before and the sysadmins who run these things often WILL NOT remove you, no matter what. I'd take all the SPAM anyday vs. not being able to send legitimate emails.

    1. Re:Definitely a bad idea... by Anonymous Coward · · Score: 3, Informative

      You really don't get it.

      The point isn't *me* using MAPS/SBL. The point is that others use it, thinking it makes a difference. Your netblock (that is, your ISPs netblock, or your ISPs ISPs netblock, etc) gets included in that list and *bang* you're a casualty of war.

      Get it yet?

    2. Re:Definitely a bad idea... by hawkbug · · Score: 2, Informative

      Right on - a company can't simply get out of an ISP contract for a lot of reasons. Technical reasons aside, imagine getting out of a 3 yr contract after 2 months. It's not going to happen.

  2. A few comments by alanw · · Score: 4, Informative
    From Paul Graham's original article http://paulgraham.com/spamhausblacklist.html
    any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam
    The primary use of the SBL is to allow sysadmins to refuse e-mail coming from listed IP addresses. The mail should be rejected during the SMTP header conversation, and the senders of genuine (non-spam and non-virus) e-mails will receive a non-delivery report from their outgoing MTA.

    I assume that what Paul Graham is complaining about must be SpamAssassin, or some other content filter, applying a score to articles containing URLs, which when looked up in DNS resolve to listed IP addresses. This is much less acceptable, since the sender has no way to know that their e-mail may have been classified as spam.

    The details of the listing can be found at http://www.spamhaus.org/sbl/sbl.lasso?query=SBL279 45. This is a /32 - i.e. a single IP address. I don't know why Paul Graham's web site (which has that IP address) has been associated with textileshop.com, which has a completely different IP address.

    The other Yahoo listing on the SBL is also a /32.

    I also note in another of Paul Graham's articles http://paulgraham.com/sblbad.html he claims

    The most notorious example is the MAPS RBL
    As any fule kno, the most notorious spam blacklist is SPEWS. ~
    1. Re:A few comments by mercuryresearch · · Score: 3, Informative

      Seeing as how this exact situation happened to me this week, I can provide some light on the /32 IP address issue.

      In my case, I moved a server to a new colo facility. Most facilities have an IP block, and you get assigned an IP from it. Six months or a year ago that IP might have belonged to someone else. For me, it turned out in February a spammer installed a server at the colo, spammed from that server for a single day before the colo ISP turned them off. That IP got listed in Spamhaus; in the beginning of June I was assigned that IP.

      So, I ended up with a Spamhaus listing for my mail server's IP address -- and _I_ can't get it removed. Spamhaus expects the colo operator to contact them (which they did on my request) but even there, if the blacklist operator doesn't like the ISP/colo people, they can ignore the request.

      Fortunately Spamhaus listened and I got the record for my IP removed. But this showed me it was trivial for a non-spammer to inherit a blacklisted IP. I've added doing DNSBL checks on colo-assigned IP addresses for future moves to prevent any future issues.

    2. Re:A few comments by sloanster · · Score: 2, Informative

      I assume that what Paul Graham is complaining about must be SpamAssassin, or some other content filter, applying a score to articles containing URLs, which when looked up in DNS resolve to listed IP addresses. This is much less acceptable, since the sender has no way to know that their e-mail may have been classified as spam.

      Um, no. That's not how spamassassin works - spamassassin uses a wide spectrum approach - it can take into account whatever blacklists you want to consult, but an RBL hit in spamassassin does not automatically mark the message as spam. An RBL hit is just one of over a thousand factors taken into consideration when making the call as to whether a specific message is spam or not.

      Other methods used include central clearing houses of known spam messages (razor, DCC etc), time offsets, examination of header content, message content, weighted statistical analysis, presence of buzzwords, phrases, URL patterns and more.

      Using all of the methods available and making a decision based on the overall picture makes spam assassin a very effective tool, with far fewer false positives than a hard coded "RBL in the MTA" approach.

      On the other hand, SA does use more machine resources than does simply rejecting a message based on an RBL result, but that's the price of intelligent behaviour - it almost always requires more effort than a knee jerk reaction.

  3. Paul is just pissed because... by SSpade · · Score: 3, Informative

    ...his website is hosted on the same IP address as a spammer (textileshop.com) was on yesterday, and because of that he's seeing some of his mail blocked.

    There's certainly a need for thoughtful and hopefully positive criticism of blacklist behaviour. This article is not it.

    1. Re:Paul is just pissed because... by SSpade · · Score: 3, Informative

      Actually the IP address that's listed is store.yahoo.com.

      Yahoo hosting is riddled with spammers, and store.yahoo.com is where most of them live, and where they accept credit cards for their purchases.

      The SBL lists IP addresses that are involved in spam. 66.163.161.45 is involved in a lot of spam. It's not been removed from the SBL because, well, it's still actively being used by spammers.

      Because countless spammers register domains on a daily basis, yet point them at the same IP addresses some people choose to resolve the URLs in incoming email and bounce the mail if any of them resolve to particularly filthy IP addresses.

      66.163.161.45 is filthy. Blocking mail that has URLs pointing there will stop a fair amount of spam. Not an approach I'd use myself, but certainly a lot more effective (in terms of spam caugh and false positives) than many, many other approaches in widespread use.

      Paul chose to host his website there, despite supposedly knowing a lot about the spam issue. That was probably not a good call.

  4. What IP is the originating mail from? by isn't+my+name · · Score: 2, Informative
    # dig paulgraham.com MX

    ; <<>> DiG 9.2.4 <<>> paulgraham.com MX
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53349
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

    ;; QUESTION SECTION:
    ;paulgraham.com. IN MX

    ;; ANSWER SECTION:
    paulgraham.com. 3600 IN MX 10 milter1.store.vip.sc5.yahoo.com.

    ;; AUTHORITY SECTION:
    paulgraham.com. 3600 IN NS st-ns1.yahoo.com.
    paulgraham.com. 3600 IN NS st-ns2.yahoo.com.

    ;; ADDITIONAL SECTION:
    st-ns1.yahoo.com. 154169 IN A 216.136.225.202
    st-ns2.yahoo.com. 134882 IN A 216.136.225.203

    ;; Query time: 228 msec
    ;; SERVER: 192.168.1.23#53(192.168.1.23)
    ;; WHEN: Thu Jun 16 14:30:43 2005
    ;; MSG SIZE rcvd: 150
    Looking up the IP for his mail server, we get:
    # nslookup milter1.store.vip.sc5.yahoo.com

    Non-authoritati ve answer:
    Name: milter1.store.vip.sc5.yahoo.com
    Address: 216.136.232.238
    A Multi-RBL check on that IP shows absolutely no black-listing in any of the many RBLs.

    Is it possible that it's his outgoing cable-modem IP address that is the problem?

    Is it, as the parent suggests, spam-assasin filtering?

    I'm more than happy to get on the wagon of unresponsive RBLs. The only way they can actually get the response they want is if cleaning up your act results in de-listing.

    However, Mr. Graham makes some big claims with nothing to back it up--and attempting to investigate on your own shows that his claims don't seem to check out.
    1. Re:What IP is the originating mail from? by kaarlov · · Score: 2, Informative

      MX records don't always tell where the mail is sent from. In fact it is good idea to have separate server for sending mail. For example if your MX in some situation sends bounces to forged aol-addresses, it gets very easily blacklisted temporarily by AOL. But sending mail directly from server which hosts multiple webpages in same ip is not a good idea. But I don't thing Graham does that either.

      From TFA and from parent article I got impression that he suffers from people having spam filters which run URL's in the email body through blacklists. And I think that spam filter which gives too much points for that is more broken than the concept of DNSBLs.

  5. 'Terrorism' my behind... MAPS' side of the story by mi · · Score: 2, Informative
    Although MAPS did, indeed, only blacklist the actual spammers at the beginning, they changed not because they 'got carried away' (Paul Graham's words), but because the spammers adapted.

    Here is the link, that responsible editors would've offered in a story like this...

    --
    In Soviet Washington the swamp drains you.
  6. Home Connectivity ISP != Your Domain ISP by billstewart · · Score: 2, Informative

    Maybe you only have three choices of broadband ISP at home, or live somewhere sufficiently rural that there are only three choices of dial ISP - that's entirely irrelevant to how many choices you have on where you get your email, send your email, or host your web servers. Sure, it's convenient to be able to run all those things from your home Linux box, but if you want to do that, you'll probably find that your cable modem company and some of the DSL ISPs that your phone company supports might not permit that. There are hundreds or thousands of companies that run POP/IMAP mailbox services, and probably more that will host web sites, and that's not even getting into options like virtual hosting.

    --

    Bill Stewart
    New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
  7. Re:Slashdot Language lesson by DavidTC · · Score: 2, Informative
    Vigilantes don't technically 'have' to break the law.

    For example, in many places it's legal to do a citizen's arrest if you see someone actually committing a crime. If someone suspects a crime will be commited and hangs around armed with the intent of bringing the person in, that's vigilantism, and perfectly legal. Or even hanging around waiting to call the cops.

    Or if, for example, people keep getting attacked in a certain part of town, so you, who happen to have a blackbelt, wander through there, waiting to be attacked so you can fight back...

    It's usually not called vigilantism if it's legal, but if you are attempting to do the work of the legal system, it is being a vigilante.

    However, vigilantism requires enforcing a law, be it an actual law or just a made up one. Or punishing someone who already broke the law. (Or, as sometimes happens, you merely suspect broke the law.)

    Whereas spam fighting may be interacting with the results of a crime, it's no more vigilantism than picking up litter is, or rebuilding a house torched by arson. The crime already happened, no one's trying to punish or catch the criminals, they're trying to undo the harm caused.

    I guess you technically could call spam reporters 'civil vigilantes', by analogy, because they are reporting a contract violation between two third parties to one of those parties. Instead of taking criminal offenses into their own hands, they're taking civil ones. But that's getting a bit silly.

    --
    If corporations are people, aren't stockholders guilty of slavery?
  8. Re:Wholehearted Agreement by aaronl · · Score: 2, Informative

    That works fine for him to keep the mail coming in. The problem is when you combine the annoying "dynamic ip range" lists with an idiotic admin that thinks using one to blindly deny is a good idea. I mentioned in another post, but Juno and Netzero do this. Neither will pay attention to you when you complain. Of course they also RBL deny their postmaster account, which is a no-no.