Slashdot Mirror
Paul Graham Describes Dangers of Spam Blacklists
CRoby writes "Paul Graham posted an essay describing the danger and corruption of the main spammer blacklists today. It discusses MAPS and the SBL, the blacklist created to try to alleviate the abuses of MAPS, and suggests (maybe) another blacklist's creation."
The problem was, as vigilantes so often do, the guys at MAPS got carried away
For some reason, journalists keep calling blackmail lists "vigilantes". But there's something they don't understand: nobody forces email system administrators to use those lists.
These lists are provided by people for free. They decide to list bad email servers, but they may as well include any server they want. After all, who's to force them to provide quality of service?
The real problem, of course, is that blacklists are needed in the first place. If ISPs did their jobs a little better (aol, hotmail and the likes), the amount of spam would already decrease significantly. And don't speak to me about chinese ISPs, since most spam comes from the US.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
A blacklist for a blacklist for a blacklist...
Personally, I find the need to disable more and more RBL's, because today a user might come thru OK, tomorrow, they're stuck in SORBS and considered a HIGH risk.
IGB: More fun than eating oatmeal!
I had the unfortunate "joy" of being blocked by some of these draconian blacklists. My sister requested some information from me for a trip that she has upcoming via my yahoo.com account. After it bounced from her ISP saying that I was sending it from a "spam-hosting" ISP, I sent it from my mac.com account. Same schtick. After a couple other choices, I finally got it sent from my .edu account.
Her ISP uses SpamBag for their blacklist. SpamBag? ScamBag is more like it.
No wonder my sister is disenchanted by email. Her yahoo account got spammed to no end, then she can't get emails from most of her friends since they get bounced back by her ISP's stupid blacklist.
Blacklists are fine and dandy in principle, but practice has shown them to be useless. IT managers, just drop them. They're more annoying than anything.
-Jellisky
So...it's okay if he goes to Federal Pound-Him-In-The-Ass penitentiary just because he rented a car from a place that also rented a car to a crack dealer?
Huh?
Sorry, but that's still bullshit. He states it clearly in his article: You can't screw over innocents just to make the guilty pay. Does the your government put a neighbor family through torture just because you got a parking ticket? No. It's YOUR fault and YOU should be punished. Not some innocent bystander.
Comment removed based on user account deletion
Blocklists are made by people for others to use if they see fit. When they become unusable, they're no longer used. Personally, I use none. The cost to me of one false positive is greater than 1000 spams that leak through. No list is that good.
We deal with this all the time. Leaving any IP on a blacklist for any period of time doesn't help. Most spammers nowdays spam and run. They unload from a hacked account through a broken formmail script or a zombie computer. After 36 hours they have dumped their million emails and moved on to another IP. Blacklists generally don't get this though. They just make a bigger and bigger list. The problem with this approach is that they already missed the spammer. One time we dealt with someone who was running a blacklist and when we asked why an IP was on the list they said because it spammed years ago. When we said we have controlled the IP for the past three years they said it doesn't matter. It's like give me a break...
The solution to blacklists is to use an AOL model in which dynamic IP blocking is used. When spam is noted from an IP that IP is automatically blocked for 24-36 hours after the last spam comes in. That way the innocents are not being blocked and the spammers email doesn't make it through. There are a couple blacklists which do this but more should.
Compare this to the opposite blacklists like BLARS which requires a thousand dollars for "him" to investigate whether an IP should be removed. I have never seen an IP which is not listed with BLARS.
Quality Hosting e3 Servers
I'd take all the SPAM anyday vs. not being able to send legitimate emails.
Except that blocklists don't stop you sending email, they merely allow others to decide whether to accept that mail. Or do you think other people should be forced to accept any and every email you send?
My next sig will be ready soon, but subscribers can beat the rush
John Reid of the SBL told me this wasn't true-- that the SBL was still clean, and that they only blacklisted hosting companies' mail servers when they were spam hosts who took on innocent users as camouflage:
He is right. That definitely is NOT how SBL actually operates. I have a site that is heavily trafficked (millions per month) and they blocked my email (from my own personal server) that has delivered mail for my site for seven years with absolutely no outgoing spam or relaying having ever occurred in its entire life.
However, a spammer with false credentials faked his way into a hosting account with my colo provider and as a result, SBL blocked multiple entire submnets, rendering my entire site and service useless for almost an entire month (we deal with auctions, meaning nobody was getting closed notices, won notices, outbid notices, addresses to send payment, registration emails, lost password emails - and when they complained, I couldn't respond to help them and explain it to them).
SBL couldn't have cared less. As far as they are concerned, if one IP is a source of spam, they all are. And they'll get to fixing it in their own damn sweet time.
But the defense of SBL fan-boys is typically "well it's VOLUNTARY!".
Yeah. Whatever. Fuck off.
People switched from MAPS because the other lists were free, not because MAPS was too aggressive.
"As of this writing, any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam."
Whisky Tango Foxtrot? *BLs block IP address ranges, not URLs.
"Because the guys at the SBL want to pressure Yahoo, where paulgraham.com is hosted, to delete the site of a company they believe is spamming."
1. Given that Paul's mixing up URLs and addresses of mail servers, I'm not prepared to take at face value the statement that SBL is blocking Yahoo's mail servers to pressure Yahoo to drop a "site", rather than (say) mail services Yahoo is providing the spammer.
2. If Yahoo is providing services to a spammer and Yahoo refuses to deny those services to a spammer, than Yahoo is being "spam friendly", no matter what their reputation is, and they may well be depending on the many legitimate lists they're hosting to avoid responsibility for their actions. That's exactly the situation that John Reid is referring to in Paul's quote.
I don't know what alleged spammer this is referring to, but what Paul's written is clearly not anywhere near the whole story.
What they do is allow others to block email between two diffrent people, simply because they run the mail servers that sit between them. If it was only individual users who were using these blocklists, it would be a diffrent issue. But it's not.
autopr0n is like, down and stuff.
Let me reword your justification of of this behaviour so others can see the flaw in it more clearly:
[66.163.161.45 is a filthy neighborhood. Lots of criminals live there. So, a group of vigilantes randomly started machine gunning people walking the street. Not something I'd do myself, I prefer to use a shotgun, but certainly more effective then using the court system. Paul chose to live there, and he should have known it's a bad area. If he gets shot at random, well, too fucking bad, he should have known better. Living there was probably not a good call.]
Some days it's hard choosing between deleting 400 spams a day and dealing with the exsistance of "spam blocking" groups. Then I read a comment from an "anti-spam" person and I think I'll be safer choosing to work that delete key.
I reserve the right to block (or accept) any mail I choose on my own system. I also make that decision on behalf of my users, weighing the pros and cons, and especially the listing policies, of any RBLs. If I get it wrong, then yes, my users won't be happy. I'm all for doing what makes my users happy. Blocklists do make my users happy. They work. The fact that there's sqealing about the effect shows that they work. I reject utterly the contention that I should somehow be forced to accept anything I don't want to receive
My next sig will be ready soon, but subscribers can beat the rush
What else do you feel strongly about?
There are websites, I am sure, that describe in detail how to commit murder and get away with it. Some readers may find those sites, and using that knowledge, go commit violent crimes -- just as some readers of spam sites may purchase email harvesting software and then go commit the crime of sending bulk email. I assume you would support blacklisting ISPs that host violent-crime advice, since surely everyone agrees that murder is worse than spamming.
There are ISPs that host neo-Nazi propaganda calling for the murder of all non-whites. Do you think that's better or worse than offering spam software for sale? Should those ISPs be blacklisted?
Escort services? Simulated rape porn? "The Anarchist's Cookbook"? A list of abortion providers' addresses? Al Qaeda recruitment and propaganda? I want to know which of these you think is equally as bad as, or worse than, hawking a CD with a million email addresses on it. How many things do you think merit blocking all of an ISP's innocent websites?
You have your list. Others have their own lists -- and, frankly, there are a billion people who think porn is vitally important and your fixation on spam is stupid. Do you really want the internet segmented? Do you think advancing your pet cause is worth walling off the internet into warring quarters? Do you really want to wield a censor's black pen?
That's the point - it doesn't matter how fast you respond to a spammer. If you ditch the spammer instantly, you're still going to end up on the list indefinately. In the case I cited, the spammer was kicked off within hours. I'm sure he was off to some other unwitting place to spam from while the rest of us went weeks without being able to send from our servers.
How is it an incentive for admins to be "responsive" when dealing with spammers if you're going to punish everyone within a certain radius for days or weeks even if the problem was terminated within hours?
What exactly is so wrong with blocking an IP at a time? You do away with the innocent bystanders while still nailing the spammers. Anyway, the reason they block the entire subnet has NOTHING TO DO WITH PREVENTING SPAM. It's merely a way of pissing off enough legitimate people to force the bad person to be dealt with (even if they've already been dealt with or it was an honestly unavoidable situation or what have you).
If you've identified chronically spam-friendly hosts and want to widen your net for them, that's great. But don't take out the entire neighborhood because of one bad neighbor.
Actually, I'm with singletoned, and I think it's you that has a problem with understanding. Understanding something involves realizing implications which are not immediately obvious. Understanding is something that few people ever really do. Reading the facts isn't enough, you need to be able to manipulate those facts and draw provable conclusions from them. THAT is understanding.
For example, in order to get revenge on people they believed were spamming, MAPS would blacklist the mail server of the company hosting their site.
The problem with blacklists is that they're human controlled and extremely susceptible to egotistical vigilante-ism. If I'm getting spam from a server, I don't have to block just that server. I could block every server in the headers, for example. What I choose to add to my blocklist can be totally arbitrary, and that's the problem with blocklists controlled by individuals that can block huge IP blocks.
And, in terms of preventing the "sending" of mail, you could consider a blacklist to be a postman who would, whenever he saw a letter from a given return address, he'd destroy it. Any time you got a New Scientist magazine? destroyed, at their discretion. How many companies use a blacklist without saying what's on the blacklist, or making the blacklist easily searchable and editable? Does a user ever get a message on a regular basis "Hello so and so, you've received 274 emails this week from addresses in our blocked address list (which contains mostly spammers; click here to make a change." ? No, they don't provide that helpful information with links to the relevant information.
The mail is just blocked, it disappears into a void. By intercepting it before it reaches its intended recipient you are effectively preventing it from being sent. Because it's not the addressed recipient that decides whether or not to accept the mail according to the blacklist, it's an unnamed middle-man or middle-men. A blacklist allows any server in-between the sender and the recipient to say "no, sorry, your ass is blocked."
I do think people should be forced to accept every email that I send. They shouldn't be forced to READ them all, but they should be forced to accept them. As email becomes more and more prevalent as a form of legally recognized communication (emails are used in court as evidence) it's important to recognize the implications of interfering with that communication without disclosing such interference. Would you like it if I were your postman and every time I saw your electric bill, I took it and destroyed it because I didn't like the electric company and I didn't think anybody should be subjected to their tortures? Would you like me totally interfering with your legal communication and then not telling you, not even sending you a friendly "the electric company is evil, go solar!" letter? Would you like the way that could impact your finances, your credit, your reputation? What happens when somebody adds an obscure credit union to a blacklist and people don't get fraud alert emails from the CU, just because one server in their datacenter was compromised and used to send 10,000 spams? Do you REALLY understand, now? I still don't think you do.
The blacklist themselves aren't really responsible for breaking any rules, which they believe absolves them of acting responsibly. The fact of the matter is that blacklists are often implemented in the most infuckingcredibly ignorant ways possible, unfortunately. No e-mails as per my suggestion above, no way for the sysadmins that use the blacklist to audit/edit it, etc.
We need a wiki-style collaborative blacklist that has a membership of thousands who all collaborate on this issue. It's just one more example of how giving one person too much power before they're ready to use it responsibly with proper discretion results in a disaster. A blacklist affects too many people to be implemented so willy-nilly at only a few peoples' (poor) discretion. We need a collaboration, a large committee who will not become corrupted by power (as none of the members will individually have any power) but will be a gathering of individuals who maintain their individual opinions and ensure that the system remains fair and balanced.
Reinvent the wheel only at either a lower cost, greater effectiveness, or your own personal enrichment and satisfaction.
Right. So then, when those of us with a .nu domain name have to change ISPs constantly because, at any moment, someone else - that we have no control over - ruins the ability for our email to go to its intended recipient - we just get to suck up the 10$ a pop IP change for our DNS? And even aside that point - while hosting companies are a dime a dozen, good hosting companies aren't. When we do find one that is, we want to stick with it. It's not their fault someone else at the same colo decided to be a jackass.
Basically, you're just saying "too bad, I'm tired of being screwed over by spam" and I'm saying "wtf, I'm tired of being screwed over by blacklists that can't keep their shit together". Put yourself in my shoes - when a blacklist service becomes worse than spam and the spammers who spam, what does that tell you about blacklists?
Gentlemen,
You do realize that Paul Graham is in the business of pushing Bayesian anti-spam filtering, which he claims as 'the best' solution to spam. For a long time Graham has been spreading FUD about other anti-spam solutions, in particular blocklists. We're well used to hearing utter bollocks about blocklists spread by him.
Yesterday we listed on the SBL an IP of a spammer which as luck would have it is being shared by Paul Graham. We of course can not simply give the spammer carte blanche to spam our users because Paul Graham is also using the same IP. Graham has no concern for the fact he's sharing his IP with a spammer, and rather than contact his ISP to ask what a spammer is doing sharing his IP he simply sees a PR oppurtunity to bolster his "blocklists are evil, bayesian is good" campaign. I'm only surprized this actually made Slashdot.
Steve Linford, CEO, Spamhaus
Considering how much my spam has been reduced by the SBL (anywhere from at least 50% up to 75%) I'd like to just say:
The mail servers under my control have always subscribed to the SBL-XBL (well, more accurately, before the XBL was established it was the SBL and cbl.abuseat.org. The latter is dedicated to short-term [72 hours, as I recall] blocking of e.g. spammers operating on DSL or cablemodem lines who are likely to appear on an IP address once or twice and then get kicked off. The CBL is now also represented in the XBL). I have so far, in the last 3-4 years or so, only been able to confirm 1 and 1/2 "false" positives in that entire time - one was from a person in China who was using a confirmed spam-haven ISP, the "1/2" from a company that, after an informative response from the CBL people, I believe were listed for appropriate reasons. In any case, the latter case cleared itself up when they were automatically re-removed from the CBL [they'd been there before] and the email lost WAS an advertisement anyway...)
I have noticed the numerous stories of overzealous blocklists, which are obviously a bad thing, but I can't think of a way to reasonably put the SBL in that category...
Besides, bayesian filtering only works AFTER the spammer has been allowed to tie up my mail server's bandwidth (and then allows them to tie up your mail server's CPU time with the bayesian analysis). I prefer to cut off known spammers before that point whenever possible. THEN I pass the remaining messages through SpamAssassin. Back in the early days of spam, I used to actually go to the effort of picking apart the mail headers and looking up the abuse addresses for the ISP whence the mail came AND the hoster of the spammers website (and on one or two occasions, even the registrar for the spammer's domain name, when I could confirm that the information was falsified). It's been a long time since I was able to keep up doing that with the volume of spam coming in, but I still can't stand the thought of allowing spammers to take ANYTHING from me that I can prevent...
Hacker Public Radio is our Friend