Slashdot Mirror
The Spam Problem: Moving Beyond RBLs
whirlycott writes "I just published a paper called The Spam Problem: Moving Beyond RBLs on my site. I comprehensively describe RBLs and list eight specific problems with them. I also get into ideas that next generation antispam system creators should read. I hope that this will be useful to anybody who is attending the Spam Conference at MIT on Jan 17th."
My spamassassin-tagged mail usually scores between 1 and 1.5 ( a 5 is needed for a **SPAM** tag) - which in the grand scheme of things seems to be enough of a weigh for the value of an RBL. Don't absolutely trust it's value, but don't ignore it completely either.
I don't really see why anyone would use RBLs just by themselves. Personally, I have spamassassin catching the "big spams", you know the ones with webbugs, html-only, forged headers, etc. etc. I occasionally tag those as junk in my Mozilla Mail, while tagging my normal mail as not-junk. The Bayesian filter takes care of the occasionally sneaky spam. Once trained it's an awesome combination.
My company was collateral damage on SPEWS last month and I kicked the *^&^#$* out of our ISP for hosting Global Travel on our netblock. They got booted and we got cleaned off the list. Bada-bing bada boom.
5 2%24Db4.726975%40twister.tampabay.rr.com
RBL's are like a fever. They tell you when something it wrong and only a dork blames the fever when the problem is the disease. Get your ISP to whack the spammer or change ISP's.
http://groups.google.com/groups?threadm=Fc6K9.262
My God! It's full of Voids!
Absolutely. Spread the message to new users. The response to spam is very small, on the order of hundredths of a percent. The spammers get negligible responses because of the sheer numbers of recipients. I can't help but think that it's mainly newbies that respond to spam; x amount of unwary sheep getting sheared the first time they see the opportunity to 'Meet lonely married people' or 'add inches to penis/bust/whatever'.
I'm not really a web designer, I just play one on the Internet.
Well, I have mod points but I have to reply.
So, this guy has a problem: his mail server is blacklisted because it is part of the same netblock as a spammer.
So, rather than switching to a responsible ISP that doesn't allow spammers on its network, he writes a long winded whine about how to solve the "problem" of RBLs (although, mind you, he doesn't give a solution, just what he thinks should be part of the solution).
What he doesn't seem to understand is that the blacklisting of entire netblocks is only done as a last resort when ISPs refuse to get rid of spammers on their networks. It is a punitive measure to try to force the ISP to act.
While I applaud this guy for doing his research, I think he is misguided and even narrow minded. If you are part of the 'collateral damage' because your ISP allows spammers on its network, do the right thing and take your business elsewhere.
I have been a very loud protestor about collateral damage in news.admin.net-abuse.email. I well understand the problem but I think you over-estimate it. SPEWS deliberately lists non-spam-source IPS - that's collateral damage, that's wrong and avoidable. Take that away and the remaining collateral damage is unfortunate but not severe.
Many have changed how they use RBLs - instead of simply rejecting they send a reply asking for confirmation the sender is a real human. If that confirmation is made the original message is delivered. That seems to be simple, straightforward, and capable of reducing collateral damage to a very low level. It even has intelligence behind it.
I advocate relay spam honeypots (and open proxy honeypots - move with the times, keep up with the spammers). The white paper doesn't even mention these. The WP has the section asking if open relays are necessary. Well, no, they probably aren't. Is there a point? For how many years has there been an effort to secure open relays? Has it succeeded? The fact is that they are there - asking if they are necessary may inform you but it doens't change the situation in any useful way.
For all these years the spammers have been given free access to the relay level - there's a self-satisfying division into the secure systems run by the wise and the open relays run by inept administrators. that division allows the operator of a secure system to condemn the operator of an open relay with confidence - he can strut. Yipee. As a spam-fighting tool it's a close to a complete bust. Well, yeah, lots of open relays have been secured. BFD - there's still enough for the spammers, and RFC 2505 said it would be this way. Yo: RTFM (in this case RTFRFC.)
You want to hurt the spammers? OK, hurt them. It's not like you have to go out of your way - accept and deliver one of their relay tests and the chances are excellent they'll send you spam that you can discard. That's still a secure system, but it has teeth instead of gums.
There's all these people falling over themselves devising elaborate filters. If you simply open up a relay enough to accept the spam but not deliver it there's no filter needed - a non-mail-server system that receives relay email receives close to pure spam - you will never get a filter as selective as that. Accept and deliver the relay tests and you have screwed the spammer. I won't even enumerate all the ways he is or can be screwed but there's a bunch.
If 5% of the Windows systems with network connections ran Jackpot then spam would be dealt a mortal blow:
http://jackpot.uk.net/
It isn't hard, and it does tremendous good. Check it out.
How about a pizza company refusing to accept orders from a paticular motel because often noone will admit to ordering there? Stay at a different motel.
If you are using an ISP that does not enforce acceptable use policies restricting unsolicited email, you are supporting spaming activity.
In the past, when just systems that were directly associated with spam were blocked, the ISPs would move the spammer to one of the unblocked ips, and move an innocent to the blocked ip. Turns into 'whack-a-mole'.
With most blocklists, the block starts out small, targeting just the spammer. If the ISP gets rid of the spammer, the block goes away. If the ISP ignores complaints, the block grows.
You (ISPs) just need to modify your IP allocation policies such that you put all known spammers in the "ghetto" address range. Said range gets blocked by RBL, none of your more legitamate users notice. The spammers can't complain because they are breaking your AUP (you have a well-defined AUP, don't you?).
People spam because it's dirt-cheap. If spammers had to pay 10 an email, you'd better believe they'd be a heck of a lot more cautious about who they send to.
And a "Stop Buying Spam Products" is doomed to fail, anyway, because it's a numbers game. If 1 person out of every 100 people spammed buys something, then it's probably an outrageously successful campaign.
The fact is, you may be throwing out 50 spam emails a day, but if you see a subject line that speaks to an immediate need, you're probably going to stop, read it, and consider a purchase.
Not exactly. Besides being a theft of end-user and mail-site resources, spamming is also a scam perpetrated upon businesses. If you got spam advertising Joe's Naked Kinky Web Site, that probably isn't because Joe thought up the idea of spamming you all on his own. Most likely, a career spammer (let's call him Alan) convinced Joe that spamming was:
- effective,
- legal, and
- everyone's doing it anyway, so why miss out?
Joe then paid the career spammer to spam for his naked kinky Web site. Since all three of Alan's claims are false, and he knows it, this means that Alan has defrauded Joe. He exploited the fact that Joe is probably neither an Internet expert nor a lawyer, but he does feel competition from other naked kinky Web sites, to convince him to pay for spamming.(Yes, Alan the spammer told the news media that spamming is effective, too ... and they believed him. He was lying there, as well -- but it got him, and spamming, free advertisement in the news!)
This scam does not rely on spamming actually being effective, so long as vendors still believe it might get them an edge over the competition. Thus, getting people to quit buying spamvertised products cannot (directly) affect it. Only when all vendors on the Internet -- yes, including naked kinky Web sites -- realize that spamming doesn't work, isn't legal, and that they can do just as well without it, will spamming go away.
SMTP already has a good way of authenticating who you are receiving email from. It is called the IP address of the machine that is contacting you and the IP sequence numbers of the packets that have to travel between you. All you need is a list of the IP addresses of the people who you want to receive email from and a list of ones you don't.
But, of course, this is what the current blacklists do!
Any email authentication system is going to run into most, if not all, of the same problems that DNSBLs run into. They are also going to have the problem of trying to get the entire world to change.
SPF support for most open source mail servers can be found at libspf2.
Ralsky. He says, in a Detroit Free Press interview, that he has 50 spam servers in Dallas.
2 .h tm
http://www.freep.com/money/tech/mwend22_2002112
Just try to get the ISPs in Dallas to act with integrity, seek out the spam servers (they should leap out in any traffic analysis) and shut them down. The DNSBL's are close to useless here, it seems. Ralsky spams from Dallas using asymmetric IP routing: he spoofs the IPs of dialup systems from the servers. If anything gets nuked its the dialup account, not the high-speed-linked system that actually sends the spam (the dialups only receive the return packets from the systems that receive the spam.)
(Maybe Ralsky spams from Dallas differently - earlier this year he surely was using the asymmetric IP approach. Ralsky did lose throwaway accounts on three different ISPs because of the actions of one honeypot operator: Michael Tokarev in Moscow. Unfortunately Michael shut the honeypot down in July:
http://www.corpit.ru/cgi-bin/h0n5yp0t )
Getting Ralsky in jail wuld be nice, and he deserves it. Before that it would be effective to so disrupt his spam operation that he experiences a negative cash flow. Honeypots are the way:
http://jackpot.uk.net/
Setting up the honeypots is the first step. Once enough are intercepting Ralsky spam notify the spam advertisers that huge amounts (don't tell them the actual amount) of their spam is being intercepted. Get them in billing disputes with Ralsky. If they also see sales going down (as they should) they may have a flash of intuition that tells them spam doesn't work any longer, and the interceptions are the reason.
But don't stop doing what works for you, of course - add in the honeypot for its effect on the spammers beyond your own system.
I am admin/postmaster for a small college. Several months ago a new hack was developed that got through my version of sendmail. This was kind of ok because the spammers didn't know I was vulnerable.
Along comes one of the RBL's and test my site. So far so good. But instead of sending an email to postmaster@the-blocked-site they post my IP and a sample of how to use my system to forward spam.
Several days later, on a weekend of course, the spammers started using me. The spammers aren't stupid either. They use the RBL's to find new relays.
I have fixed the problem. However, one small email notification would have prevented several hundred thousand spams. I wonder how many sites have been used this way?
The author of the article is yet another person who misunderstands the problem. The problem is not how to prevent the delivery of spam; that has already been solved. The problem is how to get the ISPs hosting the spammers that continue to eat up our bandwidth to disconnect them from the network. Decent ISPs will just do that upon the discovery they have spammers. And it is acceptable to slap their hand once or even twice, but three spams and you're out. The problem is many ISPs are not decent at all, and will only act upon a financial incentive. Blocking the whole ISP is what is required. DNSBLs such as SPEWS are doing that incrementally with the intent to minimize the number of others affected for long enough to show to the ISP that they had better get rid of the spammers. At this point most ISPs will realize they will lose customers in the future, and will get rid of the spammers. A few will be stubborn, and will eventually have their entire address space listed. Not only do we not want mail from spammers, we don't want mail from anyone who supports spammers. And if you are paying money to an ISP who runs in turn is providing services to a spammer, then you are indirectly supporting spammers through financial benefits, such as the ISP offering the spammers lower rates through economy of scale. And do not forget that if you are doing this, that you and your ISP are benefitting off the costs incurred by others. All this article is, is a reflection of frustration by an individual who just doesn't get it, that he needs to either turn his ISP around to be a decent member of the internet community, or he needs to switch to another ISP. It looks like a lot of work went into it, but the premise being all wrong, the article is worthless and offers no solutions.
now we need to go OSS in diesel cars
I suggest you grow up.
DNSBLs function to block spam, not to punish.
As to who is responsible, an intelligent analysis would reveal that those who herd-like joined the "secure all open relays" crusade without even bothering to read the RFC (2505) that said that was a failed approach are more to blame - they pissed away years that could have been spent in an effective battle against spam (which would have been long gone if that had been done.) Now the herds follow SPEWS - more years of ineffectuality are being risked.
It is smaller ISPs and less technological countries that are to blame? Let me just mention a few entities that stand in stark contradiction to your claim: the United States, Worldcomm (uu.net), Broadwing, Sprint, Verio, Starnet, Rackspace. You gonna tell me that the 50 spam servers Ralsky uses in Dallas are on a smaller ISP? OK, name it - let's start telling them to act. I don't care if it's big or small - name it. I'd like to know.
Still, I agree that the case made against DNSBLs by the web page is weak - too weak to heed. I loudly oppose collateral damage but I see no evidence that it is rampant.
No, your email isn't blocked. Were it blocked, it'd never leave your mail client. Here's what REALLY happens. Your email leaves your mail client, and goes to your ISPs mailserver. You have a contract with them, so they accept it. Then THEY try to send it to us. Now, at this point you're dealing ENTIRELY on OUR hardware, OUR bandwidth, and OUR good graces. Those of us who are SICK AND FUCKING TIRED of having 100x more spam then real mail have quit accepting mail from well-known spammers.
As long as you DIRECTLY support spammers by continuing to use a spam-friendly ISP, your mail will be blocked. Period. You subsudize the rape and pillage of my mailserver and the mindless wasting of my time. And you really have no choice but to move. Wah. Because the alternative is for EVERYONE ELSE ON THE FUCKING INTERNET TO CHANGE THEIR EMAIL ADDRESS EVERY MONTH SO IT'S NOT ON THE SPAMMERS LISTS. DO YOU UNDERSTAND THE COST SHIFTING INVOLVED HERE? IS THIS LOUD ENOUGH TO GET THROUGH?
YOU are DIRECTLY responsible for sending me "Young horny teens get f**ked by a horse with a 31 inch c**k!" (Yes, really *'d out in the message)
Spamassassin is useless. Spammers tune their spams to be under the 3.0... you can't really filter harsher then that without blocking legit mail. The fact that it's open source only makes about a 1 week difference anyway. (Closed filters like hotmail/AOL/earthlink get bypassed in about that long)
The 'bayesian' solution is cute, but dosn't really work beyond an individual level, which means that everyone gets to spend hours sorting through spam (and it still slips through). It also fails because it's looking at single-words. If a friend sends me a mail that includes just 15 poorly chosen words, it gets blocked. If someone implements a two-word version, it may work better.
Add to the fact that a single legit email blocked means you have to read through EVERY spam-marked message looking for more.
So far, the only solution that's made my email workable is whitelisting. And THAT is a lot fucking worse then the RBL. If you're not on my whitelist, you don't talk to me. Period. No Chineese. No Koreans. No Brazilians. No Dutch. No AOL users. Nobody from a small ISP. You're ALL off the net as far as I'm concerned. Nothing that's not a reply to an email I sent. My email is useless for you, but it works for me.
(That's actually an overstatement. I do read the discard folder. Once a week. With the 'd' key. So if you don't invite me to see your webcam, I may read your email.)