Slashdot Mirror
Using Statistics to Cause Spammers Pain
mlamb writes "Statistical mail classifiers like PopFile save time on the part of their users, but don't do anything to actively combat spam. I just published an article that suggests a way to use classifier output against a spammer while they're connected to your SMTP server, and I'm launching a project called TarProxy to implement it."
This may be just a little off topic, but the thing is that I always have to go through all my mail by hand to make sure I didn't miss anything important anyways. No anti-spam software out there seems to save me this hassle... So to this day I haven't stuck with any. It doesn't look like this will be better.
I am a viral sig. Please help me spread.
Just one question... what if the spammer doesn't connect to your SMTP server to send billions of messages from it? What if the spammer (with half a brain, and some scripting ability), only sends a few emails through your SMTP server? Most SMTP servers are wide open still, and simply sending 10 emails on one server and moving on to another open server would be so low that statistical usage wouldn't show anything on the radar screen... or did I not understand what you are trying to do?
---
Programming is like sex... Make one mistake and support it the rest of your life.
Exactly how it should be.
Perhaps public floggings and other corperal punishment as well.
However I have to wonder if all spammers are really sane ... I just got an email about chicks who crave small penis's and those who crave big penis's and then emails about penis enlargement and viagra online purchases, it just seems weird that there is so much concern for my penis. Perhaps we should just imprison them on an island as they might find tar and feathering a bit kinky and enjoy it.
Ignore the "p2p is theft" trolls, they're just uninformed
Most mail servers will only forward mail from users of their own domain. If the mailserver is sending spam for one of their legitimate users, I feel no pity for them if their server slows down.
If they forward mail from anyone who sends them mail, then they are an open relay, and again, they deserve what they get for leaving an open relay up.
TarProxy is written in Java,
Well, that's one way to do it.
The hurt-back part of the project is not new. Theo de Raadt is working on just that, in connection with an IP number list (much faster, so suitable for busy servers):
Very simply, this hangs the full list of ~12,000 spam-sending IP/mask entries listed at www.spews.org off a pf(4) rdr-anchor (which is only entered for port 25). When connections from these spammers arrive they are redirected to a daemon which minimally fakes the SMTP protocol with very low overhead -- for multiple connections at the same time -- and then the message is left on the sender's queue by providing a 550 return code.
The theory here is that most spam still comes in via open relays, and the only way we are going to convince them to clean up their act is to waste _their_ disk space, their time, and their network bandwidth more than they waste ours. For those spammers who drop messages when they received a 550, well, we have not wasted any further time or network bandwidth, and even in that situation I think some of the might remove an address if they receive a 550.
This is the same thing as OpenBSD's spamd, which Theo de Raadt wrote specifically to cause spam relays pain. spamd uses some new features of pf and blacklists from Spews to create a tarpit for incoming messages from known spam relays. It was even discussed on Slashdot in this article. Also, Daniel Hartmeier, pf developer extraordinaire and all around good guy, wrote a little piece about annoying spammers using pf, spamd, and bmf.
I was hoping to get first post, but my connection got throttled back to nothing....
Nonsense. The spammer will just run the connections in parallel. The slower they get the more he'll run. He already does this to some extent. All this will accomplish is to tie up resources on YOUR mail server.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
In his article he actually does address this very question. He even gives, what I feel at least, is an interesting answer.
So, you don't run an open relay. You're not going to slow down the spammer directly, but you will slow down all the connections that come from that open relay to your mail server. For a particularly abused open relay, that could lead to such problems that the admin of that open relay will finally get a clue and look in to configuring their server properly.
Hence, a cascading effect that will eventually harm the spammers. Admins of open relays that get a clue will tighten their servers, thus depriving the spammers of one more relay they can abuse.
so exactly WHO are you hurting?
sure, the open relay deserves some pain. but you're naieve if you think that most spammers send from their OWN systems!
I have qmail running on my mail hub and I reject mail at the time of connect simply based on the receiver they're trying to send to. when they handshake (part of the HELO exchange) I detect the user they're trying to send to, and since I only have a handful of valid users, its easy to know if they're dictionarying me or not. once I know that, I immediately cut them off, AND add an ipfw (I run freebsd) rule to block all traffic from that IP to my port 25. not only do they NOT get to send any DATA to me, but they're for now on (until it ages out, automatically) forbidden from even connecting to my box. I know that's harsh but I can be that selective since its mostly just me on my mailhub.
but I don't think for a second that even tarpitting that source IP is punishing the spammer. they've most likely broken into (or found) an open relay and they're routing thru them. they don't even see the 'address not reachable' error due to my firewalling them.
--
"It is now safe to switch off your computer."
Here are some more spam tarpits:
TarProxy
ChuckMail
OpenBSD's spamd (tarball)
Google Search Results
what if the spammer sends a message to a (good) SMTP server which haven't got the system, and the SMTP server in turn tries to deliver the "spammail" to the right SMTP server, won't that hurt the good SMTP server, who just tries to do it's job?
The situation you're describing is called relaying.
If you start with the assumption that spammers are evil, then the logical conclusion is that there is no such thing as a "good" SMTP server that would relay mail on a spammer's behalf. Servers that do are either in collusion with the spammer, or are mis-configured to allow anonymous relaying. A server that willingly acts in collusion with evil is, by definition, evil. The level of stupidity necessary to allow your sever to act as an open relay also, by definition, precludes being considered a "good" server.
So the short answer to your query is that it's a non-issue. A truly good server will, by definition, never relay spam!
The easiest solution is to have no open relays. I know I know, it ain't gonna happen, but perhaps this could convince more of those relays to close their doors:
What we do is have a small app that plugs into eudora, outlook, evolution, kmail etc. Whenever you get a spam, you click a button, it scans the header, finds the smtp server that sent the spam and then sends them 1 email informing them of the fact that they are sending spam (of course you need a way of getting the sysadmin's email address).
If enough people did this then the bad relays would be swamped with emails informing them of the spam they've been relaying, and they might close their relay. And non-open relays that just allow spammers to spam might think about being less friendly to spammers.
What do people think, is it lame?
--- I used to moderate, then I read the -1 articles and decided having to filter through them was not worth it.
Here's what I propose: setup a large number of bogus email accounts. Broadcast them everywhere, and let them be honey-pots for spam. The point is, since you NEVER use this account for anything but dropping in spammable places, anything you receive on it *must* be spam. As soon as you get a connection from a mail server to one of these addresses, you *know* it's an open relay, and you put it in your database -- automatically, with no interaction required.
Step 2: You also do a "fingerprint" on the spam you get in your honeypot (you know the routine - what's the length, average use of the word "dildo", etc) so that you can identify this particular spam "copy" by the message -- NOT the header. This allows you to automatically filter out spam messages. If the spammers want to adapt, they have to rewrite their copy. As long as your signature algorithm is fairly lose -- that is, not a true hash algorithm -- they should have to do a total rewrite if they don't want to be detected. You can then filter these at the relays. Thus, once again, you raise the cost for them to do their spam. Since you are filtering by actual known-spam content -- that is, you're doing this like they do virus signatures -- you should get virtually no false positives.
And, anybody whose friends who are emailing them about penis enlargement doesn't really deserve email anyway.
Anyway, there's step 1 and 2. To summarize:
"He who would learn astronomy, and other recondite arts, let him go elsewhere. " -- John Calvin, commenting on Genesis 1
If these tarpits were ubiquitous, they could completely change the economics of spam, creating a scarcity of bandwidth experienced only by spammers.
/dev/kqueue under FreeBSD, for example; and you can do the same, but with a bit more CPU wasted, using plain old select() on almost any Unix.
:(
Err, I don't think so. This just requires spammers to use more simultaneous connections to overcome the slowdown; it doesn't really increase their network requirements much, only their host CPU requirements. 20,000 simultaneous TCP connections from one process is quite possible with
I also don't understand the rationale behind processing the message incrementally. Why not just do your processing before sending back the final 2xx response to the DATA command? Most spam software does not hang up right after sending the final "\r\n.\r\n" from what I've heard from people who run tarpits.
How about this instead: when you are confident you are receiving spam, you stop reading from the socket entirely, and send perhaps 10MB of data back on the other side of the connection. (If the other endpoint isn't reading, and consequently you can only send one window worth of data, then do something to get your TCP stack to generate a lot of useless ACKs, or send your trash back one octet at a time and push between them, or something.) The intent being that sending spam to a large number of MTAs configured in this manner rapidly just becomes a way to DDOS *yourself*. Probably this is too disruptive for most sites to want to bother implementing, though
I don't know exactly what the profit margin for spammers is like, but I'm not convinced a small multiplier in network costs is going to matter. Anyway, a lot of these "countermeasures" are mostly going to hurt maintainers of open relays, but if that means they actually fix them, I suppose that is almost as good.
Java: the COBOL of the new millenium.
Great idea! Parse out the URLs, plug 'em into some boilerplate, and automatically submit it as a story to Slashdot! They'll never try THAT again!
Cantankerous old coot since 1957.
Easy to defeat, just use spamming software that dynamically increases it's connection pool whenever it encounters a 'slow' SMTP recipient. Even if a large part of the net population were running this, the spammer could just spawn thousands of simultanious (slowed down, yes) connections, and still maximize his bandwidth utilization. If it takes 2 minutes to send each message, it dosen't matter if he's sending 5000 messages at once!
I believe linux, for example, allows up to 8192 open sockets, and I think this can be changes with a sysctl command, and most definitely could be with a few changes to kernel headers.
Sure, it would take a machine with decent memory, but that's not too hard to find.
---
the pen is mightier than the sword, the sword is mightier than the court, the court is mightier than the pen.
I have several domain names that appear on many of the "million address" CDs and other popular spam lists, but which longer any legitimate recipients/users.
We are also working on obtaining access to true "realtime" RBL lists of currently abused open relay servers. Assistance would be appreciated.
The core of "stations of the cross" is a custom DNS server. This server is authoritative for these oft-spammed domains, and each time a request is made for an MX record, it returns (with a short TTL) a unique randomly generated list of MXes, each address on the list being a known open relay.
So when a spammer or relay first goes to deliver a message, the system will select an open relay off the list of MXes, and hands off the message to that host. Being an open relay, the host accepts the message for my domain, then goes to do a DNS lookup for the MX record. The relay receives a (different) list of other open relays...
Usually, you can get a message to traverse a dozen or more open relays (most sendmail systems default to a maximum "hop count" of 25), after which the message will bounce.
Since the only traffic my server has to deal with is DNS queries and responses, this is very low-overhead for me, but depending on the size of the spammail, very high overhead for the open relay servers.
I do not deploy Linux. Ever.
A 550 error is a permanent reject. The spam source knows that the mail cannot be delivered so it quits. A 450 error tells the connecting smtp server that your server is temporarily unable to deliver the mail, but that it's not a fatal error and delivery should be retried. This is much more likely to keep the message in the spammer's mail queue.
There's a few spammers who send direct from their own IPs. If you want to tarpit them just tarpit the traffic from their Ips - you don't need to analyze anything.
9 0]"A ni[!--HVtu--]ce la[!--HVtu--]dy
- ]im[!--WPVizB--]ited
For other spam, through open proxies or open relays, you are not hurting the spammer to tarpit. If the spammer is working through open proxies and if you got enough tarpits going then you could hurt them, but until there's enough tarpits there is still zero (0.000) percent pain to the spammer. Some open proxes are slow with one or two tarpits, the others are fast enough to keep the spammer's server fully busy. He only cares if he's running his server flat out. Delays at one or more open proxies mean little.
Right now I'm trapping spam on a relay spam honeypot. It comes to the honeypot from open proxies - theer's nothig I can learn about the spammer by learning about the proxies. It comes (usually) as 99-recipient spam messages. This particular spammer uses imbedded comments in his spam to evade Bayesian filters. Makes no difference to me - I see it is spam. I have no valid email to filter out - everything is spam. That's one of the beauties o a honeypot - the spammer does yor filtering for you.
Somewhere over 20,000 recipients so far, since Wednesday. Here's a tiny sample, showing the URL's he advertises and the random comments he uses to defeat filters:
[a href="http://www.directmailorderbrides.com/?oc=23
[a href="http://www.flati.com/silagra/"]L[!--WPVizB-
(I replaced agle brackets with square brackets - tou'll have to imagine them restored.)
I have no filter, no smarts of any kind. The honeypot is a mail server with the output queue stopped. I got the spammer to start sendng spam by delivering to him three of his relay test messages - he'd sent so many I decided to see who he was, what spam I'd get if I did deliver.
I'm trying various ways to hurt the spammer but I've not yet delivered enough hurt - he's still operating. Other spammers have succumed more readily - this guy is better at hiding himself.
Note, by the way, that he puts no comments in the URL - if you filter on those (or remove comments before filtering - that would be easy) the spam instantly is revealed. One guy simply rejects any email message with three repeated comments in a line (this spam is laced with the comments throughout, not just in the http lines.) The spammer's clever way of obscuring the spam is useful in identifying the spam - no points for Spammy.
Windows users with a permanent connection can step into running a relay spam honeypot very easily: they can run Jackpot: http://jackpot.uk.net/
There is at least one open proxy honeypot out there: Google in news.admin.net-abuse.email for it. These can be very wicked - create your own for even more fun. Or create your own open relay honeypot - see if you can make it even more wicked.
(Oversize reply packets from an open proxy honeypot might have a very interesting efffect.)
For example, if your machine only receives a small amount of email per day, why not throttle them to take 10-20 minutes of connect time overall? If you only get two emails per day (one real and one spam), getting them 10 minutes later probably won't bother you too much, but could cost the spammer or his relay-helpers a 5 minute duration on a connection.
I receive about a hundred emails per day from a number of sources, and adding six to sixty seconds of delay per email wouldn't cause me any grief. But if everyone throttled their email, it might cause someone using their '250 million Valid! Tested! Opt-In!' email lists to have to upgrade their machine to half a million connections to process it in an hour.
I don't see that differential throttling has any benefit over a contant throttling rate. For a big site, the differentiation between spam and not-spam would probably cost you any load advantage you earned in slowing the spam, and for a small system, the delay would not be noticable.
Of course, big senders like AOL, prodigy, and yahoo, might have to upgrade...
Want to find open relays? Here's a nice simple way I implemented a couple of years ago, and ran for awhile. It's quite simple, and detects single stage relays rather quickly.
Write something that listens on port 25. When it receives a connection, connect back to the calling host on port 25. If the connection attempt succeeds, copy characters back and forth. Anything they send to you, you send to their port 25, and vice-versa.
If it's a true open relay, it will gladly accept the mail over and over again. I had a few mail servers looping THOUSANDS of times through me since they didn't check Received: headers. I also realize that it would be trivial to *ahem* "break" the Received: line such that it wouldn't increment the counter.
Granted, that sucks down bandwidth, so back to the point - proving that this is an open relay. What you do is stick a magic header in the message as it heads back to them. If you receive that header back from a host, it's something you've already looped, and they're an open relay.
Now you know they're an open relay, so you can add them to your MX lists. You can also then avoid letting them run through your looper, since it won't provide any more data.
The beauty of this plan is that you're only giving them what they pushed upon you first. If they leave you alone, you leave them alone. It's a nice implementation of a concept I wish more people would honor.
Read about a method to get SpamAssassin to execute at SMTP time in exim (I'm about to impliment this on my own mailserver) and read about teergrubing which is basically the same idea as a tarpit.
Unlike the original post, Marc seems to have a stable working version of this right now.
That said, this is probably the most realistic method of causing spammers pain that we have right now, short of changing the way mail works in a fundamental manner.
I'll definately be implimenting teergrubing/tarpitting. I might even impliment it on the multi-user hosting system that I helped to build. It probably wouldn't scale too well on a busy site though
I'm going back to splinter cell.
First off, you are incredibly wrong. Almost all spam is bounced off of servers that relay...that is, they forward mail for users of any domain. That's why this concept exists; spammers search for "open relays" (that's why they're called that, btw) and use them. TarProxy would look like a normal open relay to the spammer, and therefore he would use it.
Unfortunately, there is a problem. Before TarProxy there was another thing, called a "teergrube" or "tarpit." What it did was slow down the connection (with things like ICMP source-quench and psychotically small TCP window sizes) so that it acted like a spam speed bump. In the meanwhile, it didn't actually forward any of the spam anyhow. Why didn't this technology become more widespread? I'm glad you asked! Because it was trivial for the guys who develop spammer software to recognize these systems, have their software detect such behavior, and cease using them within less than a minute. And that's what will happen with a TarProxy, alas.
For your security, this post has been encrypted with ROT-13, twice.