Using Statistics to Cause Spammers Pain

mlamb writes "Statistical mail classifiers like PopFile save time on the part of their users, but don't do anything to actively combat spam. I just published an article that suggests a way to use classifier output against a spammer while they're connected to your SMTP server, and I'm launching a project called TarProxy to implement it."

Comments

Anti-Spam software

[Visaris]

This may be just a little off topic, but the thing is that I always have to go through all my mail by hand to make sure I didn't miss anything important anyways. No anti-spam software out there seems to save me this hassle... So to this day I haven't stuck with any. It doesn't look like this will be better.

Quite sad..

[Aliencow]

That we need all these technicalities to try and fight spam... But this is just like people trying to fight piracy, there will always be a new way to get around security. Actually, what we needed was authenticated SMTP from the beginning...

Re:Interesting idea

[TheViciousOverWind]

That would still hurt the spammer alot, since it would take waaay more time for him to send all the spam, instead of just doing it through one big bulb.

Uh...

[jdreed1024]

I just published an article that suggests a way to use classifier output against a spammer while they're connected to your SMTP server,

But, but, but, why would they be connected and sending spam through your server? Unless you run an open relay. And you don't run an open relay, do you? Do you?!

Re:Uh...

[highcaffeine]

In his article he actually does address this very question. He even gives, what I feel at least, is an interesting answer.

So, you don't run an open relay. You're not going to slow down the spammer directly, but you will slow down all the connections that come from that open relay to your mail server. For a particularly abused open relay, that could lead to such problems that the admin of that open relay will finally get a clue and look in to configuring their server properly.

Hence, a cascading effect that will eventually harm the spammers. Admins of open relays that get a clue will tighten their servers, thus depriving the spammers of one more relay they can abuse.

Re:Nice idea

[LowneWulf]

Most mail servers will only forward mail from users of their own domain. If the mailserver is sending spam for one of their legitimate users, I feel no pity for them if their server slows down.

If they forward mail from anyone who sends them mail, then they are an open relay, and again, they deserve what they get for leaving an open relay up.

Re:Interesting idea

[Osty]

Two things here. First, this article wasn't about preventing spammers from using your SMTP server as a relay, but in slowing down the reception of mail at the end-point SMTP server. This will ripple up the chain to hurt the spammers by slowing down the relays they use. Second, it doesn't matter whether I get 10 spam emails or 10,000. One of the goals of TarProxy is to be ubiquitous. I may only receive 10 spammy emails, but my running instance of TarProxy will determine that those are of sufficient spamminess to throttle bandwidth to each of those connections. At the same time, you're doing the same on your SMTP server, and Joe over there is, and so is Susie, and so on. If everybody (defined as "a large number of smtp servers", and not necessarily "everybody") is running such a service, the spammers will be hurt. You're right that a single individual using this won't make much difference, but that didn't seem to be the goal of the article.

Parallel

[Spazmania]

Nonsense. The spammer will just run the connections in parallel. The slower they get the more he'll run. He already does this to some extent. All this will accomplish is to tie up resources on YOUR mail server.

Re:Parallel

[letxa2000]

I more or less agree. I actually tried this approach about a year and a half ago. I modified my Sendmail server to analyze incoming mail during the DATA phase of the SMTP connection. While it was just a simple text filter rather than a cool Bayesian approach, the idea was the same: Cause pain to the spammer because even if I filter the spam before I see it, the spammer has already done his damage. The problem is you can't really do anything to slow him down once they're on the DATA phase and the data is coming in because there is no handshaking at that point. So all I did was have Sendmail close the connection as soon as it recognized something that was sure to be spam.

I gave up on this approach. While there was a satisfaction in looking at my message log and seeing all the spam I had hung up on, spammers would often just keep trying to deliver. Some of the worst software would try a second or two after I hung up on them so they literally pounded my system. It didn't cause any problems except for a little bit of bandwidth, but it certainly didn't seem to phase the spammer.

The fact is, there's not much you can technically do to hurt the spammer. Even if everyone implements this there's no reason why spam software can't open up hundreds of tasks running in parallel and simply be patient when necessary. It could even make spam worse because spam software might evolve to where it DOES send spam out in parallel hundreds at a time by default [forgive me if this is already the case, I have no idea what capabilities spam software has].

The fact is, the only way to make spam go away is to make the response rate go down. This approach gives you, as the admin, a certain satisfaction but it really won't reduce spam--it'll just make spam software more advanced. The only way to make the response rate go down is make sure the spam doesn't get to the user, and that's filtering. Feel free to implement this system, but once the thrill of sticking it to some spammers gets old you'll be back to where you were--with the filters doing the real work.

Re:Parallel

[WolfWithoutAClause]

All this will accomplish is to tie up resources on YOUR mail server.

The spammer already IS tying up 30-50% of the resources on the mail server; if you throttle the bastards back they'll end up using less. What would you prefer a few hundred megs of spam on your hard disk or a few kilobytes of spam that trickled in over a few days till they eventually kill the run. This way you save both bandwidth AND disk space.

Either they use their own server, in which case they're easy to spot. Or they use someone else's- in which case chances are, it isn't engineered for lots of parallel connections.

This scheme may actually work.

but its usually from an open relay...

[TheGratefulNet]

so exactly WHO are you hurting?

sure, the open relay deserves some pain. but you're naieve if you think that most spammers send from their OWN systems!

I have qmail running on my mail hub and I reject mail at the time of connect simply based on the receiver they're trying to send to. when they handshake (part of the HELO exchange) I detect the user they're trying to send to, and since I only have a handful of valid users, its easy to know if they're dictionarying me or not. once I know that, I immediately cut them off, AND add an ipfw (I run freebsd) rule to block all traffic from that IP to my port 25. not only do they NOT get to send any DATA to me, but they're for now on (until it ages out, automatically) forbidden from even connecting to my box. I know that's harsh but I can be that selective since its mostly just me on my mailhub.

but I don't think for a second that even tarpitting that source IP is punishing the spammer. they've most likely broken into (or found) an open relay and they're routing thru them. they don't even see the 'address not reachable' error due to my firewalling them.

Increase prior probabilities of spams if suspectIP

[EnlightenedDuck]

I mentioned this earlier in the discussion - I'm repeating myself because it also applies here...

Using a list of the spam-sending IP's and Bayesian methods, one could assign a high prior probability of a message being Spam. The affect would be to slow down the connection on less evidence if its from a suspect IP address, and to require more evidence if its from an IP address that you trust. Thus you preferentially slow-down suspect computers, and allow your friends to get away with more spam-like messages before tarring them.

Re:Nice idea

[Snowgen]

what if the spammer sends a message to a (good) SMTP server which haven't got the system, and the SMTP server in turn tries to deliver the "spammail" to the right SMTP server, won't that hurt the good SMTP server, who just tries to do it's job?

The situation you're describing is called relaying.

If you start with the assumption that spammers are evil, then the logical conclusion is that there is no such thing as a "good" SMTP server that would relay mail on a spammer's behalf. Servers that do are either in collusion with the spammer, or are mis-configured to allow anonymous relaying. A server that willingly acts in collusion with evil is, by definition, evil. The level of stupidity necessary to allow your sever to act as an open relay also, by definition, precludes being considered a "good" server.

So the short answer to your query is that it's a non-issue. A truly good server will, by definition, never relay spam!

What the hey

[Fished]

Okay, I think you've got what to do down - this is a great idea. The problem is, when to use it?

Here's what I propose: setup a large number of bogus email accounts. Broadcast them everywhere, and let them be honey-pots for spam. The point is, since you NEVER use this account for anything but dropping in spammable places, anything you receive on it *must* be spam. As soon as you get a connection from a mail server to one of these addresses, you *know* it's an open relay, and you put it in your database -- automatically, with no interaction required.

Step 2: You also do a "fingerprint" on the spam you get in your honeypot (you know the routine - what's the length, average use of the word "dildo", etc) so that you can identify this particular spam "copy" by the message -- NOT the header. This allows you to automatically filter out spam messages. If the spammers want to adapt, they have to rewrite their copy. As long as your signature algorithm is fairly lose -- that is, not a true hash algorithm -- they should have to do a total rewrite if they don't want to be detected. You can then filter these at the relays. Thus, once again, you raise the cost for them to do their spam. Since you are filtering by actual known-spam content -- that is, you're doing this like they do virus signatures -- you should get virtually no false positives.

And, anybody whose friends who are emailing them about penis enlargement doesn't really deserve email anyway.

Anyway, there's step 1 and 2. To summarize:

1. Lag spammers.
2. Filter spammers.
3. ????
4. Profit - and make sure to send me some.

Re:Interesting idea

[ATMAvatar]

As I understand the system, it is meant for those receiving spam, not those unwittingly relaying it. The basic idea is that the laggier the network, the longer it takes to send a message. So if your mailserver pretends to be laggy, it will take more time for a computer to send Spam. Thus, less spam is sent. It has the added advantage of since it accepts every message (though it takes longer if it thinks the message is spam), there is no cost to the user for false positives.

Nope - you missed what the article was saying. The mailserver being used by the spammer would be slowed down.

I propose that the running probability from the classifier be used to throttle the connection with the offending server. If an incoming message looks like spam [1], the connection could be slowed dramatically, consuming the spammer's resources and wasting their time [2].

"Throttling" is when you send ICMP choke packets to a sender, which in turn tells the connection to stop sending so many packets. It's generally used to tell a sender that you cannot handle the number of messages it is sending.

Now, what this article proposes is that mailservers use software that statistically analyzes messages, and based upon the likelihood of a message being spam, may send choke packets to the sender. You essentially spam the smammer with choke packets until the spammer's SMTP connection slows to a crawl.

At this point, the spammer can either deal with sending *maybe* a small handful of emails at a time, or give up on spamming. For those businesses that make money off spamming, this would destroy their ability to make any decent money.

Just another stage in the arms race

[zatz]

If these tarpits were ubiquitous, they could completely change the economics of spam, creating a scarcity of bandwidth experienced only by spammers.

Err, I don't think so. This just requires spammers to use more simultaneous connections to overcome the slowdown; it doesn't really increase their network requirements much, only their host CPU requirements. 20,000 simultaneous TCP connections from one process is quite possible with /dev/kqueue under FreeBSD, for example; and you can do the same, but with a bit more CPU wasted, using plain old select() on almost any Unix.

I also don't understand the rationale behind processing the message incrementally. Why not just do your processing before sending back the final 2xx response to the DATA command? Most spam software does not hang up right after sending the final "\r\n.\r\n" from what I've heard from people who run tarpits.

How about this instead: when you are confident you are receiving spam, you stop reading from the socket entirely, and send perhaps 10MB of data back on the other side of the connection. (If the other endpoint isn't reading, and consequently you can only send one window worth of data, then do something to get your TCP stack to generate a lot of useless ACKs, or send your trash back one octet at a time and push between them, or something.) The intent being that sending spam to a large number of MTAs configured in this manner rapidly just becomes a way to DDOS *yourself*. Probably this is too disruptive for most sites to want to bother implementing, though :(

I don't know exactly what the profit margin for spammers is like, but I'm not convinced a small multiplier in network costs is going to matter. Anyway, a lot of these "countermeasures" are mostly going to hurt maintainers of open relays, but if that means they actually fix them, I suppose that is almost as good.

Re:OpenBSD Spam Blocking Engine

[mindriot]

in connection with an IP number list (much faster, so suitable for busy servers)

Another big advantage of going by IP numbers is simply this: I have an IMAP mail account at my university that I use, but I have some external Email addresses as well, which are configured to forward their mail to the university server. Now, if the university's server will add tar based on the message content, I suppose the external mail provider will not be too happy about being slowed down. I would suppose there are quite a number of users simply forwarding mails from one account to another. Maybe (depending on how many people actually use automatic forwarding capabilities) "innocent" servers could be slowed down due to forwarding mail to a "dynamic tarpit", and maybe there are some providers that would not be too happy about such stuff... on the other hand, tarpitting by IP lists seems a little more practical then. But I suppose only practice will show which works best.

Why use the statistics? Throttle it all!

[drf5n]

Do the statistics on 'spamminness' really improve the system? Wouldn't it be easier to throttle all the email to a site-adjustable rate, and have the same effect on the spammers? The ease of implementation would increase the ubiquity, and it would increase the hardware/software requirements of those who mail massively.

For example, if your machine only receives a small amount of email per day, why not throttle them to take 10-20 minutes of connect time overall? If you only get two emails per day (one real and one spam), getting them 10 minutes later probably won't bother you too much, but could cost the spammer or his relay-helpers a 5 minute duration on a connection.

I receive about a hundred emails per day from a number of sources, and adding six to sixty seconds of delay per email wouldn't cause me any grief. But if everyone throttled their email, it might cause someone using their '250 million Valid! Tested! Opt-In!' email lists to have to upgrade their machine to half a million connections to process it in an hour.

I don't see that differential throttling has any benefit over a contant throttling rate. For a big site, the differentiation between spam and not-spam would probably cost you any load advantage you earned in slowing the spam, and for a small system, the delay would not be noticable.

Of course, big senders like AOL, prodigy, and yahoo, might have to upgrade...