Slashdot Mirror
Using Statistics to Cause Spammers Pain
mlamb writes "Statistical mail classifiers like PopFile save time on the part of their users, but don't do anything to actively combat spam. I just published an article that suggests a way to use classifier output against a spammer while they're connected to your SMTP server, and I'm launching a project called TarProxy to implement it."
Just one question... what if the spammer doesn't connect to your SMTP server to send billions of messages from it? What if the spammer (with half a brain, and some scripting ability), only sends a few emails through your SMTP server? Most SMTP servers are wide open still, and simply sending 10 emails on one server and moving on to another open server would be so low that statistical usage wouldn't show anything on the radar screen... or did I not understand what you are trying to do?
---
Programming is like sex... Make one mistake and support it the rest of your life.
The hurt-back part of the project is not new. Theo de Raadt is working on just that, in connection with an IP number list (much faster, so suitable for busy servers):
Very simply, this hangs the full list of ~12,000 spam-sending IP/mask entries listed at www.spews.org off a pf(4) rdr-anchor (which is only entered for port 25). When connections from these spammers arrive they are redirected to a daemon which minimally fakes the SMTP protocol with very low overhead -- for multiple connections at the same time -- and then the message is left on the sender's queue by providing a 550 return code.
The theory here is that most spam still comes in via open relays, and the only way we are going to convince them to clean up their act is to waste _their_ disk space, their time, and their network bandwidth more than they waste ours. For those spammers who drop messages when they received a 550, well, we have not wasted any further time or network bandwidth, and even in that situation I think some of the might remove an address if they receive a 550.
I've been using bogofilter for a while now as a pass-through tagging mechanism. I filter on the client side based on the tag information. This sounds a lot like what you are doing.
The only thing close to a false positive I've gotten was having to dumpster dive into my spam folder to retrieve an amazon order confirmation.
Bayesian filtering really works, but you have to train the filter correctly and with as large a corpus as possible.
But then the only way the actual spammer would be sending from your server is if you have an open relay? So the idea would be to set up false open relays? But wouldn't the spammer just black/whitelist the servers? The place where I work once got hit by a spammer, (because we used some matt formmail script), it all happened automatic in steps: - Some webspider found out about the formmail.cgi - The spider sends a mail to some hotmail account - 15 minutes later (I guess after confirming the mail got through) it started sending mails non-stop. - 30 minutes later, we could see some other type of traffic (The bot apparently sent out mails about the open relay to other spammers (possible persons who bought access to the open relays?)). All the while we were on the phone with the police computer-crime department, which didn't know what to do. Then we denied those users access to the network and patched up the security breach (We were waiting to do that, while talking to the police, in the hope that they could actually do something, since the spammer were spamming "right now"... But apparently they were quite clueless).
My <1000 UID is with a hot chick
Instead, this is meant to be run on the incoming SMTP server, the one that receives the mail. It will only hurt the spammer if he's trying to send a bunch of spam to your domain, but every server running this can help.
"Evil company X is threatening to restrict our rights! Let's all get together to stop--OOOH! SHINEY!!!" -- AC
The easiest solution is to have no open relays. I know I know, it ain't gonna happen, but perhaps this could convince more of those relays to close their doors:
What we do is have a small app that plugs into eudora, outlook, evolution, kmail etc. Whenever you get a spam, you click a button, it scans the header, finds the smtp server that sent the spam and then sends them 1 email informing them of the fact that they are sending spam (of course you need a way of getting the sysadmin's email address).
If enough people did this then the bad relays would be swamped with emails informing them of the spam they've been relaying, and they might close their relay. And non-open relays that just allow spammers to spam might think about being less friendly to spammers.
What do people think, is it lame?
--- I used to moderate, then I read the -1 articles and decided having to filter through them was not worth it.
Easy to defeat, just use spamming software that dynamically increases it's connection pool whenever it encounters a 'slow' SMTP recipient. Even if a large part of the net population were running this, the spammer could just spawn thousands of simultanious (slowed down, yes) connections, and still maximize his bandwidth utilization. If it takes 2 minutes to send each message, it dosen't matter if he's sending 5000 messages at once!
I believe linux, for example, allows up to 8192 open sockets, and I think this can be changes with a sysctl command, and most definitely could be with a few changes to kernel headers.
Sure, it would take a machine with decent memory, but that's not too hard to find.
---
the pen is mightier than the sword, the sword is mightier than the court, the court is mightier than the pen.
I have several domain names that appear on many of the "million address" CDs and other popular spam lists, but which longer any legitimate recipients/users.
We are also working on obtaining access to true "realtime" RBL lists of currently abused open relay servers. Assistance would be appreciated.
The core of "stations of the cross" is a custom DNS server. This server is authoritative for these oft-spammed domains, and each time a request is made for an MX record, it returns (with a short TTL) a unique randomly generated list of MXes, each address on the list being a known open relay.
So when a spammer or relay first goes to deliver a message, the system will select an open relay off the list of MXes, and hands off the message to that host. Being an open relay, the host accepts the message for my domain, then goes to do a DNS lookup for the MX record. The relay receives a (different) list of other open relays...
Usually, you can get a message to traverse a dozen or more open relays (most sendmail systems default to a maximum "hop count" of 25), after which the message will bounce.
Since the only traffic my server has to deal with is DNS queries and responses, this is very low-overhead for me, but depending on the size of the spammail, very high overhead for the open relay servers.
I do not deploy Linux. Ever.
- Free: It's no good unless it's everywhere... or at least in lots of places. TarProxy is Open Source Software released under a BSD-style license and available on SourceForge (see project page for details).
- Platform Independent: TarProxy is written in Java, so it runs on Linux, Windows, Solaris, OS X, and any other operating system with a Java Virtual Machine available.
contradict one another, and therefore directly suggest incipient failure. Any program you want widely deployed had better not depend on having some buggy JVM installed.(Arguably that is the reason that Freenet has been a practical failure. Every time I have tried to use it, it has got stuck in an infinite loop, or consumed all my swap space, or crashed. I blame buggy JVMs.)
If you want software to be widely and successfully deployed, it should (must!) resemble the software that already has been. Almost all such code (99%+) has been in C or in C++. Are there any Free Software programs written in Java successfully deployed outside of Java development shops? (Rhetorical question; the answer is "not enough to matter".)
If you want portability to Unixes, to w32, and to Macosix, you already get that with Gcc and autoconf.
If it's in Java, I certainly won't run it as a daemon.
A 550 error is a permanent reject. The spam source knows that the mail cannot be delivered so it quits. A 450 error tells the connecting smtp server that your server is temporarily unable to deliver the mail, but that it's not a fatal error and delivery should be retried. This is much more likely to keep the message in the spammer's mail queue.
Personally, the spam solution I like the best is to have procmail+formail or some other tool sitting on your mail server and making unknown senders go through a confirmation step. It doesn't work for everyone (for instance, people expecting email replies to résumés! NAGI...), but if it works for you it tends to work very well. It inconveniences everyone else, but hey, everyone else is not me. I can whitelist all the people I truly care about.
Either that or we should throw out SMTP, email RFCs, sendmail, etc. and build a spam-free system from the ground up. Yeah, right.
Washington, DC: It's like Hollywood for ugly people.
> > Unfortunately the critical mass for this to really work is
> > very, very large.
Yes, it is large.
> I don't think this is necessarily true. As the article points
> out, setting it up on a few servers would be sufficient to get
> things started provided those few servers were the right ones.
Let me guess: Yahoo's several dozen, AOL's however many, and
the ones at Earthlink, demon.co.uk, and MSN -- and I close?
That's a very large critical mass, not in terms of the number of
servers, but in terms of the amount of mail handled (and, therefore,
the amount of server beef needed to implement any such measures).
> I don't think they should be doing this in Java though. Java is
> not a text parsing language and this thing really requires some
> text parsing muscle. Cross platform ability isn't as important.
No need to sacrifice the cross-platformness. Perl is a GREAT
text processing language, performs faster than Java, and as an
added bonus is much more cross-platform (provided you don't need
a GUI (which for this you don't)). It does use quite a bit of
RAM sometimes, but so does Java. And doing SMTP stuff in Perl
is really easy. (Net::SMTP rocks in a significant way.) And
any operating system that's remotely appropriate for use as a
mail server probably comes with Perl out of the box these days.
Cut that out, or I will ship you to Norilsk in a box.
There's a few spammers who send direct from their own IPs. If you want to tarpit them just tarpit the traffic from their Ips - you don't need to analyze anything.
9 0]"A ni[!--HVtu--]ce la[!--HVtu--]dy
- ]im[!--WPVizB--]ited
For other spam, through open proxies or open relays, you are not hurting the spammer to tarpit. If the spammer is working through open proxies and if you got enough tarpits going then you could hurt them, but until there's enough tarpits there is still zero (0.000) percent pain to the spammer. Some open proxes are slow with one or two tarpits, the others are fast enough to keep the spammer's server fully busy. He only cares if he's running his server flat out. Delays at one or more open proxies mean little.
Right now I'm trapping spam on a relay spam honeypot. It comes to the honeypot from open proxies - theer's nothig I can learn about the spammer by learning about the proxies. It comes (usually) as 99-recipient spam messages. This particular spammer uses imbedded comments in his spam to evade Bayesian filters. Makes no difference to me - I see it is spam. I have no valid email to filter out - everything is spam. That's one of the beauties o a honeypot - the spammer does yor filtering for you.
Somewhere over 20,000 recipients so far, since Wednesday. Here's a tiny sample, showing the URL's he advertises and the random comments he uses to defeat filters:
[a href="http://www.directmailorderbrides.com/?oc=23
[a href="http://www.flati.com/silagra/"]L[!--WPVizB-
(I replaced agle brackets with square brackets - tou'll have to imagine them restored.)
I have no filter, no smarts of any kind. The honeypot is a mail server with the output queue stopped. I got the spammer to start sendng spam by delivering to him three of his relay test messages - he'd sent so many I decided to see who he was, what spam I'd get if I did deliver.
I'm trying various ways to hurt the spammer but I've not yet delivered enough hurt - he's still operating. Other spammers have succumed more readily - this guy is better at hiding himself.
Note, by the way, that he puts no comments in the URL - if you filter on those (or remove comments before filtering - that would be easy) the spam instantly is revealed. One guy simply rejects any email message with three repeated comments in a line (this spam is laced with the comments throughout, not just in the http lines.) The spammer's clever way of obscuring the spam is useful in identifying the spam - no points for Spammy.
Windows users with a permanent connection can step into running a relay spam honeypot very easily: they can run Jackpot: http://jackpot.uk.net/
There is at least one open proxy honeypot out there: Google in news.admin.net-abuse.email for it. These can be very wicked - create your own for even more fun. Or create your own open relay honeypot - see if you can make it even more wicked.
(Oversize reply packets from an open proxy honeypot might have a very interesting efffect.)
Want to find open relays? Here's a nice simple way I implemented a couple of years ago, and ran for awhile. It's quite simple, and detects single stage relays rather quickly.
Write something that listens on port 25. When it receives a connection, connect back to the calling host on port 25. If the connection attempt succeeds, copy characters back and forth. Anything they send to you, you send to their port 25, and vice-versa.
If it's a true open relay, it will gladly accept the mail over and over again. I had a few mail servers looping THOUSANDS of times through me since they didn't check Received: headers. I also realize that it would be trivial to *ahem* "break" the Received: line such that it wouldn't increment the counter.
Granted, that sucks down bandwidth, so back to the point - proving that this is an open relay. What you do is stick a magic header in the message as it heads back to them. If you receive that header back from a host, it's something you've already looped, and they're an open relay.
Now you know they're an open relay, so you can add them to your MX lists. You can also then avoid letting them run through your looper, since it won't provide any more data.
The beauty of this plan is that you're only giving them what they pushed upon you first. If they leave you alone, you leave them alone. It's a nice implementation of a concept I wish more people would honor.
First off, you are incredibly wrong. Almost all spam is bounced off of servers that relay...that is, they forward mail for users of any domain. That's why this concept exists; spammers search for "open relays" (that's why they're called that, btw) and use them. TarProxy would look like a normal open relay to the spammer, and therefore he would use it.
Unfortunately, there is a problem. Before TarProxy there was another thing, called a "teergrube" or "tarpit." What it did was slow down the connection (with things like ICMP source-quench and psychotically small TCP window sizes) so that it acted like a spam speed bump. In the meanwhile, it didn't actually forward any of the spam anyhow. Why didn't this technology become more widespread? I'm glad you asked! Because it was trivial for the guys who develop spammer software to recognize these systems, have their software detect such behavior, and cease using them within less than a minute. And that's what will happen with a TarProxy, alas.
For your security, this post has been encrypted with ROT-13, twice.
It's most fun to do the dirty work against the spammer. What he thinks is an open relay doesn't have to be one.
This one whacked Ralsky hard for several months - Ralsky never caught on: http://www.corpit.ru/cgi-bin/h0n5yp0t
You can do it, too:
http://jackpot.uk.net/
And please do.