Paul Graham: Filters that Fight Back

Mortimer.CA writes "Paul Graham is back with another article about combating spam. It's entitled Filters that Fight Back: 'One intriguing idea is to literally fight back: to make filters disable spammers' servers by automatically following all the links in each incoming email. We may be driven to this in order to achieve accurate filtering anyway. Why wait?' One danger is someone doing a DDoS by sending fake spam."

Comments

Following links validates your address

[PeekabooCaribou]

If I load an image or a link from spam, it's possible that a spammer could be validating my e-mail address for future sale, or perhaps increased spamming since he knows someone is actually reading the message. For example, http://server.foo/image.gif?id=ab0a98df12j3 could be unique to the spam that was sent to me. If any user-agent accesses that URL, the spammer knows that my e-mail is active and I'm reading his junk. I don't know if they actually do this in practice, but I'm wont to load HTML messages because of it.

Re:Following links validates your address

[hankaholic]

I've been thinking for a while about maybe having a Slashbox that displays images included in spam in a 1x1 pixel box.

Every load of Slashdot would hit spammers' servers.

Comparison of Bayesian spam filters

[kreide33]

I recently switched from a keyword-based spam filter to a bayesian filter. However, there exists several bayesian filter projects and the choice of which to use is not obvious. Therefore, I decided to do an actual test and write up my findings in a review so others can benefit as well. Read it and find out how to win the War on spam.

Filter web-pages through bayesian filterss

[flux]

How about using the bayesian algorithms we have today and apply them to the referred web pages? I'm sure they would have plenty of good material for the filters to detect.. Plus this would propably be more effective with spam that effectively is only an url.

Secondly, I don't call this any kind of DDoS, even though it might seem such to spammers (is slashdotting a DDoS?). If anyone sends me a mail with an url, chances are they _want_ me to check it out. If my system fetches the pages and stores them to a cache, I'm doing exactly what the sender wants. (Mailing lists may be a problem though.)

Thirdly, does it really hurt you to let spammers know that your address is valid? Chances are the address will receive spam nevertheless..

Re:Do they really care?

[Anonymous Coward]

You can have a domain/subdomain with no A records or MX records and they will keep trying. You can also have nothing but blackhole MXs - hosts that don't exist, but are on routable networks. I've had a domain since 1994, and it was in one of the above states for about 2-3 years.

Last month I put a real MX record in there and pointed it at box that's running a mail server. Sure enough, the spam flows continuously. It's not just the "make up random shit and put @aol.com" idiots either - the big outfits with permanent networks and domains are mailing it too.

I've taught my mail server to quarantine any host that attempts to mail my long-dead domain, so having it go to a routable address is actually useful again. Every attempt they make ruins another open proxy or relay for every other spammer that may find it later.

You might consider using those "never valid/previous owner" accounts as spam traps. Anything coming to them now is obviously worthless, so why not make them suffer for trying?

Thoughts on active countermeasures and relays...

[atcroft]

Just finished reading the section of the article that was headed as "Filters that fight back." I think that the biggest issues that keep such an approach from working are fundamental features of the e-mail infrastructure itself: 1) the lack of verification, and 2) the store-and-forward and replicative nature of email itself.

In other systems I am aware of in which active countermeasures may appear (such as firewalls, and tcpwrappers), the adversary can be established with reasonable certainty in most cases; however, because the From and Reply-To addresses can be (and often are) forged and most owners of relaying machines are unaware they are misconfigured, it seems doubtful countermeasures would work at that step. If one uses the URLs, as suggested in the article, it is not guaranteed that the "million" emails sent out will hit the next server along their path at a particular time, so it seems doubtful you can guarantee a massive traffic burst at once. Indeed, what may be seen instead is incremental bursts of traffic at the delivery retry intervals of various mailserver software.

Other questions also arise, such as: 1) how much additional load will a mailserver experience from hitting the links; 2) what additional security issues are introduced in doing so (what if, for instance, the code to do this results in a security vulnerability); 3) how can it be done in such a way that DDOS attacks against innocent victims can be avoided; and 4) how can you get enough people to both upgrade their systems and cooperate in a useful way to do this. Issues 1 and 2 are probably obvious questions to ask-issues 3 and 4, however, I believe suffer from the same weaknesses as some of the current BL schemes. Also, some localities have legal codes which prohibit the interruption of legitimate access to a system, and the server in this case definitely has a way to track back to you at that point, which potentially make participants vulnerable to legal or civil actions.

While I admire Mr. Graham and his efforts in the spam-wars, and find it an intriguing idea, I do not think this approach will truly be successful until changes are made to the underpinning email system that may reduce some of the issues mentioned, but hopefully will themselves make an impact on the issue without being too onerous to prevent wide-spread adoption.

The people who PAY spammers would not

[The Monster]

In the situation where the spammer gets paid by hit, the spammer would be rich overnight. But, then the customer might see somthing a little fishy, then start asking questions.

So you're saying that the long-term effect would be to destroy the spammers' business model?

Looking for a downside to this plan . . . still looking . . . Nope. I can't see one.

Paul's good at this stuff, but this is no good...

[wavecoder]

The way I see it, these are the beefs people have:

• Multiplies bandwidth exponentially, automatically. Big corporations, especially, would be hacked off by this, and it has the added downside of slowing whole sections of the net (imagine what happens when a college dorm gets hit and 800 little bots go check out the site 57 times...).
• Accidental DDoS on good sites - yes, Victoria, spam can be spoofed VERY convincingly.
• Accidental DDoS on good sites (2) - if you've ever maintained a mailing list of more than 20 people, you know that, eventually, some idiot complains he/she got spammed, even if they double-opted in. I've been accused of spamming when I was quoted 2/3 of the way into someone else's (double opt-in) message! I know great sites that are blacklisted, out of human stupidity, alone.
• Accidental DDoS on good hosts - imagine the impact on any shared host, or even some virtual hosts, when one bad client mails 5 million spams - before they could react, they could be taken offline!
• Bad programmers (gasp!) - yes, those exist, and some of these filters could really go haywire and start thrashing all sorts of sites.
• Lawyers - IANAL, but I shudder to think what happens the first time Microsoft or Big Blue sues some programmer, because an abused copy of their software took them down for an hour! (What is the M$ site worth, per hour? Too much, for sure.) Granted, the suit should go the other way, but that's another topic.
• Abuse of ISPs - you'd be amazed how many ISPs will pull the plug on paying accounts for even innocent behavior (like sending 1,000 messages on a DSL account in under an hour, even if it's a business and all the messages are unique). This could get a lot of folks kicked offline.

There are probably others... My thought is this - build a really good, Bayesian, SBPH filter like CRM114, and incorporate a "grab questionable sites" option for the "spams of the future," then filter that page as though it were spam. That'll get us all up into the 99.9% range (the noise), and spammers will eventually either (a) go out of business, or (b) only be able to get their messages to the few people that think they're worthwhile, anyway.

My $.02.

-Ed

Sorry, bad idea

[mikeswi]

When my newsletter (confirmed Opt-in for the NANAE people who may be reading) goes out every Tuesday and 8,000 people open it, how am I supposed to deal with these filters DDoSing my site? For that matter, how do I deal with these filters attacking my site when some other newsletter links to it? What do I do when I piss off Ronnie Scelson and he links to every individual page on my site and spams 100,000,000 people with them?

Links are more likely to be found in legitimate email than in spam. We're going to whitelist every single existing domain on Earth, and then remove the bad ones? Do you have any idea how large that list would be and how long it would take to download it to compare with the domains found linked in an email?

Let's say this idea becomes used widely. It will be used as a weapon by the spammers themselves.

1.) Pay-per-click links sent in mass mailings. Spammer gets paid for every link clicked. I'm sure some of the advertisers will get wise, but there will be plenty who just sign the checks without looking deeper.

2.) Ronnie Scelson or Alan Ralsky get pissed at someone who owns a web site (SPEWS perhaps), and send the address to several hundred million people.

For the ISP sysadmins reading, you think it's bad when 20,000 spams land on your mail server? How are you going to like it when each of those 20,000 spams produce 3 or 4 (or 30 or 40) HTTP requests?

Sorry, bad idea. I can't see how the idea of "attack filters" does anything but discredit the whole idea, especially after thousands of perfectly innocent web sites are knocked offline by the sort of malicious software being advocating, or when spammers inevitably abuse it.

SETI@HOME ?

[axxackall]

I think that some sort of SETI approach can be used:

1. your filter recognizes the spam and gets URLs from it;
2. all such URLs are gathered in the central authority and statistically verified (how many filters have claimed the same site);
3. only the most often claimed sites are left in the list, while more rarely claimed sites are considered as claimed by mistake or by the anti-filter attack;
4. people willing to help to fight spam download the screensaver aka SETI@HOME, working at your CPU and net idle time;
5. the screensaver downloads the fresh list of sites to be fought back along with a centrally generated schedule;
6. the filter actually attacks back at the scheduled time points (if it's still the idlle time for client PC), not massively from the individual PC (so it doesn't look suspicious for the individual client *AND* it doesn't create any peak bandwidth problem for the attacker);
7. the spammer's web site is /.ed;

All problems I see resolvable:

• a schedule must be smart to avoid a local bandwidth problem, but still flood the spammer, but with many such screensavers even a smooth atack will be not very smooth when it's multiplied to millions;
• a central authority can be a subject for a counter-attack as well (will it start cyber-wars?), but if the central authority will really decentralized (p2p, SETI, other techs) that it should not be a problem;
• spammers may use some sort of logging, but what can they do with it?
• to avoid if someone will organize the fake claim in order to /. the innocent site, statistics should help - only really massively claimed sites will be counted;

The main idea of the spam is to send email massively on a very low cost. So if the attack will be also very massive, it will increase their cost of operation and at least some of them will go out of business.

Any attmpts of spammers to go through filters will not work, as you can manually submit the spam claim to (what is its name? NOSPAM@HOME?) the central authority. If the amount of such claims will be big enough, then the claimed sites will be included.