The Next Step in Fighting Spam: Greylisting

Evan Harris writes "I've just published a paper on a new and unique spam blocking method called "Greylisting". The best thing about it other than achieving better than 97% effectiveness in blocking spam, is that it practically eliminates the main problem of other solutions: the false-positive. There's even source code for an example implementation written as a perl filter for sendmail, along with instructions for installing, so you can get up and running quickly."

Comments

your first mistake

[frieked]

I'm going to try to say this as nicely as possible and without trolling:
You have just rendered Greylisting pretty useless by making it open source. Spammers are much smarter than you think and what you have basically done is shown them what they need to do in order to get around Greylisting. That's just my take on the issue, maybe I'm wrong but I doubt it.

Re:your first mistake

[Soko]

I'm going to try to say this as nicely as possible and without trolling:

Not trolling at all - you have a legitimate (though perhaps misguided) problem with this method.

You have just rendered Greylisting pretty useless by making it open source. Spammers are much smarter than you think and what you have basically done is shown them what they need to do in order to get around Greylisting. That's just my take on the issue, maybe I'm wrong but I doubt it.

So, the spammers themselves will be of significant help in debugging and helping to fix the code so they can't circumvent it, won't they? OSS means anyone who finds how the greylist script is beaten can figure out a fix and post it. Sounds like the best thing to do IMHO.

Soko

Time critical

[Synithium]

Time critical mailing will go out the window. I can see how this might make any corporate user irate. The same thing goes for challenge-response, the time delay in the business world is unacceptable.

This would be great for personal mail, but that's about it. ISPs would have the same problems with it because their business-class users most likely use the same servers as their consumer-class users.

Re:Time critical

[IncohereD]

How often do you get time critical e-mail from someone you've never recieved e-mail from before?

some guy telling you to BUY THIS NOW != time critical.

your wife telling you to BUY THIS NOW == time critical, and in theory, your wife == whitelisted (or blacklisted, depending on personal preference).

security through obscurity, again?

[dh003i]

If they can get around it by looking at the source, then something was wrong with it, waiting to be exploited. Might as well fix it.

Re:security through obscurity, again?

[SillySlashdotName]

I see that, in fine /. tradition, you didn't RTFA.

From the article: If we have never seen this triplet before, then refuse this delivery and any others that may come within a certain period of time with a temporary failure. (emphasis addded)

Later in the article it goes into much more detail about the delay, how long to delay if the triplet has not been seen before, life time of the whitelist, etc.

It also talks about configuring the times - they mention the default delay is 1 hour, but that their records suggest that 1 minute would have caught 99% of the same spam messages - "The data collected during testing showed that more than 99% of the mail that was blocked with the tested setting of 1 hour would still have been blocked with a delay setting of only 1 minute. At that point, having a larger initial delay will definitely help, as it gives time for other blocking methods to act. For this reason, it is suggested that at least a one hour delay value be kept as a default, since spammers will start adapting as soon as this method becomes known and starts being used. (again, emphasis added)

RTFA!

Re:security through obscurity, again?

[blakestah]

RTFA!

There is no magical waiting period or re-try period that cannot be trivially coded around. And, with good money on the line, will be trivially coded around.

You don't get it. Really smart people are getting paid a whole lot of money to make programs to exploit every possible crack in the way we send email. There is no general rule to spammers, except that it is a lot of money and they are very clever. Little bandaids are not going to stop this one - there needs to be a much more fundamental change. And I am not talking about laws against spam - I am talking about changes in the protocols we use to send email.

spam.....hrmmm

[chef_raekwon]

with all of these solutions to spam..and all of the spam now flooding mail servers...

isn't it time to change the specification (RFC) and possibly the manner in which our current system works? i haven't come up with anything yet, but surely there must be some sort of handshaking/secure type connection that could be used - - some sort of postage (free) that is encrypted into the mail, that states that it is genuine....kind of like the hologram on those windows cds...

i dunno. file this story under redundant.

Poor use of statistics

[GGardner]

The data in this article claims that 1% of all corporate mail servers in the UK allow open relaying, down from 91% in 1997. For all we know, the total number of corporate e-mail servers has grown by a factor of 100 (or more) in the last six year, meaning that perhaps there are more open relays now.

The article also doesn't measure the amount of spam coming through those relays. Even if there are only 10 open relays in the UK at any one time, it still might be possible for all of the spam to be coming through them.

Certainly, closing down open relays is a good thing, but lowering the percentage of open relays doesn't prove anything about the source of spam

Easy for end-users, sure.

[Medievalist]

Just encode your e-mail address on web pages & don't sign up to any dubious mailing lists.

Many of us must maintain contact addresses in the global whois database - so that people can contact us when something is broken.

Look at it this way: you can stop crank calls by unlisting your phone numbers. But you can't unlist the hospital, the ambulance service, the fire department, etc.

We're not all end-users. Some of us are the plumbers.

Delaying email by one hour!

[pjrc]

From the linked paper:

An hour is short enough that in most cases, users will not notice the delay.

I'm wondering how I'm going to explain that to a new customer over the phone who says "I'll just email that file right now so we can go over it together".

Re:Delaying email by one hour!

[vidarh]

Agreed. I've been involed in operating a larger (hundreds of thousands of active users) mail system a couple of years ago, and users would complain if their mail took more than seconds. We had to upgrade our system at one point because rapid growth had made mail delivery take a couple of minutes on average, and it caused bad publicity - a lot of users had a clear expectation that e-mail should be delivered in a few seconds and that if it didn't something was wrong.

I think changing that perception of e-mail as near instant will be incredibly hard. And if you succeed it will just move even more traffic over to the IM networks and cause spamming of IM networks to escalate instead.

One good point about this proposal

[Anonymous Coward]

It deals with spam at the server level. All the wonderful user-level solutions don't do jack to stop spam from being sent. Look at the numbers the spammers show for return rate, and look at how fast spam programs can go, and you'll see that the only solutions that will work are those that make it expensive to send spam. Anything else will just make the spammers send more spam to try and get the hit rate they need.

Re:Published a paper?

[vidarh]

To me publishing a paper in a peer reviewed journal instead of on the web would mean that I'd expect audience to be reduced to a ridiculously small fraction of people that might be interested. If I wanted to publish something I'd do it on the web first, and if it stacks up people I respect would start talking about it and link to it.

Yes, I realize that for "serious" science still expect things to be published in peer reviewed journals, but in most cases I can't help but think that getting the article out there would be more useful. Sure, peer review is important, and somewhere to look for some kind of verification of the value of a paper is useful. But I much prefer the Research Index way, where I can get a good indication of the value of a paper by looking at how many people have cited a paper and WHO have cited a paper.

Anyway, pretending that putting up a document on a website is somehow less publishing a paper than having it printed in a journal, is just plain elitist. You should propably be a bit more critical to papers that are published that you don't know have been through a proper review, especially if you're not a domain expert yourself, but being aware of the source is something that you always need to be.

Re:Bayesian Filtering

[anti$pam]

The key is to make spammers not make money!

If people start adopting anti-spam technologies we would reduce the return spammers get from sending spam. Reduce this enough and the spamming business will no longer be profitable.

POPFile is great. I've also used SAProxy (http://saproxy.bloomba.com/) under windows and it works great too.

Again, the idea is not to eliminate all spam, but to reduce the return rate, and therefore the money made by spammers.