Slashdot Mirror


The Next Step in Fighting Spam: Greylisting

Evan Harris writes "I've just published a paper on a new and unique spam blocking method called "Greylisting". The best thing about it other than achieving better than 97% effectiveness in blocking spam, is that it practically eliminates the main problem of other solutions: the false-positive. There's even source code for an example implementation written as a perl filter for sendmail, along with instructions for installing, so you can get up and running quickly."

14 of 481 comments (clear)

  1. can't believe their numbers by sqrt529 · · Score: 5, Informative

    most spam today is sent through open relays. Those relays will simply retry the delivery no matter which software the spammer uses, so the method won't work.

    1. Re:can't believe their numbers by McDutchie · · Score: 5, Informative

      Eh, open relays are soooo 20th century. :) Actually most open relays today are either blocked or closed, and newly installed MTAs are secure against third-party relaying by default, so this spam method is dying out. Most spam today is sent either directly to the receiving MTA, through open proxies, or through formmail.pl and similar exploits.

  2. Tempfailing is not new and unique by HiKarma · · Score: 5, Informative
    This idea isn't so new or unique. It's been discussed a fair bit on the ASRG mailing list under the name "tempfailing".

    First I heard of it was from Landon Noll and Mel Pleasant. It is noted in brief as one of the techniques in this plan to end spam (though their plan, which did include the triplets, is not laid out in full there.)

    It is a worthwhile technique for a little while, and if spammers were rational, would be worthwhile for some time to come. But spammers are not rational, and already this technique is not as useful as would be hoped.

    Do a Google Search for Tempfailing especially in ASRG to see statistics etc.

  3. Time critical by Synithium · · Score: 5, Insightful

    Time critical mailing will go out the window. I can see how this might make any corporate user irate. The same thing goes for challenge-response, the time delay in the business world is unacceptable.

    This would be great for personal mail, but that's about it. ISPs would have the same problems with it because their business-class users most likely use the same servers as their consumer-class users.

  4. spam.....hrmmm by chef_raekwon · · Score: 5, Insightful

    with all of these solutions to spam..and all of the spam now flooding mail servers...

    isn't it time to change the specification (RFC) and possibly the manner in which our current system works? i haven't come up with anything yet, but surely there must be some sort of handshaking/secure type connection that could be used - - some sort of postage (free) that is encrypted into the mail, that states that it is genuine....kind of like the hologram on those windows cds...

    i dunno. file this story under redundant.

    --
    We're like rats, in some experiment! -- George Costanza
  5. Re:your first mistake by Schnapple · · Score: 5, Funny
    You have just rendered Greylisting pretty useless by making it open source.
    You're assuming the spammers can read source code.
  6. I think not by Monoman · · Score: 5, Interesting

    Doels this mean all public crypto algorithims are useless?

    --
    Keep the Classic Slashdot.
  7. Easy for end-users, sure. by Medievalist · · Score: 5, Insightful
    Just encode your e-mail address on web pages & don't sign up to any dubious mailing lists.
    Many of us must maintain contact addresses in the global whois database - so that people can contact us when something is broken.

    Look at it this way: you can stop crank calls by unlisting your phone numbers. But you can't unlist the hospital, the ambulance service, the fire department, etc.

    We're not all end-users. Some of us are the plumbers.
  8. Re:your first mistake by TheCarp · · Score: 5, Informative

    not at all

    Read the paper. Spammers would figure it out eventually. What it buys is what they have to do to get around it.

    It means they have to do retrys...that means spam runs take longer, especially since they have to run...then wait for a locally defined timeout, and run all those addresses again

    AND they have to do it from the same IP.

    This raises their bandwidth profile. It wastes their time... all in all... it raises their cost of doing buisness and cuts into their profit margins.

    It means they will have to upgrade their tools again. It means they get headaches. And of course, the next step is to impliment spam traps that watch activity and see that a spammer is spamming, and promotes them to a blacklist before they can even retry. (oh gee 1000 new greylist triplets from 1 IP in under 5 mins? Set the timeouts for that IP to 12 hours)

    -Steve

    --
    "I opened my eyes, and everything went dark again"
  9. Re:1 false positive is not acceptable. by pclminion · · Score: 5, Interesting
    Wrong. 1 false positive can be acceptable, and in fact is probably better than how things are now.

    At USENIX '03 there was a paper presented on artificial intelligence techniques for spam detection. I can't provide a link since only USENIX members can download the paper (at this point, at least). I was a coauthor of that paper.

    One of the things we've discovered in our research is that some classes of filters (most notably, the one I have been developing along with a few other individuals) are actually more effective at correctly classifying email than humans are. That is to say, you can train the learning algorithm on mostly-correctly-classified data, then re-run it over the training data, and almost miraculously, it discovers all kinds of email in the training set that was incorrectly classified.

    I.e., this filter has discovered mail that I myself incorrectly thought was spam. It's scary, because there's a lot of it.

    To assume that a human will always be 100% accurate at classifying their own email isn't just arrogant, it's plain wrong. Newer filters that will be introduced in the near future might possibly be more accurate than you, a frail human, could ever be.

  10. Delaying email by one hour! by pjrc · · Score: 5, Insightful
    From the linked paper:

    An hour is short enough that in most cases, users will not notice the delay.

    I'm wondering how I'm going to explain that to a new customer over the phone who says "I'll just email that file right now so we can go over it together".

  11. One good point about this proposal by Anonymous Coward · · Score: 5, Insightful

    It deals with spam at the server level. All the wonderful user-level solutions don't do jack to stop spam from being sent. Look at the numbers the spammers show for return rate, and look at how fast spam programs can go, and you'll see that the only solutions that will work are those that make it expensive to send spam. Anything else will just make the spammers send more spam to try and get the hit rate they need.

  12. clever hack for WHOIS contact addresses by phr1 · · Score: 5, Interesting

    The registrar I use (jumpdomain.com) has a clever hack for despamming WHOIS contact email. Basically they change your published contact address once a week. The published address i automatically generated, looks like gibberish, and forwards to your real address. If someone wants to contact you by looking up your address by WHOIS and writing to you, it works fine. But if they add the address to a mailing list, it stops working in a week. That has eliminated almost all my WHOIS spam. Good scheme.

  13. Re:security through obscurity, again? by letxa2000 · · Score: 5, Interesting
    is reject the mails on the greylist after holding the connection for, say, 10 minutes. That will help deter spamming software,

    I doubt it. I would assume the spam software would have a timeout, and I doubt it's ten minutes. If they want to hit-and-run and aren't even willing to make a second delivery attempt when an error code is returned, I doubt they're going to wait 10 minutes. I'm sure that within 30 seconds or less they'll consider it a dead connection and hang up.

    Problem is, I used to have my sendmail HANG UP in real-time on an incoming connection as soon as it realized a message was spam. I.e., the incoming message was filtered in the DATA phase and if it was spam I hung up immediately. It worked great and it felt good, but there were many spam programs that took the disconnection as some kind of TCP/IP failure and immediatelty tried again. So I had one day where a single message was attempted to be delivered about 30,000 times as the spammer connected, I hung up, spammer software said "Oops, let me try again!" About one delivery attempt every second or so.

    I'd be willing to bet if you put a 10 minute timeout in sendmail you'll see lots of spammer software disconnecting sooner and just trying again. It takes more of their resources, but takes more of yours, too.