Slashdot Mirror


Reviewing Anti-Spam Offerings

Joel Snyder writes "Just finished looking into the innards of 40+ anti-spam products at Network World. The biggest, ugliest, and most comprehensive look at this market that's ever been done. Conclusions: lots of great products to choose from at the top (a dozen or more); a few stinkers in the bunch; and it's basically impossible to review Spam Assassin, which is unfortunate."

21 of 311 comments (clear)

  1. Thunderbird by ack154 · · Score: 1, Interesting

    I find that Mozilla's Thunderbird has excellent anti-spam control. That's just from my own "testing" though...

    1. Re:Thunderbird by casuist99 · · Score: 2, Interesting

      I certainly do get a lot of false positives with thunderbird's spam-controls, and would really like an interface through which I can view the filtering logs (words, frequency, etc) that thunderbird must be creating.

      A reporting feature (even if thunderbird just exports a database csv file) would provide more value to me. I'd also like to be able to transfer my thunderbird spam filtering profile to new installations (after reformatting, for example).

      A lot of other packages (e.g. spamassassin) support some of these, but I see no reason that thunderbird couldn't try to include some of them too.

  2. In-line SPAM filtering - never hits your server by dj42 · · Score: 3, Interesting

    Mine isn't in the list.... http://www.mxlogic.com

    I have said it before on here, but I use Mx-logic.com to filter e-mail before it even gets to my mail server (as their filtering is in-line). They run multiple concurrent virus scanners, and you can set all policies related to attachments, sizes, virus scanning, quarantines, SPAM (deny, accept, etc, for different "levels" of probability).

    It's really efficient. I haven't gotten a virus in any attachments and maybe just 2-3 SPAM messages / month (down from 100+ / day). It also does cool stuff like remove the imbedded tracking images from SPAM HTML messages (should one get through), etc. No, I don't work for them. I used to quarantine messages and review it weekly (that were medium / high probability spam), now I trust their service so much I just deny receipt to my mail server of any Medium+ probability SPAM

    --
    We are one consciousness experiencing itself subjectively. Back to you with the weather, Bob!
    1. Re:In-line SPAM filtering - never hits your server by Not_Wiggins · · Score: 2, Interesting


      Only problem with in-line scanning is the time/resource it takes to do it.

      While great for low-volume mail servers, you really need a beefy box to enable you to have enough MTA threads for handling the initial SMTP communication, threads for doing the virus scanning/spam filtering, and CPU to do it in the time allowed by the SMTP standard (I *think* it is 180 seconds... probably enough time).

      I don't know if there's an advantage to not accepting virus-laden mail as one can biff it "off line" without inviting more infection attempts (ie, after the message is accepted by the transport).

      But, there's plenty of good reason to do spam filtering at that point: reject the message before you even store it, so your server doesn't even have to bother with trying to deliver to forged bounce addresses.

      Last problem with that, however, is attack through backup MX host, but... I'm starting to digress.

      Postfix has great integration for smtp proxies with their Before Queue Content Filter.

      (That's not for the author, but for others who might want to learn more about plugging his suggestion into their mail server, or upgrading to an MTA that supports it).

      --
      Diplomacy is the art of saying, "Nice doggie!" until you can find a rock.
  3. Re:SpamAssassin? by chadpnet · · Score: 1, Interesting

    I second that. Any real comprehensive review would include some sort of mention of Spam Assassin. Yes, it's highly configurable and has plural avenues of use, but I think that's what makes it even neater.

  4. Re:SpamAssassin? by chadpnet · · Score: 1, Interesting

    His comment is still valid. That's like saying "We did a comprehensive review of the leading web servers, IIS, PWS, and Netscape's baby. We recognize that Apache exists, however we couldn't review it because we couldn't figure out how to get it to work.

  5. Spamassassin by confusion · · Score: 4, Interesting
    All-in-all, they didn't blast spamassassin as hard as I thought they were going to. It was sad to see that they didn't think they could get anyone to help them review SA, and it was sadder that they got a lot of the facts wrong about SA, like that it is built around a bayes database.

    The mere appearance of SA, though, is impressive because those trade rags rarely include anything open source (partly due to marketing opportunity for commercial, paying companies).

    Jerry http://www.syslog.org/

  6. Just regurgitating marketing numbers by Anonymous Coward · · Score: 4, Interesting

    The buying guide is useful just for putting all the contenders together. But don't believe the claims until you test them. Barracuda, for example, touts the capability of millions of messages a day, but we are sending our second test unit back because it just can't handle a modest load of real world mail. Their 600, for example, claims it can process "25 million messages per day" but that assumes it is rejecting 95% of the mail -- that's nowhere in their fine print.

    1. Re:Just regurgitating marketing numbers by joel_snyder · · Score: 2, Interesting

      The buyer's guide definitely is just pure marketing numbers. The article gives more realistic performance numbers, and that exposes some of what you're bringing up in the text. I found exactly what you're reporting (and mention it as an issue): vendors advertise based on 'oh, yeah, we throw out 50% of the mail using RBL-type technology...' kinds of things. It's broadly dishonest, which is why the performance numbers in the article are so very important to revealing 'worst case' scenarios.

  7. Re:SpamAssassin? by rednip · · Score: 2, Interesting
    His comment is still valid. That's like saying "We did a comprehensive review of the leading web servers, IIS, PWS, and Netscape's baby. We recognize that Apache exists, however we couldn't review it because we couldn't figure out how to get it to work.
    Add IBM HTTP Sever and Stronghold (both of which are Apache based), then you would have a fair analogy. For some reason the authors perfered to have an offical representative of the Spam Assassin group, rather than some vendor who was willing to step up to the plate.
    --
    The force that blew the Big Bang continues to accelerate.
  8. Re:That is unfortunate by JimDabell · · Score: 2, Interesting

    The latest version of SpamAssassin is 3.0.2. I've found that 3.0.1 misclassifies all Outlook 2003 email as spam, I haven't had a chance to see if 3.0.2 is the same yet.

  9. MessageLabs by tgignac · · Score: 3, Interesting

    This is a spam filtering service that I use, In 52 weeks 22,624 spam messages out of 93,714 have been blocked before entering my users inbox. The nice thing about this service for us is our IT dept is very under-staffed and makes it useful to have someone else worry about it. The do our anti-virus scanning as well and am proud to report that they have stopped all 5213 infected messages before even touching my server. Very worth while service if you are in a under-staffed situation like I am.

  10. Re:Objective by Anonymous Coward · · Score: 2, Interesting

    I posted the parent. The quote is entirely contrived, and is not part of the article. You should mod it down now.

    Conclusion: Mods don't check facts - if you want excellent karma post completely false information hinting at a evil corporate/government conspiracy.

  11. RBLs rule by mabu · · Score: 4, Interesting

    A well-designed RBL blocks 95+% of spam and consumes less resources than all the other solutions. Plus it has the added benefit of stopping virus and worm propagation, phish e-mails and lots of other scenarios where unauthorized SMTP relays operate.

    I see no reason to use client or server-side products that analyze the mail content, when this slows down mail service and reliability. RBLs, blocking mail based on the legitimacy of the source address has proven to be the most effective method of curtailing spam, and unlike all the other solutions, this one aversely affects spammers by not allowing them to consume your resources.

    If you're in the business of making money off selling spam products, I can see your support of these various half-way solutions, but otherwise, the best way IMO is to employ RBLs at the server level and slowly work towards SMTP whitelisting. I contend this is an inevitability if the authorities don't start prosecuting spammers for their illegal computer tampering.

    1. Re:RBLs rule by joel_snyder · · Score: 2, Interesting

      You would have difficulty finding stats that support the 95% assertion. Folks like Brightmail & Postini and SenderBase aim closer to 50%, but it's a different statistic: that's blocking 50% of the incoming TCP connects, not 50% of the spam. In our own testing before the spam review started, I got numbers similar to those using SenderBase as the reputation-based scoring ahead of our mail servers.

      I would agree that a well-designed reputation-based DNS blacklist can immensely increase the spam catch rate AND block a bunch of mail before it hits the servers. However, if you did the intersection of all the random RBLs out there, you'd end up with an enormous false positive rate.

      You can also take DNS BL information and mix it into your cocktail. I discussed that topic specifically in the article in greater depth.

      It's also a question of environment. I have friends who have little 2-or-3-person mail servers that basically intersect ALL the blacklists they can find and are perfectly happy---because they don't correspond with more than a couple hundred different people. But talk about a big corporation with thousands of users, and the DNS BL strategy doesn't work quite as well because of the false positive issue.

      As with everything, different strokes for different folks...

  12. Barracuda by charnov · · Score: 2, Interesting

    Actually, the #1 selling enterprise anti-spam device (the Barracuda line) is a SpamAssassing core device.

    --
    [RIAA] says its concern is artists. That's true, in just the sense that a cattle rancher is concerned about its cattle.
  13. Re:I don't know how much I trust their conclusions by joel_snyder · · Score: 4, Interesting

    Yes, you're right; it's an error. My notes show that you can turn on SSL for management, but what got written in the article is wrong. It'll get fixed online immediately. That crept in as part of the editing process.

    On the other hand, I don't understand why ANYONE ships ANYTHING that talks on port 80 anymore. It's not like OpenSSL hasn't been proven through-and-through (or you can write your own). Port 80 might be fine for pictures of your vacation, but the management interface on a corporate mail server should be encrypted and authenticated.

    However, if you want to discount a 10,000 word article for a single error, then you're going to have a hard time believing anything you ever read anywhere ever.

  14. gmail? by museumpeace · · Score: 2, Interesting

    The way their testing was conducted, they probably had to overlook spam filters that are embedded in proprietary email services but if you are only interested in getting all your mail and none of the spam, google is doing a great job.
    My gmail account has had 2 false positives out of 500 messages. Given the vulnerability to having your address fall into unknown hands that is inherent in Google's viral marketing technique for promoting the product, I would bet LOTS of other GMAIL users have the large number of spams coming in...even on new accounts where they have been careful who they gave the address too. I get about a dozen spam items a day but when one of the sh!theads sells his address list to the next spammer, I can get a burst. Bottom line: ZERO spams in my inbox...none...not any. The Bayesian stuff that spammers try to circumvent, the spoofed headers...so far none of it fools Google. And since it buffers the spam in its capacious 1Gb-per-account holdings, I have 30 days to check for false positives at my liesure.
    Questions?
    1. what vulnerability?
    when you accept a google gmail invitation, no matter how many hands it has gone through, Google posts a notification of your new address to the original giver of the invite...who could be some spammer you never met....happened to me.
    2. any pattern to the false positives?
    not sure...only have two data points. Those two items were email alerts from newspaper subscriptions which tend to be crambed with ad text and ad links...in which case, gmail is clearly trying to do me a favor and I appreciate the effort.

    --
    SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
  15. Re:Worthless accuracy table by joel_snyder · · Score: 2, Interesting

    Right. The table is poorly designed. This is what happens when you take a spreadsheet and feed it into an art department. The spreadsheet actually has 10 different columns.

    But we don't want to add percentages. That would not be fair. You have two columns, each sorted from "best" to "worst" and you can read down each of the columns. What they should have done is broken them into two separate tables rather than one.

    SO you have to actually say whether you care more about false positives (read left table) or false negatives (read right table) or make your own conclusions based on the combination of the two.

    In general, people love scorecards and pure rankings because it means they don't have to actually come to their own conclusions. (I'm not accusing you of this; just explaining why articles often have them) In this case, the real answer is that you have to decide what's important to YOU and then you can rank them youself. For example, if you don't want to quarantine and you have zero tolerance for FP, a few FN won't bother you. On the other hand, if people have good quarantine or you REALLY need to see the FPs, but you want the FN rate to be low, that's a different set of criteria.

    I'm opposed to compromises and mixing statistics together that don't belong together; see the rant about statistics in the review itself. (http://www.nwfusion.com/reviews/2004/122004spamsi de3.html)

    It's hard to compress six months of research into an article, even one as long as this one...

  16. Re:Which one? by joel_snyder · · Score: 2, Interesting

    The only thing I can say about RBLs is that you need one that is an amalgam of others. This is the same theory that drives SpamAssassin: you may be able to fool one, but you can't fool them all.

    I am doing testing with SenderBase and it gives any IP address a -10 to +10 score. Pick your own false positive/false negative threshold and you can slice out a big chunk of garbage. But SenderBase is not generally available except through a web interface. It's gone through a couple million messages of ours with one false positive.

    I know that Symantec/Brightmail and Postini both have their own 'reputation-based' services as well that seem to work.

    What I don't know of is any RBL that is itself an amalgam of other RBLs, returning a score (as opposed to a "go"/"no-go" answer). My own luck with RBLs before SenderBase was so poor that I basically discounted them as either (a) not helping enough to be worth the effort or (b) too many false positives.

    A number of the products that I looked at had "RBL voting:" they lookup things in more than one RBL, and if they meet a threshold you set ("must appear in 2 RBLs..."), then the message is marked as spam. Others consider the RBL as a component---if it's in an RBL AND has "Viagra" and a URL in it, then it's probably spam.

    I think that either a combo-RBL or RBL-voting has to be the way to go.

    They seem to have gotten a lot better in the past couple of years.

  17. Where is spamgourmet? by SysKoll · · Score: 2, Interesting
    Too bad spamgourmet wasn't reviewed. It's free, it's open-source, it works.

    Not only does it allow you to cut off spam, it gives you traceable addresses that can be used to see who leaked email to spammers. And it's perfect against phishing attempts.

    --

    --
    Mad science! Robots! Underwear! Cute girls! Full comic online! http://www.girlgeniusonline.com/