Reviewing Anti-Spam Offerings

Joel Snyder writes "Just finished looking into the innards of 40+ anti-spam products at Network World. The biggest, ugliest, and most comprehensive look at this market that's ever been done. Conclusions: lots of great products to choose from at the top (a dozen or more); a few stinkers in the bunch; and it's basically impossible to review Spam Assassin, which is unfortunate."

Comments

Objective

[Anonymous Coward]

From deep within the article:
"Although these tests were conducted with the assistance of Borderware, we where careful to ensure results where fair and objective."

So, that would be why borderware's product got the #1 position?

Re:Objective

[joel_snyder]

> From deep within the article:
>"Although these tests were conducted with the
> assistance of Borderware, we where careful to
> ensure results where fair and objective."

So deep that... they must be in some other article. I don't know where you cut-and-pasted that out of, but it sure wasn't the article referenced in this post.

Re:Objective

[stupidfoo]

Spam appliances aren't for mom and pop home user who are running Windows 98 on their 6 year old PC.

Re:Objective

[Moofie]

pretty box!=good design.

Re:Objective

[Anonymous Coward]

I posted the parent. The quote is entirely contrived, and is not part of the article. You should mod it down now.

Conclusion: Mods don't check facts - if you want excellent karma post completely false information hinting at a evil corporate/government conspiracy.

Re:Objective

[CoffeeJedi]

what they REALLY say about bias in their report:

IronPort Systems, a messaging appliance vendor, was asked not to participate in the test because Opus One has an existing consulting contract with this company - including them in the test would have created a conflict of interest.

Re:Objective

[slavemowgli]

That might explain why SpamAssassin didn't get tested, too: even with an insane amount of manipulation, it probably still would've beaten the crap out of borderware.

Re:Objective

[Shabazz Rabbinowitz]

pretty box!=good design

What if you're designing...boxes?

Re:Objective

[Moofie]

Then it needs to be more than pretty to be well designed.

Re:Objective

[JPriest]

"Although a few well-meaning souls volunteered to be the contacts for SpamAssassin, when it came time to test no one would step up to the plate and represent the product at a level that would make it competitive to the other enterprise-focused vendors."

Re:Objective

[slavemowgli]

Do you believe that?

Thunderbird

[ack154]

I find that Mozilla's Thunderbird has excellent anti-spam control. That's just from my own "testing" though...

Re:Thunderbird

[pctainto]

MOD PARENT UP!

Thunderbird's spam filtering really is amazing. Spend 2 weeks 'training' it with what is spam and what is not, and then tell it to automatically move spam to the junk folder. I have 150 junk mails from the past week -- never saw any of them in my inbox and not one is a false positive.

Re:Thunderbird

[casuist99]

I certainly do get a lot of false positives with thunderbird's spam-controls, and would really like an interface through which I can view the filtering logs (words, frequency, etc) that thunderbird must be creating.

A reporting feature (even if thunderbird just exports a database csv file) would provide more value to me. I'd also like to be able to transfer my thunderbird spam filtering profile to new installations (after reformatting, for example).

A lot of other packages (e.g. spamassassin) support some of these, but I see no reason that thunderbird couldn't try to include some of them too.

Re:Thunderbird

[ack154]

I don't think I've seen any false-positives since about 0.7 - but it does miss some emails now and then, so it's not really 100% success rate, but really, what is?

But I do see your point... however, you also have to understand that with Thunderbird, you're not really running a separate application to filter your spam (or running anything on your server for it) - this is just a free email client that does it's own filtering.

Though as I said, I'm quite satisfied. And of course, your mileage may vary.

Re:Thunderbird

[fireduck]

Thunderbird's anti-spam is nice, but I wouldn't call it excellent, at least from my experience. I've been using the junk mail feature since 0.7 or thereabouts, on a mail account that gets anywhere from 10 to 30 messages a day, 90% of which are spam. When I recently downloaded 300 or so messages, I still had 25 junk mails that it didn't flag. After several months of training, I'd think it should be more effective than that.

Re:Thunderbird

[pqdave]

It's pretty good, but it still hasn't figured out that Rolex=spam.

Re:Thunderbird

[cbiltcliffe]

300 messages, of which 90% are spam (according to your own figures). That means 270 total spam messages. It missed 25, which is
25/270 = 9.25% of your spam missed.

So, it eliminates nearly 91% of your spam, and you don't think that's great? I've seen commercial programs that don't work that well. You've also got to consider that this is just a bunch of rules that decide based on a few criteria that a message may or may not be spam. I'd much rather a few false negatives than a single false positive, so 91% is fantastic for me.

Re:Thunderbird

[LearnToSpell]

I don't think that's great. I get about 6-800 spams a day, and probably 3 or 4 of those make it to my inbox. Spamassassin.

Re:Thunderbird

[wheany]

Unfortunately Thunderbird's filtering rules seem to break from time to time. Yesterday it let through 3 messages that POPFile had marked as spam. Also, it seems that Thunderbird isn't deleting the spam messages from the POP server even though I have ticked the checkbox in the filter's configuration.

Now when I want to read my emails using a web interface, the inbox will have two weeks worth of non-mailing list ham messages, as well as all the spam I've received within two weeks. Very annoying.

Re:Thunderbird

[cbiltcliffe]

I realize SpamAssassin is the bees knees as far as anti-spam goes, but if you never get a false positive with that kind of ratio, I'd be very surprised....

Re:Thunderbird

[casuist99]

I'm not asking for a public-domain reporting of mail filtering rules, just a private report for myself - it would be useful and interesting for a variety of things. As for un-tagging false positives, I do this daily, but thunderbird never seems to learn 100% - I probably have a static 5% false positive rate. I've had this problem on numerous installations and multiple versions all the way back to 0.7. My friends have the same sort of problems. I think the things I suggest would be useful and interesting, and would certainly not increase the spam I receive or that makes it through the spam filters. Spammers are already adept at that.

Re:Thunderbird

[JPriest]

I used to use an email client called PocoMail, it too has a Bayesian engine, but it also has a configurable wordlist. eg. "rolex",+10 "realname",-10 etc. It also has advanced support for creating mail filters to sort mail, writing scripts etc. It is really an email client for power users.

Re:Thunderbird

[LearnToSpell]

I wouldn't say never, but it's been about 6 or 7 months as far as I can tell. That inspired me to whitelist my address book, and I haven't seen any since. Just don't mention Vicodin penis Rolexes when you talk to me. :-)

Re:Thunderbird

[WuphonsReach]

Thunderbird is horrid.

I use SpamBayes on my one work account with MS Outlook. It's probably 99.99% accurate. Plus, since it scores numerically, it can split spam into "ham", "might be", and "spam". The "spam" is stuff that's 99.999% certain to be spam, and I only have to look at the 2% of messages that fall into the "might be" folder.

One of these days I'm going to breakdown and figure out how to hook SpamBayes into Mozilla Mail.

(My work account gets roughly 200 spam/day.)

In-line SPAM filtering - never hits your server

[dj42]

Mine isn't in the list.... http://www.mxlogic.com

I have said it before on here, but I use Mx-logic.com to filter e-mail before it even gets to my mail server (as their filtering is in-line). They run multiple concurrent virus scanners, and you can set all policies related to attachments, sizes, virus scanning, quarantines, SPAM (deny, accept, etc, for different "levels" of probability).

It's really efficient. I haven't gotten a virus in any attachments and maybe just 2-3 SPAM messages / month (down from 100+ / day). It also does cool stuff like remove the imbedded tracking images from SPAM HTML messages (should one get through), etc. No, I don't work for them. I used to quarantine messages and review it weekly (that were medium / high probability spam), now I trust their service so much I just deny receipt to my mail server of any Medium+ probability SPAM

Re:In-line SPAM filtering - never hits your server

[mattdm]

...to filter e-mail before it even gets to my mail server (as their filtering is in-line).

"In-line"? That doesn't really make any sense. Sounds like what you're doing is just sending all of your e-mail through someone else's server before it goes to your server. That might be an okay solution for some, but it's not like it's really anything special -- you can easily set that up yourself if you like using another server under your own control.

Re:In-line SPAM filtering - never hits your server

[dj42]

http://www.mxlogic.com/technology/ "In-line Message Streaming MX Logic's service architecture creates a proxy gateway for inbound email to the enterprise or destination mail host. The service never actually acknowledges receipt of an email message. Instead, MX Logic accepts the inbound email traffic with the Simple Mail Transport Protocol (SMTP) and immediately opens a connection to the destination recipient email server. Messages are then passed through MX Logic's preprocessing filtering layers as they are streamed to the recipient SMTP server. Once the recipient messaging platform signals the acceptance of the inbound email, the accepted protocol message is then passed back to the originating SMTP sender. Messages are processed in a network stream environment and are never stored to disk."

Re:In-line SPAM filtering - never hits your server

[Not_Wiggins]

Only problem with in-line scanning is the time/resource it takes to do it.

While great for low-volume mail servers, you really need a beefy box to enable you to have enough MTA threads for handling the initial SMTP communication, threads for doing the virus scanning/spam filtering, and CPU to do it in the time allowed by the SMTP standard (I *think* it is 180 seconds... probably enough time).

I don't know if there's an advantage to not accepting virus-laden mail as one can biff it "off line" without inviting more infection attempts (ie, after the message is accepted by the transport).

But, there's plenty of good reason to do spam filtering at that point: reject the message before you even store it, so your server doesn't even have to bother with trying to deliver to forged bounce addresses.

Last problem with that, however, is attack through backup MX host, but... I'm starting to digress.

Postfix has great integration for smtp proxies with their Before Queue Content Filter.

(That's not for the author, but for others who might want to learn more about plugging his suggestion into their mail server, or upgrading to an MTA that supports it).

Re:In-line SPAM filtering - never hits your server

[joel_snyder]

MX Logic participated last year, but didn't get into the "final fab five" or whatever it was. I am not sure why they didn't participate this year. You'd have to ask them.

Re:In-line SPAM filtering - never hits your server

[Elwood P Dowd]

Do you have a link handy for last year's review?

Re:In-line SPAM filtering - never hits your server

[joel_snyder]

My apologies:

http://www.nwfusion.com/reviews/2003/0915spam.ht ml

is last year's review.

Re:In-line SPAM filtering - never hits your server

[SirNAOF]

Things I've learned about outsourcing spam filtering...

Number 1 - Too many missed messages. I've been adding domains to the block list for a year now. I still get more spam messages.

Number 2 - Poor configuration options. The only things I can change is the "aggressiveness" in 4 or 5 catagories (bulk email, porn, attachments, etc), or adding addresses/domains to a white/blacklist. Spam Assassin lets me change scores for different things, which is very nice.

Number 3 - Dependance on offsite server. Generally not a big deal, with redundancy and all, but occasionally it happens. And it blows.

For the record, my experience is with Postini. It wasn't my choice, but I still use that account anyway. My work account, using Spam Assassin, lets no spam through, and has only the occasional "false positive" (meaning it caught it according to my rules because a valid sender did something dumb, like send HTML mail).

Re:In-line SPAM filtering - never hits your server

[aaarrrgggh]

We use Postini as well... not too impressed! One bad crash on their side made for a rough week on e-mail. Three days for them to institute a backup system when the primary server had a hard crash-- no company e-mail for three days, and crippled e-mail for a week.

Out-sourcing has its places, but... I'm thinking it is companies with fewer than 50 employees.

Re:In-line SPAM filtering - never hits your server

[Matts]

I work for an outsourcer (I won't provide the name in case moderators think I'm advertising), and I'm astonished by your story. We've NEVER had that kind of outage. Once we had some of our infrastructure down for about 8 hours while one of our upstreams had big problems, but in 4 years working here that's the only outage we've ever had. We filter mail for some pretty important customers so we just couldn't allow that kind of downtime to happen.

I know that outsourcing involves a lot of trust, but to paint all outsourcing as bad because of one bad experience is a little unfair.

Re:In-line SPAM filtering - never hits your server

[wonderdog]

Nor mine... http://mailsift.com/. Mx-logic sounds very similar, tho MailSift also handles individual POP accounts. I've gone from 1000+ spams a day, to 1 or 2 a month. Haven't had a false-positive in months.

Simple to use, eerily accurate, cheap: choose any 3!

Re:In-line SPAM filtering - never hits your server

[Elwood P Dowd]

No sweat. Thanks.

clickable:

last year's review.

They tested 16 products. Postini "won", and MX-Logic got 4th place with 77% accuracy and a 0.5% false-positive rate.

Re:In-line SPAM filtering - never hits your server

[Alioth]

Virus/worm rejection is easy - just reject any mail with an attachment which is a Microsoft executable. I have a simple Exim system filter that does this job at the server level. Sending Windows executables in email is such a seldom legitimate task that it's easier to blanket reject them all, and tell any legitimate sender (I've not had one yet) to rename the executable to something else.

Re:In-line SPAM filtering - never hits your server

[macdaddy]

I *think* it is 180 seconds... probably enough time

Actually it's longer. The minimum set in RFC 2821 with strong wording (SHOULD) but not as a strict requirement) is 5 minutes per required command on the server side. The client side is also specified with strong wording and is set at a minimum of 30 minutes and optionally much longer in 3 minute increments while waiting for the completion of each TCP SEND call. RFC 2821 talks about this at 4.5.3.2. "SHOULD" is defined in section 2.3 as:

"SHOULD -- This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course."

In plain english it says don't mess with these values unless you have a damned good reason and know what you're doing. Sendmail's default will cause Sendmail wait up to a minimum of 372 minutes.

180 seconds is nearly enough in many circumstances including when receiving a message from a lareg mailing list.

Re:In-line SPAM filtering - never hits your server

[travd]

Haven't had a false-positive in months.

How do you know?

Re:In-line SPAM filtering - never hits your server

[walt-sjc]

FYI: Regardless of the RFC's, you will find many of ther larger ISP's have short timeouts - 30 seconds in some cases.

Re:In-line SPAM filtering - never hits your server

[macdaddy]

I've never come across any with timeouts less than 5 minutes for the major commands. I've been administrating mail systems for a long time now.

Re:In-line SPAM filtering - never hits your server

[walt-sjc]

While great for low-volume mail servers, you really need a beefy box to enable you to have enough MTA threads for handling the initial SMTP communication, threads for doing the virus scanning/spam filtering, and CPU to do it in the time allowed by the SMTP standard (I *think* it is 180 seconds... probably enough time).

There are many ways to speed up / scale up in-line scanning if you think about the problem for about 30 seconds or so. It doesn't matter whether you are running a home-email server or serve millions of accounts. Large ISP's have huge clusters of relays that may show up as a single server to the outside world. There really isn't a need to have a "backup MX" host that doesn't know everything that the main relays do. Backup MX's were useful 10 years ago, but are not today.

I'm not a fan of non-realtime scanning as it causes collateral spam - bouncing instead of rejecting. At this point, bouncing any mail (as opposed to rejecting it outright) received from an outside mail server should be VERY rare.

Re:SpamAssassin?

[stupidfoo]

RTFA:
We also reached out to the SpamAssassin community (see "What about SpamAssassin?"), but couldn't find someone who could act as a representative for support and configuration assistance. However, two commercial vendors, Roaring Penguin (on Unix) and NoSpamToday! (on Windows) sent products that exposed their SpamAssassin cores.

They have a whole page discussing this.

InboxCop

[Marthisdil]

Does great for Windows

Re:SpamAssassin?

[chadpnet]

I second that. Any real comprehensive review would include some sort of mention of Spam Assassin. Yes, it's highly configurable and has plural avenues of use, but I think that's what makes it even neater.

Re:SpamAssassin?

[ack154]

I never thought I'd get to use it... but... RTFA jackass. Don't just see a question and post something about it. Answer: http://www.nwfusion.com/reviews/2004/122004spamsid e6.html

That is unfortunate

[suso]

I just upgraded my server to the latest version 3.0.1 of spamassassin and the difference is amazing. I haven't had one piece of spam get through to my inbox today. And from what I can tell, there are no false positives yet. Unless you think that Darcy really wants me to come over and check out her new webcam.

Re:That is unfortunate

[Anonymous Coward]

Unless you think that Darcy really wants me to come over and check out her new webcam.

Well I did, but just forget it now. That's what I get for going after geek guys!

-Darcy

Re:That is unfortunate

[JimDabell]

The latest version of SpamAssassin is 3.0.2. I've found that 3.0.1 misclassifies all Outlook 2003 email as spam, I haven't had a chance to see if 3.0.2 is the same yet.

Re:That is unfortunate

[suso]

Aww man.

Penny Arcade had a strip about this exact kind of thing, I can't find it for the life of me though.

Re:That is unfortunate

[wackysootroom]

What do you mean "misclassifies"? ::Evil Grin::

Re:That is unfortunate

[DunbarTheInept]

How did you verify that there were no false positives when a false positive would mean you don't see the message??

Re:That is unfortunate

[241comp]

I've got 3.0.2 and I still get 20-30 spam/day. What am I doing wrong? I've got Bayes turned on and pretty much all the rules at their defaults (except a few where I turned up the score because they would definitely be SPAM). It's checking spamcop, DCC, Pyzor, Razor and I still get a fair amount of spam (especially Rolex stuff). Any suggestions?

Re:That is unfortunate

[Anonymous Coward]

Don't you mean SexyLosers?
http://www.sexylosers.com/159.html

Re:That is unfortunate

[legirons]

"I've found that the old version 3.0.1 [of SpamAssassin] misclassifies all Outlook 2003 email as spam"

Presumably the old version is still available to download?

Re:That is unfortunate

[GoRK]

This was happening to me too, so I cleared the baysesian database and retrained it. It helped a lot but I sitll get a fair amount of spam coming through... I'd say 10 per day to my box, which is pretty good considering almost 1000 are blocked per day to the same box.

Re:That is unfortunate

[DunbarTheInept]

Ah - that sounds useful. Very useful. (So SpamAssasin just sticks a header into the e-mail that you can do whatever you like with?)

Re:SpamAssassin?

[shic]

It is clearly impossible to review SpamAssasin because there is insufficient conflict of interest.

CF the Stock analyst.

SpamAssassin?

[ajs]

They say, "Although a few well-meaning souls volunteered to be the contacts for SpamAssassin, when it came time to test no one would step up to the plate and represent the product at a level that would make it competitive to the other enterprise-focused vendors."

I can only wonder what it was that they asked and who they asked. There are several companies that provide products based on SA, and the developers are very responsive.

I'll have to look in more depth later and see if any of the products they reviewed were SA-based.

Still, a review that does not cover common open source implementations such as DSPAM and SA is not a review that I would put much stake in.

yahoo.com mail is ok

[CrazyJim1]

I use yahoo mail, and I don't get much spam, even from the mailer I use to sign up for stuff on the web. Its actually sort of lonely to go weeks without recieving emails.

Re:yahoo.com mail is ok

[Moofie]

Can I have yours? I get 1000 spams a week in my yahoo account.

Re:yahoo.com mail is ok

[JJ22]

i've had a yahoo account since 1997-98, and it is only in the last three months that spam has started getting through. i use a hotmail account for most signups, yahoo for personal and "legit" stuff (travelocity receipts, etc). now i'm getting about 70 spams/day that are caught by the spam filter, and 4-5 that are missed.

Thunderbird very good

[thedudemrl]

Using Thunderbird greatly cuts down on the amount of spam you see in the inbox. After using for only about a month, 90% of spam was automatically deposited in the "junk mail" folder. Surely this isn't as good as a paid spam-prevention service, but its free :)

Re:Thunderbird very good

[g0dz]

today i check my inbox and there was no new messages. the junk folder got 108 new messages i check one by one and thunderbird got it right, all spam. thunderbird really make my day.

Re:Thunderbird very good

[bleeware]

I have used Thunderbird for several months with one of my mail boxes. It leaves about 10% of the spam sent in the inbox. The amount varies over time as spammers try different techniques.
Yahoo does a much better job, IMO. The spam that gets through to my inbox is less than 1% and the volume of spam sent to that address has dropped dramatically. Not sure why. Could be that Yahoo is dropping obvious spam before it gets to my mail box.

Re:Thunderbird very good

[Just Some Guy]

Using Thunderbird greatly cuts down on the amount of spam you see in the inbox.

...but does nothing at all to reduce an ISP's bandwidth, storage, and tech support costs. As such, "just convert everyone to Thunderbird" is more or less useless as a first-line response against spam.

The real payoff is in blocking spam before it ever gets into the system. This is where greylisting, RHSBLs, and server-side spam filtering can save a bundle of cash, both in hardware and reduced administration time.

Disclaimer: I'm in the process of writing a magazine article on exactly this subject, so I might be a little biased.

Avoiding spam

[narcolept]

Maybe it's just me and I'm one of the few lucky people in the world, but out of 5 regular email addresses that I use on a daily basis, I rarely if ever recieve spam, and during the workday, watching mailserver logs, the only people in my company getting silly amounts of spam (to me, one or two messages a day is just a minor annoyance) are people who click every popup and put their email addresses in every form available. If it wasn't for the built in spam filtering of Kerio Mail server, which is what we use here, it would probably be impossible for them to get any real work done, as out of 200 people, these 5 or so get more spam directed towards them than the rest of the company gets regular emails. Some common sense goes a long way in avoiding spam.

Re:Avoiding spam

[AKnightCowboy]

At work I've made it a point to NEVER put my work address down for anything. I just treat it like it doesn't exist, so I get very very little spam. The spam I get is mostly viruses through mailing lists and aliases. At home however I have used the same address since 1996 and I'm not about to change it now so I just rely on spamassassin to filter my mail. It catches about 99% of it. It's amazing how fast it comes in.

Re:Avoiding spam

[narcolept]

I do, however, use my work email address as a trap for emails from Prometric and the like, so that I only get bothered with certification/testing details at work, rather than a personal email address. In that case though, you have to look at it as that I accept I will get crap from them, but it's something that I used that address for on purpose. That's about the extent of giving out my work address for anything other than work related matters.

Re:Avoiding spam

[Yaztromo]

Maybe it's just me and I'm one of the few lucky people in the world, but out of 5 regular email addresses that I use on a daily basis, I rarely if ever recieve spam, and during the workday, watching mailserver logs, the only people in my company getting silly amounts of spam (to me, one or two messages a day is just a minor annoyance) are people who click every popup and put their email addresses in every form available.

There are many ways in which spammers harvest and generate spam messages, and not all of them require entering your e-mail address into web forms.

I have a number of e-mail addresses, some of which date back to the early 90's and use daily, and others which are more recent and which I've never used at all.

My oldest e-mail address was my primary e-mail adddress for newsgroup postings for many, many years. I haven't given or used that address in roughly 2 years now (as I'm using a different address that forwards to this old mailbox), and yet I still get dozens of spam messages being sent to this address daily (all of which are thankfully auttomatically filtered).

On the other end of the spectrum is my Gmail account. I have never used this account for anything at all. I've never sent an e-mail from it, or used it to register for anything. And yet it too receives spam (all of which Google also does a good job of filtering automatically). An old e-mail account I got from my ISP when I signed up for my first cable modem was similar -- I already had a mailbox and never used that account. I never even bothered _checking_ it, until one day nearly a year later out of curiousity to see how many spam messages it may have received -- only to find the mailbox was filled with hundreds of spam messages.

I often see messages where the list of recipients was obviously generated by attaching a list of user names to each entry in a list of domains and then sending the results. And who knows how many Windows e-mail worms out there are sending users address book entries back to spammers.

Best practices can reduce your spam load from certain vectors, but not all of them, making some form of filtering good policy. When even unused mailboxes are getting clogged with spam, however, you know that best practices alone just aren't enough.

Yaz.

Re:Avoiding spam

[rawg]

My email server gets scans all day long for email addresses. [randomname]@kenoyer.com. It goes on and on until I blackhole the IP it's coming from. Then a few hours later another will start up.

They just send email to random names at whatever domain name until something goes through. Then they spam the hell out of that account that works.

I have spam assassin running and I still get 20-30 spams a day in my inbox. Mostly Cialis and Rolex. Boy am I getting sick of that Rolex spam.

I just found out that my new Postfix install doesn't run my procmail scripts...bummer.

Re:Avoiding spam

[GaryKT]

There are many ways in which spammers harvest and generate spam messages, and not all of them require entering your e-mail address into web forms.

But still, webforms remain the main route through which an email address first gets out into the public domain. I've always thought twice before entrusting my email address to unknown forms, but you really need to give an email in most sites these days to create and activate accounts.

I've always wished we had a toolbar (like the Google/Alexa bar) which would rank the trustability of web forms. A high rank would indicate that you could type your email address without second thoughts and a low rank would mean we should close and exit, these forms are used to harvest your email for distribution.

But this idea seems to have occured to other people too(http://www.orchy.com/fg/). Nothing complicated is attempted - he will attempt to post to all forms on the web with an unique URL being submitted for every form. Then he will track the spams received for each address. He will thus rate the web forms as malicious/good depending on the number of spams received.

Sounds simple. I sincerely hope something useful is done soon as the idea seems to be novel and will be effective, if implemented properly.

Re:SpamAssassin?

[chadpnet]

His comment is still valid. That's like saying "We did a comprehensive review of the leading web servers, IIS, PWS, and Netscape's baby. We recognize that Apache exists, however we couldn't review it because we couldn't figure out how to get it to work.

Spamassassin

[confusion]

All-in-all, they didn't blast spamassassin as hard as I thought they were going to. It was sad to see that they didn't think they could get anyone to help them review SA, and it was sadder that they got a lot of the facts wrong about SA, like that it is built around a bayes database.

The mere appearance of SA, though, is impressive because those trade rags rarely include anything open source (partly due to marketing opportunity for commercial, paying companies).

Jerry http://www.syslog.org/

Re:Spamassassin

[joel_snyder]

Actually, the reason that trade magazines don't often review open source materials is that they don't fit well within the methodology of how we review products.

Magazines cannot look at all products. So they have to look at the ones that have significant market share and are of interest to significant numbers of their readers. It is rare when an open source product rises to the level of prominence of a commercial product---for every Apache, BIND, Linux, and Spam Assassin there are a thousand other perfectly good programs that aren't really applicable.

But when the open source stuff does get sufficient market share, it's still very difficult to review. Products that enterprises use generally have things like technical support, documentation, release cycles, and such. Open source products tend to be underdocumented, undersupported, and come out "whenever the author thinks there should be a new revision." It's hard to put these things up against each other in a head-to-head review.

Much more importantly is the use of open source as a 'toolkit' for people. Enterprises use LOTS of open source things, but they generally do so by picking-and-choosing pieces and assembling their own best solution. This is often because the commercial products can't meet their requirements! But you can't review a "tool" in comparison with a fully blown commercial product.

Look at Spam Assassin, for example. It's one piece in a much bigger picture of how to manage spam. But it's just one piece. Without a mailer, quarantine support, a GUI, complete documentation, and technical support---it's not appropriate to put Spam Assassin up against a commercial product.

Does that mean that it cannot be a VERY successful tool that you build into your own customized spam solution? Absolutely not! But it does mean that if we download an RPM of some product and it doesn't measure up to a fully supported, fully documented, fully configurable, fully featured commercial product, it really doesn't seem like a fair review.

It's HARD to review open source products when there is not someone willing to say "I will represent that product." In the case of this review, we actually had a bunch of people who had incorporated SA into their commercial products---and so SA got a fair review. It just didn't get called "Spam Assassin."

What is VERY true is that commercial concerns have nothing to do with it. The editors want to serve the readers, and the readers have told us that they want to hear about open source. So the fact that SA will never buy an ad is irrelevant, at least for the reputable publications.

Re:Spamassassin

[grahammm]

While not in an enterprise situation, my experience is that opensource software is often better supported than most 'commercial' software. All support for commercial software often offers is help in using the product (and then only if you are attempting to use it in a way 'supported' by the supplier) and workarounds for bugs. Opensource support, though, often offers fixes to the bugs. Also it often provides support for using the software in ways which the author did not anticipate.

The Best Defense...

[TrollBridge]

...is to treat your e-mail address like you treat other personal, abusable personal information.

Do what I do: create a Yahoo (or some other free e-mail) account and use that address for all questionable forms you fill out.

I've had the same address now for almost three years now and receive about five spams per week, at most.

Re:The Best Defense...

[Monkelectric]

Thats great advice...but for a lot of people the cat is out of the bag. I participated in discussion lists 5 years ago when spam wasn't on the radar... now those lists are open, my permanant email address is having the *CRAP* spammed out of it, and theres simply nothing I can do about it.

Also, it only takes one unscrupulious company to ruin your email address forever. I get 50 spams a day that use my *FULL* name and address on a private email i have never posted.

Re:The Best Defense...

[StevenHenderson]

Likely because you have a tough email address to "guess." Your suggestion does not apply if you have an email address like {CommonWord}@gmail.com. You will get spammed a good amount, as I do, even if you are CAREFUL with your email address.

Re:The Best Defense...

[sapped]

Your suggestion does not apply if you have an email address like {CommonWord}@gmail.com. You will get spammed a good amount, as I do,...

stevehenderson is a common word? Is it a verb or a noun?

Re:The Best Defense...

[StevenHenderson]

Long, Complicated Email Addresses Beat Spammers

Re:The Best Defense...

[Tenebrious1]

...is to treat your e-mail address like you treat other personal, abusable personal information.
Do what I do: create a Yahoo (or some other free e-mail) account and use that address for all questionable forms you fill out.
I've had the same address now for almost three years now and receive about five spams per week, at most.

Maybe that works well on a personal level, it's what I suggest to my friends. However, on a professional level, it doesn't work. You need to give your address out to people, you need them to be able to contact you. That's the nature of doing business, and being careful who you give it to only goes so far.

All it takes is for one person who has your address to be careless and have their address book harvested by a worm. That may be beyond their control, maybe their IT department is clueless. Maybe they use your address on a webform to send you "info" or a "greeting card".

That's why spam filters are necessary, some of us cannot work without having our email addresses out in the real world.

Re:The Best Defense...

[pjrc]

Do what I do ....

and also not do the things you don't do:

1. Answer questions, comments, feedback from customers
2. Publish any public writing and accept comments from your readers
3. Participate in public forums (like this one) and provide a reliable way for anyone interested to contact you

Sure, the disposable address idea works great if you only use email for personal conversation exclusively with a small group of people you already know, and as a consumer.

But if you want to publish anything, participate publically, or respond to current and prospective customers... you just gotta make your email address available.

Re:The Best Defense...

[jumpingfred]

stevehenderson is a pretty obvious email address that would be subject to dictionary style bulk spamming.

Re:The Best Defense...

[e40]

Same for me. However, greylisting saved me. I get 2-3 spams a day, whereas before greylisting I was getting 150+ per day.

Re:The Best Defense...

[curunir]

You're lucky...you don't have non-tech-aware friends. I've taken every precaution with my email address. I have my own domain, so I don't get SPAM from dictionary attacks like you do at large providers. I've only given it to close friends and relatives and everytime I do, I give the lecture on never putting my address in the CC field (it's either *only* to me or my address goes in the BCC field).

Yet occasionally, I get some virus scare, joke or the like forwarded to me and 50 other people and I know that at least a few of those people are going to forward it on to their friends and eventually, it will make it into the wrong mailbox where it will be subjected to some perl regexp and added to a list.

So now my 5-year-old inbox gets > 30 SPAM messages a day. This was a huge problem until I enabled Thunderbird's "Junk Mail" features...now 1 or 2 get through each day and I can't even remember the last time I got a false positive (and I do check the SPAM folder...it doesn't take more than a couple of minutes to scan 200-300 SPAM messages for false positives.) It's also easier on my personal realationships since I no longer have to give paranoid lectures to my friends about the trust I'm placing in them by giving them my email address.

What you're talking about is like teaching abstinence in school. Sure...it works in theory, but, in reality, things go wrong and you need to have a more realistic plan.

Re:The Best Defense...

[initialE]

It worked fine for me on my yahoo account, until one of my friends (you _do_ give out your real email address to your friends, don't you?) had a virus. One that whored out my address to others, and from there it made it's way onto the mailing lists. Still that didn't really last long as I didn't find my way onto the real bad ones.

Re: The Best Defense...

[gidds]

Using throwaway addresses for web forms isn't a bad idea, but I don't think it's necessary.

Until recently, I always entered my address as web_site_name@my_domain, with the result that I could trace any spam I received back to the web site who released my details.

But, apart from a few properly-attributed and traceable newsletters, none of the mountain of spam I got resulted from web forms. It's all from other sources, in particular:

• Usenet posts (a handful of posts made back about a decade ago), and
• web site text (my email address was shown in the clear for a while on a couple of friends' web sites).

So provided you can keep your address from anything directly Googleable, then you're probably fairly safe.

Funny

[StevenHenderson]

Funny how when you click the link to go to the article, the popup invites you to register for their spam^H^H^H^H newsletter. :)

Re:Funny

[StevenHenderson]

@work = using IE. Thanks. I do use FF at home.

Built In Tools??

[Flamesplash]

What about built in spam blocking like that in yahoo, MSN, gmail mail as well in Outlook and other mail apps?

Another left out...

[RM6f9]

No eAcceleration/eAnthology/Stop-Sign? hmmm.

Where is the best?

[farsideofthemoon]

DynaComm i:mail? www.futuresoft.com

Too bad

[Holi]

It doesn't include GFI Mail Essentials. I would like to have seen how that stood up to the competition.

On a side note I have started using SpamBayes-Experimental on my outlook box and it is working well so far.

Re:Too bad

[joel_snyder]

GFI got a horrible review last year. The product they submitted was a pure 'word checker' (i.e., if you've got Viagra anywhere, you're spam) and so their false positive rate went through the roof. They also had some horrible heuristics, such as "if you're not on the "to:" line, it must be spam." My experience is that it was architected for a small office where you can tune it out the wazoo. They have since (I have heard) fixed their product, but they were so heavily burned by last year that they didn't want to come and play this year. I can't really blame them; once burned, twice shy. But we'll never really know, will we?

Re:Too bad

[Holi]

Well spambayes is a clent tool and they were reviewing server tools but I too like spambayes.

Re:Too bad

[neonsam]

Our office (about 45 email users) uses GFI and yes I had to tune it a little at first, but it works pretty well now. Best part is that with an Exchange server we can configure it to dump the spam to a subfolder of people's inbox. That way the "false positive problem is reduced - users can check for themselves.

Using this in conjucntion with some RBL capabilities of our firewall, I have determined of all inbound mail only 7% is not spam. That just sucks.

Re:Too bad

[initialE]

Exchange 2003 can do everything you talked about with it's Intelligent Mail Filter optional component. The problem is that there's no real configuration, only a number they assign called the SCL. You configure 2 thresholds - one for dumping the message and another for putting it in the junk mail folder. However the real drawback is that you don't really know what it is that goes into the SCL rating engine. How they assign spam probabilities. wee. -__-

Re:SpamAssassin?

[nine-times]

I'll have to look in more depth later and see if any of the products they reviewed were SA-based.

From what I gather, there were. They're saying they couldn't review SpamAssassin as such because you're dealing with a community and not a company, but they do have SpamAssassin based products.

Just regurgitating marketing numbers

[Anonymous Coward]

The buying guide is useful just for putting all the contenders together. But don't believe the claims until you test them. Barracuda, for example, touts the capability of millions of messages a day, but we are sending our second test unit back because it just can't handle a modest load of real world mail. Their 600, for example, claims it can process "25 million messages per day" but that assumes it is rejecting 95% of the mail -- that's nowhere in their fine print.

Re:Just regurgitating marketing numbers

[joel_snyder]

The buyer's guide definitely is just pure marketing numbers. The article gives more realistic performance numbers, and that exposes some of what you're bringing up in the text. I found exactly what you're reporting (and mention it as an issue): vendors advertise based on 'oh, yeah, we throw out 50% of the mail using RBL-type technology...' kinds of things. It's broadly dishonest, which is why the performance numbers in the article are so very important to revealing 'worst case' scenarios.

Re:Just regurgitating marketing numbers

[itwerx]

Their 600, for example, claims it can process "25 million messages per day" but that assumes it is rejecting 95% of the mail

Out of curiousity, what's your mail volume and what percentage of that is legit?

Re:Just regurgitating marketing numbers

[joel_snyder]

For the spam test, approx. 75% was spam. The mail flow was about 30K messages. In Feb of 2003, the number was almost exactly 50%.

One of our consulting customers with a well-known domain name reports 95% spam (1 in 20 is not spam) over 20+million a day. Another one with a VERY well known email address reports 99%+ spam to that address---but around 50 to 75% for other, less well-known addresses.

Re:Just regurgitating marketing numbers

[itwerx]

Interesting.
One of my larger clients handles about 12k per day of which a good 8-9k is spam. They have a Barracuda which gets about a dozen false positives a day (usually crude humor, chain letters etc. :). Dunno what the pass through rate for spam is though as the users don't usually mention it (ad hoc QA shows it to be very low though).

Re:SpamAssassin?

[rednip]

His comment is still valid. That's like saying "We did a comprehensive review of the leading web servers, IIS, PWS, and Netscape's baby. We recognize that Apache exists, however we couldn't review it because we couldn't figure out how to get it to work.

Add IBM HTTP Sever and Stronghold (both of which are Apache based), then you would have a fair analogy. For some reason the authors perfered to have an offical representative of the Spam Assassin group, rather than some vendor who was willing to step up to the plate.

Enterprise support

[sterno]

If you're going to review things for the enterprise, then you need to keep in mind the requirements of an enterprise. Very few large businesses are willing to trust a product that doesn't have some sort of obvious support structure behind it. If the reviewer could not find a solid support structure for it, then it isn't suitable as an enterprise spam solution.

Re:Enterprise support

[Neil Watson]

Very few large businesses are willing to trust a product that doesn't have some sort of obvious support structure behind it.

Translation: Someone else to blame.

Re:Enterprise support

[ajs]

"Very few large businesses are willing to trust a product that doesn't have some sort of obvious support structure behind it"

Red Hat Enterprise Linux is specifically geared to this market (to the exclusion of smaller business customers, who are generally priced out of Red Hat's support pricing), and ships with SA as a supported piece of the OS.

Next concern?

Re:Enterprise support

[cheezit]

That's exactly right. What's your point? "Blame" is just the flip side of "accountability."

Why is that a problem? People who know what they are responsible for are more likely to do a good job.

Re:Enterprise support

[Neil Watson]

If someone is planning to deploy an open source tool or any tool for that matter, they should know enough about to be able to help themselves. Having another entity to be 'responsible' harbours a false sense of security. Depending on another software company, should something go wrong, is a serous error in management. Have you read EULAs? They state expicitly that the software is supplied as is and that the vendor will not be responsible for data or revenue loss. Should you have a support contract with your vendor then perhaps there is some security but, do not assume that because you have purchased something the vendor is compelled to support you.

Re:Enterprise support

[EvilStein]

And they promptly tell you that it's unsupported if you make custom rulesets/too many changes.

Been there, ran right into that. Screw Red Hat.

Re:Enterprise support

[cheezit]

The environment we are talking about is the enterprise, where EULAs don't apply. Instead, if a vendor doesn't support their software/hardware adequately, they get sued.

If I hire someone to do a job, I don't want to have to learn how to do it myself just so I can watch them do what I know how to do. I hire them and evaluate the results. That means trust and it means allowing them to be responsible for their work.

Otherwise I'd just do everything myself, which is not realistic in a 10,000 sq ft datacenter.

Re:Enterprise support

[adolf]

The typical description of enterprise-level support is thus: Being forced to pay a fuckton of money to be able to call a telephone number, whereupon one gets to speak to five different people who have never even actually touched the product in question, and one person who has, but who still has no idea how to solve your problem. Eventually, that person will leave a message for an engineer, who will either a) rectify the problem immediately or b) deny that it exists.

There are obvious variations of this, depending on the amount of money your enterprise has already divested itself of in the name of the nonfunctional enterprise-level product. These variations may include cross-shipped product replacement (look! another one that's just as broke as the first!), and/or having Fedex deliver a monkeyboy to remove and reinstall the cabling on your behalf.

I might be missing something, but I'm pretty sure that any ignorance on my part has already been made up for by all the cash in my pocket that never got spent on superfluous hoops through which to jump.

That said, I've been rather pleased with SA.

Re:Enterprise support

[ajs]

Well of course. Red Hat supports SpamAssassin, not your code base, which uses SpamAssassin. Why would this be shocking? Do you have any concept how much money they (or anyone for that matter) would have to charge to reasonably make money from supporting any random thing that anyone wanted to do with the over 700 packages that make up a distribution? You would essentially be hiring a Red Hat employee.

Re:Enterprise support

[EvilStein]

To me (and many others) that's still a far cry from "Red Hat supports Spamassassin." what exactly about it do they support? Seems to be nothing more than the basic "rpm -i spamassassin-2.63.rpm" and that's about it.

I'd hardly call that supporting the product. I would certainly not tell management that "Red Hat supports Spamassassin" if that's all I was getting from them...

Re:Enterprise support

[ajs]

So let me see if I get this right. By way of analogy, if you got a car and then added your favorite bits to the engine, went back to the dealer and said, "it's not working," then you would expect the dealer to support your modifications?

What's more (and this is the part that really breaks my brain), you don't consider the company saying that the car, as sold, is supported to be a valid statement?

Personally, when I buy an OS (or a car), I want it to be configured well, and if it's not, I'll feed back to the comapny and tell them where it's not. I then install the updates/fixes provided. At home, I download source, twiddle with the various bits that interest me and have fun, but if I'm supporting hundreds or thousands of boxes, the last thing I'm looking to do is make the support organization's job harder when it comes to helping me do MY job!

This is all moot, however. You are replying to a topic that specifically refers to one magazine doing a test, and requiring that support be available in terms of initial setup. No user modifications to the Spam Assassin rule base would have been required, and the test could have taken place with standard Red Hat Enterprise Linux support (which Red Hat's PR folks might even offer for free for evaluations like this, I don't know).

Re:Enterprise support

[cheezit]

I've never seen a lawsuit filed but I have seen legal threats. I've also seen major maintenance payments and new business suspended until the vendor complies.

You're right that vendors can survive an awful lot of failure. The point is, there is a relationship at stake when a vendor screws up. When someone signs up to support Apache, etc., the relationship is not the same. Dump that support vendor, get a new one, dump them, lather rinse repeat, but forget about having your problem solved until you take over development/maintenance. Not everyone wants to be in that business for all their software.

Re:SpamAssassin?

[Anonymous Coward]

From TFA:
"However, two commercial vendors, Roaring Penguin (on Unix) and NoSpamToday! (on Windows) sent products that exposed their SpamAssassin cores. Although neither met our false-positive threshold for inclusion in the top 12 finalists (probably because of difficulty of tuning Bayesian engines and neural networks in a test lab setting), we were very pleased to have them participate in the project."

Still, a poster that does not RTFA before making such a comment is not a poster I would put much stake in.

Did you read the article?

[Anonymous Coward]

They tried to get it to work well enough to review, but couldn't. You can flame them for not spending more time on it, but not for not trying, because they did.

Re:Did you read the article?

[chadpnet]

Flame suit on, if they can't even get Spam Assassin working... why should I trust them to be knowledgable enough to truly provide a unbaised and effective review of Anti-spam solutions?

Spam Solution

[MightyMartian]

Easy. A Postfix server running Postgrey and Anvil. Before mail ever hits a mailbox most spam (and a lot of viruses too) are weeded out. It can protect against distributed dictionary attacks.

Re:Spam Solution

[legirons]

"Easy. A Postfix server running Postgrey and Anvil."

Ahh, greylisting. All the convenience of deleting mail indiscriminately, with none of the guilt...

MessageLabs

[tgignac]

This is a spam filtering service that I use, In 52 weeks 22,624 spam messages out of 93,714 have been blocked before entering my users inbox. The nice thing about this service for us is our IT dept is very under-staffed and makes it useful to have someone else worry about it. The do our anti-virus scanning as well and am proud to report that they have stopped all 5213 infected messages before even touching my server. Very worth while service if you are in a under-staffed situation like I am.

Re:MessageLabs

[jaseuk]

You've failed to mention the false positive rate. I've had sketchy performance so far with messagelabs SPAM checking, the false postive rate is so high that I've had to switch off all their SPAM checks.

I can't fault their virus checking though.

Jason.

Re:MessageLabs

[babbage]

Another item in MessageLabs' favor: Matt Sargeant, one of the original authors of SpamAssassin, works for them writing spam filters.

Why block spam?

[IHateSlashDot]

If you block spam you'll never increase the size of your penis.

Re:Why block spam?

[WolfWithoutAClause]

Yes, I never have, and now my penis is 32 foot long!

Re:Why block spam?

[787style]

I would have never ordered my wife from overseas without Spam. Of course, if the penis enlargement pills had worked, I wouldn't have had to order her.

Re:Why block spam?

[TheTomcat]

I get ~250 spam per day.
Let's assume that 25 of those, a mere 10%, are penis enlargement spam. Let's also assume that these products do what they claim, add 2-4 inches to the length of one's penis.

So, with those numbers, I'd be gaining approx 75" PER DAY.

S

Bullshit review inclusion criteria

[rich42]

Seems like a lot of decent-sized players were left out... Ie - where is SpamKiller (client-side), SpamBully or SpamButcher?

"We invited every anti-spam vendor in our online Buyer's Guide to participate"

And what is there "online Buyer's Guide"? - a pay for inclusion directory!

Between that and their #1 choice helping them with the review process - I have serious questions as to the value of this report

. Accurately simulating a bunch of different anti-spam systems all getting the same e-mail is a bit of a trick. If one of the major players is helping set the rules - its way to easy for them to stack the deck.

Re:Bullshit review inclusion criteria

[joel_snyder]

Thanks for reading the article so carefully.

The Buyer's Guide is free and there is no fee to be included. Any spam product can be in it, and all were invited.

We didn't have a #1 choice, but NO ONE helped us with the review process. Where do you see that we had a choice, and where do we say that they helped us with the article?

Do you actually read the article, or do you just post?

Re:SpamAssassin?

[gcaseye6677]

What he's really saying is that they couldn't find anyone willing to PAY them to review SpamAssassin on Apache. That's about what passes for "comprehensive reviews" these days.

bspam also excellent

[brw12]

Though it's a small project, bspam is an excellent Bayesian filter for *nix... I tried bogofilter and some others but nothing jived with my qmail/procmail/pine setup as nicely as bspam.

I don't agree with their assumption.

[khasim]

For example, a multinational company might have many employees who don't read or speak Italian, and might train all their Italian mail as spam - something that would upset the Milan and Rome offices. Or imagine IDG, which owns many publications, all which have specialized vocabularies. No one set of training mail would work for the different communities.

It doesn't base it on the language. Just the strings. The non-Italian speakers who were getting Italian spam, would classify it as spam, but that wouldn't affect the non-spam Italian messages which would be learned as ham.

The only thing I can see would be the possiblity of increasing your database size to accomodate twice as many strings.

Reject on SMTP.

[eddy]

RBL (list.dsbl.org : bl.spamcop.net : blackholes.mail-abuse.org : sbl-xbl.spamhaus.org : multihop.dsbl.org : cbl.abuseat.org) + greylistd == average 0 spam in inbox/day.

What I like best about this approach is that you reject most of the spam at SMTP-time without accepting it. If I could I'd add spam-assassin-on-SMTP to the end of the chain, but my server is tight on memory :-(

(Unfortunately there's a bug somewhere between the debian greylistd and python whereby the daemon shuts down on me all the time, but I've lodged a bug report and hope to get some help tracking it down.)

Re:Reject on SMTP.

[Eggplant62]

Sounds similar to my own solution. I don't bugger about with client-based spam filtering, it's either firewalled or filtered at the smtp level or nothing.

IPTables --> Postfix --> (new filter that checks SPF records, not fully implemented yet) --> private access list --> RBLs (multiple, at least 20) --> Amavis --> clamd --> spamassassin --> procmail

I don't care what anyone says, RBLs are the best solution, next to my own personal access list and my iptables blocks.

Best solution for iptables spam blocking, for those of you not handling mail for large numbers of people is to block everything from these netblocks:

218/8
4/8
220/8
221/8
222/8
219/8

Re:Reject on SMTP.

[mabu]

I completely agree with you. While you're at it, should also consider adding 80/8 - 83/8

If you notice, the people who are promoting other "solutions" typically fall into one of two categories: a) people making money by promoting these inefficient content-based systems, or b) tinkerers who enjoy creating more complex systems than what might be necessary.

Personally, I just want spam to go away. I don't want to spend a ton of time programming things. I have learned the best way to deal with it is to put the burden on the SMTPs to ask for permission to communicate with us if they're nestled among a lot of rogue IP space. It has shown to work well.

Re:Reject on SMTP.

[eddy]

With regards to the bug I mentioned I can report that I believe I've found the problem. At least on my system the new feature(s) introduced in the 0.6-series to store the triples in plain text, do not work. Disabling this feature in /etc/greylistd/config seems to have solved my problem.

Re:Reject on SMTP.

[MightyMartian]

>If you notice, the people who are promoting other "solutions" typically fall
> into one of two categories: a) people making money by promoting these inefficient
> content-based systems, or b) tinkerers who enjoy creating more complex systems
> than what might be necessary.

Well I guess I fall under B, sort of. Our Postfix config is based upon a set of configs called IMGate, though I've been doing a lot of customization. It's still not all that complex a configuration, mainly because Postfix is just such a durn purdy MTA.

Due to the wonders of decisions by a previous admin, we're running a Winblows mail server, IMail, which actually does okay, but simply could not handle the dictionary attacks we were getting nailed with (at its worst, several million a day).

As an emergency measure (the IMail daemon would become unresponsive for several minutes at a time), I got a Linux box up and running on an old 233mhz Pentium with 128mb of RAM, installed Postfix and set it up between the real world and our mail server. Postfix saved our a** big time, and did it for a lot less than a spam appliance (free O +, free MTA + old Pentium box = cheap).

The long term goal is to move entirely over to Linux for POP3/IMAP/Webmail (IMail's developer has basically cancelled the mail server), but we're still running the same old Pentium. That's what I love about Linux, all this old hardware collecting dust on the shelves can be recycled. If it blows up, well, I've only got five or six more old boxes sitting around.

Re:Reject on SMTP.

[Yottabyte84]

FYI, sbl-xbl.spamhaus.org includes all data from xbl.spamhaus.org which in turn includes all data from cbl.abuseat.org.

Re:Reject on SMTP.

[dodobh]

sbl-xbl.spamhaus.org includes the CBL.
bl.spamcop.net is prone to false positives, often blacklisting legit servers.
mail-abuse.org is not free, and I haven't heard of it being really useful at blocking garbage.
The duhl.dnsbl.sorbs.net is a far more effective DNSBL.

RBLs rule

[mabu]

A well-designed RBL blocks 95+% of spam and consumes less resources than all the other solutions. Plus it has the added benefit of stopping virus and worm propagation, phish e-mails and lots of other scenarios where unauthorized SMTP relays operate.

I see no reason to use client or server-side products that analyze the mail content, when this slows down mail service and reliability. RBLs, blocking mail based on the legitimacy of the source address has proven to be the most effective method of curtailing spam, and unlike all the other solutions, this one aversely affects spammers by not allowing them to consume your resources.

If you're in the business of making money off selling spam products, I can see your support of these various half-way solutions, but otherwise, the best way IMO is to employ RBLs at the server level and slowly work towards SMTP whitelisting. I contend this is an inevitability if the authorities don't start prosecuting spammers for their illegal computer tampering.

Re:RBLs rule

[joel_snyder]

You would have difficulty finding stats that support the 95% assertion. Folks like Brightmail & Postini and SenderBase aim closer to 50%, but it's a different statistic: that's blocking 50% of the incoming TCP connects, not 50% of the spam. In our own testing before the spam review started, I got numbers similar to those using SenderBase as the reputation-based scoring ahead of our mail servers.

I would agree that a well-designed reputation-based DNS blacklist can immensely increase the spam catch rate AND block a bunch of mail before it hits the servers. However, if you did the intersection of all the random RBLs out there, you'd end up with an enormous false positive rate.

You can also take DNS BL information and mix it into your cocktail. I discussed that topic specifically in the article in greater depth.

It's also a question of environment. I have friends who have little 2-or-3-person mail servers that basically intersect ALL the blacklists they can find and are perfectly happy---because they don't correspond with more than a couple hundred different people. But talk about a big corporation with thousands of users, and the DNS BL strategy doesn't work quite as well because of the false positive issue.

As with everything, different strokes for different folks...

Re:RBLs rule

[nicklott]

I run a small postfix server using the following RBLs: opm.blitzed.org, list.dsbl.org, bl.spamcop.net, sbl-xbl.spamhaus.org, bl.technovision.dk plus some basic reject rules: reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unauth_destination, reject_unauth_pipelining, reject_invalid_hostname

I get between 500 and 600 rejected mails a week and around 150 delivered mails. Assuming all the rejected mail is spam (and I really wouldn't know if it's not :), that's 75-80% connections rejected.
Measuring the % of actual spam stopped is hard. I personally only get 1-2 a day, but I've a low volume account and I can't tell how much of that rejected mail is destined for me (not without some fancy pants scripting anyway).

Re:RBLs rule

[nicklott]

By RBLs I obviously meant DNS BLs.... doh...

Re:RBLs rule

[mabu]

I can provide stats proving my assertion. I log and generate reports on all mail traffic.

Here are stats from yesterday:

start,OK mail,invalid user,check_rcpt,rbl1 spamcop,rbl2 spamhaus,rbl3 sorbs, internal rbl
Dec 19 00:00:00, 3861, 2898, 24592, 4217, 752, 1453, 17705

These are approximate summary stats. If you look at yesterday, approximately 86% of the connections were immediately RBL'd. Right away that's 86% spam catching, but if you factor into account that many of the spammers sort messages by host and send multiples per connection, it easily jumps much higher.

98% of the invalid users are spam and virus mail.

Extrapolating the amount of spam that gets past the RBLs in boxes I monitor, it's at most 1% of the spam that I'd otherwise receive, so I was being conservative when I said my RBLs catch 95+% of the spam hitting my server. It's probably higher than that.

Personally I think there's a fatal flaw in the analysis of content-based anti-spam systems. They CAN NOT handle an equitable volume of mail as a comparably-equipped RBL-only system due to the exponential increase in system resources they need to operate. These systems can't handle mail as fast and probably throttle or miss transactions during peak hours so their stats are flawed.

Re:RBLs rule

[pjrc]

While we're using stats to "prove" assertions, here's some hard data from my spamassassin filtered inbox:

grep RCVD_IN_SORBS * | wc
200 817 13465

grep ^X-Spam-Status: * | wc
1201 6029 86914

If I had followed your advice and used all those RBLs, including SORBS, to immediately reject 86% of incoming connections, then 200 of the 1201 legit messages currently in my inbox (none are spam) and various archived mail folders would not be there. That's over 16% false positive rate!

Perhaps not all of those 86% rejected connections were really spam, but rather legitimate mail that bounced. You'll never know, since you dropped the connection before getting the message.

Maybe you don't care about false positives. But I do. That's why I use a cpu-intensive filter, rather than RBLs that are notorious for high rates of false positives.

Maybe you're an admin at a cash-strapped ISP with high mail loads and old servers that can barely handle them. But in my world, CPU cycles are cheap... and hassles of false positives, expecially from prospective customers, are expensive.

Re:RBLs rule

[pjrc]

Assuming all the rejected mail is spam (and I really wouldn't know if it's not :)

Therein lies the problem. The blacklists are horrible regarding false positives.

SORBS, recommended in the parent post, is particularly bad. I currently have 1201 message in my inbox and stored mail folders. 200 were flagged by spamassassin with a hit from SORBS.

Spamcop is much better. Only 14 are flagged by spamcop. Still, that's a lot worse that using spamassassin, which is still very effective against spam with a high threshold that virtually eliminates the possbilitiy of flash positives.

Re:RBLs rule

[rmstar]

Well - sorbs has a few different blacklists. I know that if I used the one containing dynamic ip ranges I would block lots of legit mail.

I use spamhaus + greylisting, and a few of the confirmed compromises lists from sorbs. It has reduced spam on the server I run by 99% at least. Rejected mail gets a full error message explaining things, and I'm pretty sure that if I rutinely blocked legit mail, I'd know by now - as our users would let us know by other means (these are tech savvy people).

Re:RBLs rule

[kindbud]

A well-designed RBL blocks 95+% of spam and consumes less resources than all the other solutions.

I disagree, good software can do content analysis and also be frugal on resources. Besides, there are many badly designed and operated RBLs out there, maybe even more bad RBLs than bad content filters.

Plus it has the added benefit of stopping virus and worm propagation, phish e-mails and lots of other scenarios where unauthorized SMTP relays operate.

But it does nothing to stop those things when the immediate previous hop is a "legit" (as reported by the RBL) mail server. Content scanners stop viruses and phishing scams regardless of the IP address of the immediately previous hop (checking more than the socket connection endpoint involves content scanning the Received headers, of course, which is anathema to your way of thinking).

I see no reason to use client or server-side products that analyze the mail content, when this slows down mail service and reliability.

None of the modern content scanners slow down mail delivery, unless you try to run them on a underpowered hand-me-down PC.

RBLs, blocking mail based on the legitimacy of the source address...

"Legitimacy" in this case is in the eye of the administrators of the RBL, not in the eye of the administrator using the RBL on his mail server. I value the opinions of others, but I am not willing to trust their opinion as the last and final word. Content scanners like SpamAssassin give me the means to weigh the opinions of RBL maintainers, and formulate my own synthesis of all the opinions about the legitimacy of the current socket endpoint (or hop recorded in Received headers). ...has proven to be the most effective method of curtailing spam, ...

Really? Then how come there is even more of it around since RBLs started coming into use a few years back? Spam is on the increase, not the decline, and it is doing so in spite of RBLs, which have not affected the overall volume of spam at all. ...and unlike all the other solutions, this one aversely affects spammers by not allowing them to consume your resources.

Spammers do not care about using resources. They spam because there are some people who end up clicking on their ads. Any method that prevents an ad from getting clicked on hurts the spammers. That includes RBLs and content scanners.

Re:RBLs rule

[mabu]

I use RBLs because I absolutely, positively care about every single legitimate e-mail. The problem with content-based filters is that the sender never knows whether his e-mail was received or not, or put into some spam bin that the user may never look at.

With my system, EVERY bounced e-mail, legit or not has an error message returned with an index to the specific rule violation and a web address with instructions and a contact form which bypasses the spam filter so they can let me know there was a problem.

Every once in awhile I get a false positive, but the percentage is very trivial, and the problem gets immediately solved because, unlike content-based systems, the sender immediately knows his mail didn't go through and can bypass it manually.

Re:RBLs rule

[Khazunga]

Borderline RBLs are very very useful on high traffic mail servers. You stop the mail *before* it gets in. No wasted CPU cycles, and you don't need to send a non-delivery notification. relays.ordb.org, at least, is a must. I'm quite happy with bl.spamcop.net also (It's a kind of spamAssassin oriented to IP address past behaviour).

Re:RBLs rule

[Khazunga]

I see no reason to use client or server-side products that analyze the mail content, when this slows down mail service and reliability.

None of the modern content scanners slow down mail delivery, unless you try to run them on a underpowered hand-me-down PC.

We process 60 messages per second, on average, on normal weekday. Peaks go well above that value. Anything, even the lightest processing of a mail message, is felt on the servers load. I can certainly agree with the parent post.

Borderline rejection saves lots of resources, because it leaves the grunt work of dealing with a failed delivery to the source server. Its not just content processing. It is also delivery failure notification processing.

"Legitimacy" in this case is in the eye of the administrators of the RBL, not in the eye of the administrator using the RBL on his mail server.

Untrue. Legitimacy is in the rules defined for the RBL. Spamcop follows a public rating procedure. Idem for ORDB open relay RBL. When using these, the sysadmin is setting a rule. Hosts which follow a pattern that gets them flagged by either of these are rejected. It is easy to get out of the lists, and senders are notified of the cause.

Re:RBLs rule

[joel_snyder]

You wrote:

"Every once in awhile I get a false positive, but the percentage is very trivial, and the problem gets immediately solved because, unlike content-based systems, the sender immediately knows his mail didn't go through and can bypass it manually."

You're mixing things up here incorrectly.

If I use an RBL OR if I use a content-based filter, I can either quietly drop or noisily bounce mail. I can do it at SMTP time, or I can do it after I've accepted the mail. Your choice of technology has nothing to do with what you choose to do with the mail.

Now, of course, there are products that don't give you that kind of flexibility, but that's a defect of the product and has nothing to do with whether the technology does or does not meet your needs to reject RBL-ish mail at SMTP-time.

Re:RBLs rule

[AnotherBlackHat]

if you factor into account that many of the spammers sort messages by host and send multiples per connection, it easily jumps much higher.

And if you factor in that many spammers retry from different IPs if the first connect is blocked, it's much lower.

Which effect is greater is left as an exercise for the reader.
(Hint - greylisting blocks 97%, but only reduces spam by 85%)

98% of the invalid users are spam and virus mail.

Even though most would consider 2% false positives unacceptably high, I'm very impressed.
All the RBLs I've tested that blocked more than 80% had much higher false positive rates.

-- Should you believe authority without question?

Re:RBLs rule

[ender81b]

Perhaps because RBL's are such worthless steaming piles of crap nobody in their right mind would use them? Nearly every RBL's false positive rate must be enormous compared to these other offerings.

Here's an exmaple. At the ISP I work for, we have TMDA for some of our customers. When spam email is recieved by these customers it sends a challenge email, theory behind it if the person isn't on the customers whitelist they can jst reply to the challenge email and it will add them automatically. Some of these emails get sent to spamcop's challenge boxes. SpamCop then counts these as spam. Their solution? Shut off TMDA since they won't say which boxes these spamtraps are (secret and all that). Yeah, no thanks.

After all my experiences with RBL's i'm convinced that only spamassasin type solutions are good (which use IPs etc as a way to help determine if an email is spam but not just blacklist ips).

Re:RBLs rule

[dodobh]

http://nixcartel.org/~devdas/minute.png

August at Outblaze (real numbers).

This was a sample I whipped up to show one of the managers what we want for our website. At the moment, we are redoing the antispam porttion of the website, and that will have live stats.

Re:RBLs rule

[dodobh]

Hell, clickable link:
http://nixcartel.org/~devdas/minute.png

And if /. allows an inline img:

Re:RBLs rule

[kindbud]

We process 60 messages per second, on average, on normal weekday. Peaks go well above that value. Anything, even the lightest processing of a mail message, is felt on the servers load. I can certainly agree with the parent post.

Sounds like a fairly light load. What hardware do you use to host your mail server? My content-scanning systems (anti-spam and anti-virus) process about 10 times that many messages, about 750,000/day, of which 70% or more are rejected after analysis as spam or a virus. Yep, I take the whole message in and give them a "554" at the end of the DATA segment. The rest are tagged with a score and relayed on to the internal mail system. There are eight of them, 4 each in 2 separate data centers for redundancy. Each host is a dual P4-1.4Ghz system, with 1 Gb of RAM. They run 90% idle most of the time, but can reach 50% cpu usage when some spammer is hammering them.

Obviously, the internal mail system is quite a bit less "muscular" than the public-facing content scanners. After all, it only has to deal with 1/5 the volume, and no content scanning.

The users love not having to wade through piles of spam each morning (our previous RBL-only sendmail setup allowed a lot of spam through). The recapture of lost productivity was well worth the investment. We don't really care to change spammers behavior, although if we can we're happy to help do that. We just want to get our work done. Keeping the spam out of the users inbox is our metric for success.

Re:RBLs rule

[mabu]

My choice of technology uses exponentially less resources than content-based systems, and therefore makes mail processing faster, more reliable and more efficient, with a dramatically reduced chance of missing mail.

The irony is that most of the decent content-based systems take into account the source of the mail... which in my opinion is one of the most efficient ways to identify legitimate/illegitimate messages.

Analyzing content is problematic, slow, resource-intensive, invasive and even more prone to error, so it makes no sense to use it IMO.

Re:BitDefender w/ Spamassassin

[advocate_one]

1 false positive out of ~250. Most of the time they are Netflix shipment notifications

well move those netflix messages back into your inbox and train spamassassin on your "ham"

sa-learn --ham --dir /home/username/Mail/inbox/cur

sub your own username for "username" of course :)

Re:SpamAssassin?

[bimskalabim]

While it has been a year or two since I played with the Borderware product, I seem to remember that it was also based on SpamAssassin-Amavis-ClamAV. In fact, it was actually the exact same setup that I used on my mail server, except it was in a 1U system, had a nice GUI, and a 5 digit price tag.

I'm still waiting...

[787style]

To get a junk mail filter for my real life mailbox that auto sorts into my real life recycle bin.

Discovered on Previously Cached Version...

[Carcass666]

Where's SpamAssassin?

Although a few well-meaning souls volunteered to be the contacts for SpamAssassin, when our marketting department contacted them regarding advertising no one would step up to the plate and shell-out for print ads like the other enterprise-focused vendors.

I don't know how much I trust their conclusions

[CerebusUS]

The one product that I am familiar with is Barracuda, as we run that where I work. They claim that Barracuda doesn't support SSL for management, which is dead wrong. In fact it's very simple to _force_ the Barracuda to use SSL for this purpose.

It's only one point, but they make a fairly big deal out of it.

Re:I don't know how much I trust their conclusions

[joel_snyder]

Yes, you're right; it's an error. My notes show that you can turn on SSL for management, but what got written in the article is wrong. It'll get fixed online immediately. That crept in as part of the editing process.

On the other hand, I don't understand why ANYONE ships ANYTHING that talks on port 80 anymore. It's not like OpenSSL hasn't been proven through-and-through (or you can write your own). Port 80 might be fine for pictures of your vacation, but the management interface on a corporate mail server should be encrypted and authenticated.

However, if you want to discount a 10,000 word article for a single error, then you're going to have a hard time believing anything you ever read anywhere ever.

Re:I don't know how much I trust their conclusions

[CerebusUS]

It's just the only place I could self-verify the facts. Seeing that fact wrong means I pretty much have to check everything I'm not sure about.

As for port 80 vs SSL traffic, why expose the management ports to the internet at all? The device only has one interface, stick it on your DMZ and firewall off everything but smtp.

Be sure to note that SSL is turned on by default, but the device allows configuration to force SSL to be used. Also, you get the option of generating your own cert, instead of using the Barracuda Networks one that ships with the unit or a self-signed one.

I've been using this product for a few months now and I'm very impressed with it.

Re:I don't know how much I trust their conclusions

[kindbud]

On the other hand, I don't understand why ANYONE ships ANYTHING that talks on port 80 anymore.

There is nothing preventing running a SSL server on port 80. No port number has any magical properties of encryption that are exclusive to that port.

Smart Spam Usage.

[jellomizer]

My Spam Level Rarely gets out of control. With Some Basic Filtering.

1. First I setup my own domain Name which I only give to people who I want to email me, and I have it point to a mail server that will allowing emails to me@mydomain.com, but also has a different more/popular domain name accessed to it. So Whenever I need to put my Email Address in a possible Spam Area I give it me@populardomain.com So when Spam is sent the to tag will be me@populardomain.com even for the Spams that just try to guess your name it will go to popular domain to my longer domain. Next I setup my email client to filter all emails to me@populardomain.com and put them in a spambox then forward them to spam@ftc.gov. I could go further and delete them. But in the case I might get a false positive I can receive it. But normally I just highlight all and delete them. I have never gotten Spam on me@mydomain.com and 99.99999% all emails at me@populardomain.com are Spam. With the exception of the site that gives your confirmation password by email. (In that case I normally use a hotmail, yahoo account to check for the password.)
2. Never Ever give put your email in textmode. Always make a graphic to post your email address if you really want an email contact. Or make a recording of it by voice if you are communicating to people who are visually impared. Or when you want people to email you have them go to a web form and fill out the form. (Make sure the form is programmed so people cannot make it relay to an other person)
3. Pick a good email address. Use similar rules like you use to make a good password try to mix letters and numbers and avoid common names such as bob, ted, todd, bill, jim, max, john, jeff... these are the ones that get the most Spam.
4. Use an email client that allows you to choose not to load images. So they cannot verify that you got the message.
5. Report all Spam to spam@ftc.gov. If you want the spammer to go to jail then you should at least get him on the governments radar
6. Windows user make sure you have proper virus protection. And if you give your email address to another user make sure they are doing the same.
7. Don't send Spam yourself! if you send Spam to other people you will get Spam back as well. (And combined with other nasties)
8. Check the privacy agreement on the website and make sure that they will not give you email address to everyone
9. Check those checkboxes and make sure that you will not receive Spam from them they may be worded funny so that you will check yes to them.
10. Anonymous cowards Don't get Spam! On the Internet try to maximize you anonymously.

Spammers will Spam you if they can Guess or Get your Email Address so the trick is to make it hard for them to get it.

Re:Smart Spam Usage.

[msblack]

This is not very practical for those running an existing domain, especially one with 40,000 users. Many coments like those of the posters state that they found effective methods. However, most lack any insight of how one might apply their methods to other users. It's easy to say, "this works for me."

Re:Smart Spam Usage.

[Infonaut]

Some Basic Filtering

Basic? I do not think that word means what you think it means.

Re:Smart Spam Usage.

[burns210]

My setup gives me 1-2 spam a week, at best. I post it online just to fuck with them. Bring the spam for all I care, it gives me a couple clicks and a feeling of accomplishment.... It sounds lame, but I just use gmail for everything. VERY good spam, no setup, no hassle. Just start flagging spam and it seems to learn/adapt. Set a decent password, and use filter/labels for address book people you get a lot of mail from(couple a week, or more), etc.

Plus I get virtually 365/7/24 uptime... I mean, if google goes down, the world probably has bigger problems... Like the internet going down.

Re:Smart Spam Usage.

[Just Some Guy]

8. Check the privacy agreement on the website and make sure that they will not give you email address to everyone

9. Check those checkboxes and make sure that you will not receive Spam from them they may be worded funny so that you will check yes to them.

Rule #1: Spammers lie

If a website is going to collection your personal information to sell to third parties, they're going to do so regardless of whether they have a nice privacy notice. Put another way, these people make their living my lying and stealing, but you expect their privacy notice to be an accurate reflection of their real intent?

Disregard privacy notices. If they're an honest company, then they won't need one. If they're spam-friendly, then they won't care about adding one more lie to the mix.

By the way, I find it interesting that your homepage is a link into an MLM website. I clicked the link, added a random junk item to my shopping cart, and proceeded to checkout. When it asked for my "advisor number", I followed the link to their "Finding your Advisor" search. I typed in "fras" (based on the "advno" parameter in your URL) and determined that your name is Todd Fraser, and you live in Troy, NY.

That's about as far as I'm interested in fleshing out your personal information that you posted to the Internet. I'd call you to talk about it in person at the number Google returned when I searched for "todd fraser troy, ny" (you just live a block from a golf course - is it a nice one?) but I'm still at work.

For trying your hardest to protect your email address, you're awfully eager to give away your real name, address, and phone number. I've given up even attempting to hide mine, but I also post to Slashdot with my real email address so I tend not to worry about such things.

Re:Smart Spam Usage.

[jellomizer]

Well good for you you found my secrect Identy. But my address is different because I moved. I never said I followed all my rules. So you can pat yourself on your back to see how good you are. I am sure you can find out more information about me like my new address, my Age, where I was born, But you havn't found my email address.

Re:Smart Spam Usage.

[lachlan76]

Got a way that DOESN'T require X11?

Mail.app?

[Wizard Drongo]

I dunno what it uses, but I get over 40 e-mails a day usually (about 5-10 are *real* mail, the rest is intriguing offers from diverse companies offering ways for me to extend my growth? or buy 'erbs'). Of them 40 mails, Mail.app get's all the spam, and leaves all my mail alone. Never had a false positive, and after the first week or so, no false negatives either. So why don't everyone use Mail.app? Of course it would mean switching to the worlds best OS, and the worlds nicest computers, but I see no bad side here. Mind you, I do own some stock in a certain computer company with a propensity for fruit....... :)

Re:Mail.app?

[hostyle]

So why don't everyone use Mail.app?

Possibly because of the database corruption issues that occur daily once your mailbox(es) exceeds a gig or so (graphic design shop, so lots of attachments from clients)? Thunderbird is being tested and seems to handle these volumes fine. Can't comment on a comparison between their spam filtering abilities just yet - its too soon to tell.

Where's SpamAssassin?

[Anonymous Coward]

FROM TFA:

The short answer is that no one submitted it, but of course there's more to it than that. This year we reached out to the SpamAssassin community and asked them to participate. Although a few well-meaning souls volunteered to be the contacts for SpamAssassin, when it came time to test no one would step up to the plate and represent the product at a level that would make it competitive to the other enterprise-focused vendors.

Interest in SpamAssassin is understandable. In the small-business market, the open source SpamAssassin dominates many anti-spam systems. When well tuned and integrated by a value-added reseller (VAR) that knows what it is doing, it turns out to be a very effective system. SpamAssassin users routinely report 100% spam reduction and 0% false positives (although these self-reported statistics are probably biased), and are generally overjoyed with the results.
Advertisement:

By itself, SpamAssassin is little more than the software implementation of an interesting idea: apply statistics, neural networks and Bayesian probabilities to the problem of classifying mail as spam or not. Train the engine by giving it desirable and undesirable mail, and it can tell you for each new message what pile it most resembles. It turns out to work astonishingly well, especially in small businesses where mail flow is very homogeneous. SpamAssassin's Bayesian engine even redefines the meaning of spam by letting you say, "This is the mail I want," and "This mail I don't want." SpamAssassin also mixes other tools into its scoring system, such as DNS-based blacklists and collaborative scoring, as well as more traditional keyword searches and formatting tests.

The key to SpamAssassin's success, though, is a smart VAR or IT person installing it. SpamAssassin requires a significant amount of integration work to make an enterprise-class installation succeed. Without a GUI, database, quarantine, anti-virus scanner, policy or per-user configuration, SpamAssassin is a great tool for those who want to build their own anti-spam system, but is in no way a solution by itself.

This doesn't mean that SpamAssassin wasn't well represented in our test. The important core of SpamAssassin, a Bayesian engine, was recognizable in at least one-third of the products we tested and might well have been hidden in the guts of more. The strategy of combining multiple tests to identify spam is in nearly all modern, anti-spam products, including SpamAssassin.

The difficulty in testing or recommending products that require heavy engine training, or ones based on trained neural networks, is that companies with many employees have very diverse mail flows, and the training will likely generate false positives or negatives across large numbers of users. For example, a multinational company might have many employees who don't read or speak Italian, and might train all their Italian mail as spam - something that would upset the Milan and Rome offices. Or imagine IDG, which owns many publications, all which have specialized vocabularies. No one set of training mail would work for the different communities.

Products that successfully include a Bayesian recognizer, such as SpamAssassin, do so by considering it as one factor in the larger cocktail of spam identification. By weighting the Bayesian verdict with other information, vendors have followed the trail that SpamAssassin blazed and made it enterprise-ready.

Barracuda

[charnov]

Actually, the #1 selling enterprise anti-spam device (the Barracuda line) is a SpamAssassing core device.

Re:Barracuda

[milkman_matt]

Actually, the #1 selling enterprise anti-spam device (the Barracuda line) is a SpamAssassing core device.

I've actually been wondering about these things... Does anyone have any real-world experience with them? I mean i've read the testimonials and reports about them, but I tend to trust other people in the trenches who have to admin these things like I'd have to, not the people trying to sell me one.

Re:Barracuda

[Shishak]

We have a Barracuda 300, process e-mail for about 8k unique addresses. I *love* the Barracuda, it was the best money I spent last year. My customers also love the Barracuda. It is a 'set it and forget it' device (sorry RonCo) and currently processes about 500k messages/day, 95% of them are blocked, 3% are tagged as spam and 2% get delivered. We used to run 4 mail servers with linux/qmail/SpamAssassin which would melt when the SPAM firehose was aimed at them. We now have a Barracuda and a single linux/qmail server with a load of 0.1 delivering mail to MailDir. The Barracuda handles the load perfectly and feeds mail at a nice even pace to my server (10-15 messages at a time).

Re:Barracuda

[eljasbo]

The barracuda's work great! I have one filtering mail here and would definitely recommend them. We have had one for about a year. Last month ours went down because of a memory failure i think, and everyone noticed the tremendous increase in spam that they got. The Barracuda service was superior and they had a replacement unit to us first thing the next day with no hassle at all! They have a dummy one set up that you can play with on the internet so you can see the menus and such. It is the best new device we got last year for the company, and it benefits all and keeps our email pretty much spam free and entirely clean of viruses.

POPFile

[daeg]

I happily run POPFile (http://popfile.sf.net/ http://www.getpopfile.org/). Perl-based, acts as a proxy. I can't run SA on some of my mail accounts (work, contractual jobs, etc). It's a basic word filter, and lets you see/change how words rate. It also explains its decision process to help you tweak it, for instance, any email with "penis" for my setup is 99.99999% spam.

Re:POPFile

[trilks]

I use POPFile too, and it's been a great filter. Around 99% accuracy for me.

I wonder why they didn't include it in their tests?

Re:POPFile

[daeg]

It looks like their tests focused on enterprise solutions. POPFile is hardly an enterprise ready solution, it would take too long to deploy and deploy accurately. It also has some issues running under a non-administrator account in my experience.

Re:POPFile

[JohnGrahamCumming]

> I wonder why they didn't include it in their tests?

Because the POPFile project hasn't got any money to buy ads in their magazine. I'm sure that if we did then they'd review it.

In the meantime word of mouth is vital to POPFile and other similar projects.

John.

Re:POPFile

[dbacher]

Maybe because this was for enterprise solutions. I use PopFile, but it has no place in this article in its current state. An enterprise means a few hundred or thousand users. Maintaining a few hundred or thousand copies of popfile is a mess. You cannot use a central server, because it shows all e-mail its ever processed in the log, to anyone who can happen to establish an IP connection to it. It doesn't require any authentication, etc. It requires per-machine configuration, and per-user training, and is has buckets on only a per-install basis, so you better have an install per user. These solutions that were reviewed were reviewed for large companies (read: someone like IBM) with dozens of mail servers, etc. that have to be kept in sync for thousands of users. Any solution that isn't server side isn't applicable. That includes Thunderbird. The reason that they aren't applicable is simple. Anything that isn't done on the server, isn't enterprise.

yahoo.no is better?

[tuxette]

I have a few mail accounts on yahoo.no, and only one of them has gotten spam, all of which has been caught by yahoo's filter.

They thought it was spam

[Duhavid]

And deleted it.

gmail?

[museumpeace]

The way their testing was conducted, they probably had to overlook spam filters that are embedded in proprietary email services but if you are only interested in getting all your mail and none of the spam, google is doing a great job.
My gmail account has had 2 false positives out of 500 messages. Given the vulnerability to having your address fall into unknown hands that is inherent in Google's viral marketing technique for promoting the product, I would bet LOTS of other GMAIL users have the large number of spams coming in...even on new accounts where they have been careful who they gave the address too. I get about a dozen spam items a day but when one of the sh!theads sells his address list to the next spammer, I can get a burst. Bottom line: ZERO spams in my inbox...none...not any. The Bayesian stuff that spammers try to circumvent, the spoofed headers...so far none of it fools Google. And since it buffers the spam in its capacious 1Gb-per-account holdings, I have 30 days to check for false positives at my liesure.
Questions?
1. what vulnerability?
when you accept a google gmail invitation, no matter how many hands it has gone through, Google posts a notification of your new address to the original giver of the invite...who could be some spammer you never met....happened to me.
2. any pattern to the false positives?
not sure...only have two data points. Those two items were email alerts from newspaper subscriptions which tend to be crambed with ad text and ad links...in which case, gmail is clearly trying to do me a favor and I appreciate the effort.

Re:gmail?

[wonderdog]

My gmail account has FP's almost daily. In fact, I've been quite shocked/disappointed that Google dropped the ball so badly on the spam filter. I expected better out of them. (I correct the filter daily, but it doesn't seem to be getting any brighter.)

Re:gmail?

[museumpeace]

well, like we say, YMMV. I don't have as bad a time with gmail as you report but on closer inspection of my trashcans, I see gmail has been tossing more than a few of my washington post alerts so I sure can't say you are having an unusual experience. I guess I have to conclude that the success of Google's filtering is dependent on the particular flavor of e-sewage you happen to be getting.

Re:Postini

[joel_snyder]

RTFA. Postini was in it, both in the big table and in the Dirty Dozen finalists.

Re:Copycat, clueless cat

[joel_snyder]

Thanks for the compliment... because, you see, I first used the methodology in 2003, in the original Network World test (see http://www.nwfusion.com/reviews/2003/0915spam.html ).

Or, you could go back to February, 2003, and see the same methodology being prototyped at the Demo conference (http://www.nwfusion.com/reviews/2003/0224antispam demo.html)

Let's see: Feb 2003: 2 products.
Sept 2003: 16 products, with 4 top overall performers.
Dec 2004, 36 products, with 12 top overall performers.

And Network Computing? 23 products with 10 finalists, in between my two reviews for Network World.

Yeah, I'm feeling like what Network Computing does in between my reviews makes me a copycat...not.

What are you, a NWC ad salesman? Or just a bit clueless yourself?

Re:Stop complaining about spam

[MightyMartian]

> It's economy at work, you pinky commies

My fine capitalist customers pay to get email, not to get unwanted bulk advertising, much of it fraudulent, and a lot of it in fact coming from computers that have been made into zombies by worm writers breaking the law.

Worthless accuracy table

[Ekman]

The way they reported the results was pretty bad. The left two columns ranked products by false positives, while the right two ranked products by spam caught. It is very difficult to look at this table and get a sense of which products performed the best. For example, the top product for false positives, BorderWare at 0.04% looks very impressive until you look at the other column and see that it only caught 88%. It's easy to have a low false positive rate when your catch rate is low, too.

At minimum, they should have taken the false positive rate, added it to the percent missed and ranked by that. Doing so sends BorderWare into the middle of the pack where it belongs, and more likely winners rise to the top. (Postini and MailFrontier). Pretty shoddy reporting when the end reader has to take your numbers and plug them into a spreadsheet to make any sense out of them.

They could have also weighted the two error rates, but deciding on weights would be pretty subjective. Some might think false positives should be weighted higher, while others might think the opposite. Ranking them without weights would have been an acceptable compromise.

Re:Worthless accuracy table

[joel_snyder]

Right. The table is poorly designed. This is what happens when you take a spreadsheet and feed it into an art department. The spreadsheet actually has 10 different columns.

But we don't want to add percentages. That would not be fair. You have two columns, each sorted from "best" to "worst" and you can read down each of the columns. What they should have done is broken them into two separate tables rather than one.

SO you have to actually say whether you care more about false positives (read left table) or false negatives (read right table) or make your own conclusions based on the combination of the two.

In general, people love scorecards and pure rankings because it means they don't have to actually come to their own conclusions. (I'm not accusing you of this; just explaining why articles often have them) In this case, the real answer is that you have to decide what's important to YOU and then you can rank them youself. For example, if you don't want to quarantine and you have zero tolerance for FP, a few FN won't bother you. On the other hand, if people have good quarantine or you REALLY need to see the FPs, but you want the FN rate to be low, that's a different set of criteria.

I'm opposed to compromises and mixing statistics together that don't belong together; see the rant about statistics in the review itself. (http://www.nwfusion.com/reviews/2004/122004spamsi de3.html)

It's hard to compress six months of research into an article, even one as long as this one...

MailWasher Pro

[eric76]

I know one person who uses MailWasher Pro and swears by it.

But because of certain lame functionality, I refuse to recommend it to anyone.

The problem is that it sends fake bounce messages to the return addresses unless you configure it otherwise. That may have changed since I looked at it, but a quick look at their web page shows that they still do the fake bounces.

Fake bounce messages are incredibly lame since the vast majority of spam does not have the return address of the real source. On top of that, spammers don't pay attention to those even if they do come back.

All the fake bounces demonstrate is that the people behind MailWasher Pro don't have a clue what they are doing. Of course, if they are that clueless, you don't even feel like checking out their other products.

IT Department?

[Adam9]

No one has an IT department willing to support it? Our university recently implemented SpamAssassin for the 20k+ email accounts. I'm sure there are corporations out there of our size that have a larger IT budget than us. Although Miami tends to lean towards open source more often than not (SquirrelMail, SpamAssassin, PHP, etc.). I'm glad they're spending money on enhancing existing projects than giving it to some company because they have a customer support line.

GG, mod *this* up

[jonhuang]

well played.

Which one?

[Adam9]

All of my RBLs that I have tried end up not doing me much. Usually I try to stack 2 or 3 of them on qmail. Do you have any recommendationson which RBL(s) I should be using? Thanks.

Re:Which one?

[joel_snyder]

The only thing I can say about RBLs is that you need one that is an amalgam of others. This is the same theory that drives SpamAssassin: you may be able to fool one, but you can't fool them all.

I am doing testing with SenderBase and it gives any IP address a -10 to +10 score. Pick your own false positive/false negative threshold and you can slice out a big chunk of garbage. But SenderBase is not generally available except through a web interface. It's gone through a couple million messages of ours with one false positive.

I know that Symantec/Brightmail and Postini both have their own 'reputation-based' services as well that seem to work.

What I don't know of is any RBL that is itself an amalgam of other RBLs, returning a score (as opposed to a "go"/"no-go" answer). My own luck with RBLs before SenderBase was so poor that I basically discounted them as either (a) not helping enough to be worth the effort or (b) too many false positives.

A number of the products that I looked at had "RBL voting:" they lookup things in more than one RBL, and if they meet a threshold you set ("must appear in 2 RBLs..."), then the message is marked as spam. Others consider the RBL as a component---if it's in an RBL AND has "Viagra" and a URL in it, then it's probably spam.

I think that either a combo-RBL or RBL-voting has to be the way to go.

They seem to have gotten a lot better in the past couple of years.

Re:Which one?

[mabu]

I like bl.spamcop.net, sbl.spamhaus.org, dnsbl.sorbs.net.

However, I have a much higher hit ratio with my own, homebrew IP blacklist, which includes darkening out most of China, Korea and other major spam havens. If anyone wants to communicate with our network from the black IP space, they need to be whitelisted, and any bounced e-mail has instructions within on how to do that.

Re:SpamAssassin?

[legirons]

They have a whole page[http://www.nwfusion.com/reviews/2004/122004sp amside6.html] discussing this.

Bingo!

(10 instances of "enterprise class" in one article is bingo, isn't it?)

The AUTHOR posted that! Mod it up!

[jonhuang]

Unbelieveable (or perhaps *too* believeable)

Grandparent does not RTFA. Posts an attack consisting of one lie + one rumor propogated by an anonymous coward as a social experiment: which he admits. Grandparent is modded up.

The AUTHOR of the article posts a defense...and is modded a troll.

great-o.

Where is spamgourmet?

[SysKoll]

Too bad spamgourmet wasn't reviewed. It's free, it's open-source, it works.

Not only does it allow you to cut off spam, it gives you traceable addresses that can be used to see who leaked email to spammers. And it's perfect against phishing attempts.

Re:Where is spamgourmet?

[jqh1]

For some reason, disposable email address services and TDMA-style challenge response systems are classified separately from filtering tools by the tech media. There are conceptual differences, but it's all "anti-spam". When they go on to equate filtering tools with "anti-spam" software, giving the impression that they're doing a comprehensive treatment of ways to fight spam, it can be misleading.

There's no reason not to multiple approaches simultaneously, of course. It would be refreshing to see one of these articles at least point that out, though....

Re:Where is spamgourmet?

[joel_snyder]

In our test, it was very clear that we wanted to look at head-to-head products. You can't match TDMA against Brightmail---they do VERY different things for VERY different people.

If you wanted to write a feature, you could talk about Graylisting, TDMA, Content Filtering, etc., all at the same time and discuss the pros and cons of the technology. But in a test, you want head-to-head.

Magazines consider features and tests to be very different things.

So, in this article, we ONLY looked at SMTP-in/SMTP-out products which were directly comparable. We didn't look at TDMA, graylisting, client-side tools, commercial services (like gmail or yahoo or hotmail), or mail server add-ons that don't talk SMTP. And there were still too many products...

Postini was there and did quite well.

[Richard Steiner]

My main ISP uses it, and I think it's the best spam filtering service I've ever used. So far, anyway.

My choice is eProvisia!

[kompiluj]

eProvisia Spam Eradicator. If it's good enough for lcamtuf then it's good enough for me.

Does SpamCop help?

[Zathras26]

I know this is only tangentially related, but as long as we're on the subject of spam, does reporting your spam thru SpamCop do any good? I've been doing that for a while (I have a script that handles it mostly automatically), and while I do get a sense of satisfaction from filing the complaints, I also wonder whether it actually helps.

Re:Does SpamCop help?

[joel_snyder]

Yes, it does help. You know all those RBLs that everyone seems to really love to much? It's complaints that get them populated (among other things).

So it's kind of altruistic if you ONLY complain and never use an anti-spam product, but there are a bunch of tools like DCC (distributed checksum clearinghouse), Vipul's Razor, and almost all of the RBLs that depend heavily on folks making sure that the bad guys are exposed...

Re:Does SpamCop help?

[Zathras26]

I don't receive much spam since I've been pretty careful about keeping any of my addies from getting "out into the wild". I typically get about twenty or thirty per day total on all four accounts that I use. I filter it all to the junk mail folder, then I scan over everything in the folder to make sure it's all spam, then I run a script that reports it all thru SpamCop automatically. I'm glad to hear that it does make a difference -- I've occasionally considered stopping, but now that I know better, i won't. Thanks for your help.

Re:Conflict of Interest?

[joel_snyder]

FrontBridge and MessageLabs pulled out of the review at the last minute, after their products had already been in place and in operation. You can infer any reason you want to that.

A full conflict-of-interest discussion appears in the review at http://www.nwfusion.com/reviews/2004/122004spamsid e2.html

Several of the vendors said something to the effect of "how can you do a fair review when you use one of our competitors?" (Obviously, the competitor would be 'disconnected' during the review, something that I think FrontBridge and MessageLabs didn't quite understand). My only answer is that anyone who thinks that they can review anti-spam products but is not yet using them is clearly not qualified to discuss the matter.

It'd be like asking my Mom to review Linux vs. Windows vs. Mac. Hey, she doesn't use any of those, so she's unbiased!

Web Administration? WTF??!

[eno2001]

I noticed that their only complaints about the Barracuda Spam Firewall were the use of a non encrypted web administration interface and it's early LDAP integration. Let's be real about this. What kind of moron does remote (ie. not within your network or over an encrypted VPN) administration with a web browser over the internet? If YOU do this kind of thing, look for another line of work. Whenever I do any remote admin, I do it over a secure connection only. This could be VPN, a point-to-point private link, within the network (from my office to the computer room) on a private VLAN or even over an SSH link with tunneling. So the protocol that a web admin interface uses shouldn't matter if it's not accesible to the public in any way.

As far as their LDAP complaint... it's a relatively new feature and hasn't been given time to have the edges smoothed. I can understand that complaint, but ther rest seem invalid to me. I use the Barracuda and it "just works".

Re:SpamAssassin?

[Elwood P Dowd]

What he's really saying is that they couldn't find anyone willing to PAY them to review SpamAssassin on Apache.

You did not read the article. From the Who got left out or opted out page:

IronPort Systems, a messaging appliance vendor, was asked not to participate in the test because Opus One has an existing consulting contract with this company - including them in the test would have created a conflict of interest. If you are interested in IronPort's spam catch rates, you can infer them from Borderware or Symantec's numbers because all three are based on the same anti-spam engine.

...

Full disclosure statement - After last year's test naming the Postini anti-spam service as one of the top products, Opus One signed up as a paying customer of Postini. Some of the companies participating in the test thought that this would present a conflict of interest. We feel that any test lab that hasn't already implemented an anti-spam system by now could hardly be considered qualified to evaluate how such a product would work. In addition to Postini, Opus One is a customer or purchases products or services from Symantec, Sophos, NetIQ, Ipswitch, and Process Software, all of which participated (or wanted to participate) in this test.

I'd say that given these two statements, their motives are impeccable. They did review SpamAssassin-based products. They did not review SA on it's own because there was no way to make it fit with their methodology. There were many other products that also got left out for these reasons, and their reasons make sense.

You are full of shit.

Enterprise Level

[dbacher]

Posting it out here as a root because it's applicable to 3/4 of the "why isn't listed?"

1. Solutions like PopFile or Thunderbird:
These require per-machine or per-user configuration beyond "point the program at the mail server and go." If you had 10,000 users, these solutions wouldn't work. I love PopFile, I love Thunderbird, but for any solution to be enterprise level, it needs to occur on the server.

2. Solutions like SpamAssassin:
The packages reviewed had graphical interfaces, installs and actual support teams.

Spam Assassin was invited, but the support was lacking. When they went to the community, the community let them down. This is far more often the case than a lot of us would admit. Usually there are about 10 to 15 useful people on any given projects mailing list or on any projects community site, and a legion of trolls, flamers and other morons who will just repeatedly post messages like "fix it yourself" rather than letting the people who are in the list to actually contribute usefully can respond.

Even in that case, if you're managing 12 servers, or 100 servers, or all of hotmail (these are enterprise scenarios), you want a nice UI, you want to be able to sync all those servers, you want to be able to check their status without going out to each of them, desktop notifications, etc.

The article went to great lengths to point out that many of the products use Spam Assassin internally, calling out several by names, and saying that is wasn't excluded because of this.

3. Graduate college and spend a couple weeks in commercial IT, and then see how much patience you have for RPM, APT, etc. and editing config files. Try talking a user who can't get their e-mail through configuring their client for pop file. I'm not talking people who read /., I'm talking actual users.

When some VP who barely understands how to work a power switch can't get their e-mail, you don't want to be trying to talk them through typing a bunch of "garbage" into a configuration field.

4. Security also comes into play here.

PopFile is not an enterprise solution. Anyone with a web browser and access to your machine can pull up PopFile and view every e-mail it has ever processed. I know of very few executives or even common employees who would consider that to be a "good thing."

Re:Enterprise Level

[omb]

Never have I heard more nonsense
The last thing you want in an enterprise solution is a GUI
In a Enterprise SPAM filter you want to minimise
noise, maximise flexability
and

be Hippocratic 'Primum non nocere'

that means NOT guessing attachments are viri

sponsored testing??

[DigitalCrackPipe]

Network World maintains an online Buyer's Guide, which allows any anti-spam vendor to submit its product information...we decided that any vendor who wasn't in the Buyer's Guide wasn't very serious about participating in a product test

"Buyer's guides" based on company submissions tend to not be very objective (i.e. the advertisers own them). A true scientific endeavor would involve finding out which products to use, regardless of how agressively they market themselves.

Re:sponsored testing??

[joel_snyder]

Absolutely. SO there's a buyer's guide so you can learn about ALL the products. AND, then there's this mammoth test that is an independent, objective, scientific look at products that are a subset of the Buyer's Guide who wanted to engage in head-to-head competition and comparison.

We're aiming for the best of both worlds.

ASSP

[defsdoor]

I use ASSP - its a transparent SMTP proxy that does RBLs, Bayesian, attachment scanning and most recently virus scanning (using clamav dbs).

Its simple to setup and works great.

ASSP

Sigh... STILL no assp!

[heavyboots]

IMHO, these guys run rings around everyone else. Sure it's not dirt simple to install, but it's certainly simpler than Spam Assassin used to be (I haven't tried 3.0 so I can't comment on the ease of installation of that). And it blocks spam quite effectively on any OS that can run perl. More to the point, it rejects spam immediately, so you're not dealing with messages piling up in the queue waiting to be filtered. They either make it onto the mail server or they don't. Also, if it's a real, valid mail server at the other end, the user should get a notice that their message was rejected, generated by that mail server. However, a spam zombie isn't going to care that you rejected its message with a 550 error and you aren't going to generate an outgoing piece of email to the wrong reply-to address.

http://assp.sourceforge.net

How come no clamav plugin for SA?

[lorcha]

TFA mentions that there is no virus scanner builtin to SpamAssassin, and counts that as a strike against it. Of course, my mail server runs all messages through clamav and rejects anything that is a virus, but how come nobody has written a SA plugin for clam?

You'd think that SA+ClamAV would be a pretty common configuration.

Re:How come no clamav plugin for SA?

[lorcha]

Hey, thanks for the tip! Wish I would have known about that when I set up my MTA.

Oh well. I've already got things set up where I want them, but maybe next time.

"Significant performance tuning"

[kindbud]

The Unix factor: We spent more time tuning Unix, Sendmail and various Unix system utilities than we did tuning products from vendors that ran on Sendmail, including Roaring Penguin, Privacy Networks, Proofpoint and Cloudmark. In some cases, the differences were dramatic. A single-line change in Sendmail configuration, for example, tripled the throughput of Roaring Penguin's CanIt Software. This means companies that install their own software, rather than going with an appliance, need to be prepared for significant performance tuning.

Wow, a one-line change in sendmail.cf is "significant performance tuning". I guess I'm not overpaid after all.

SpamAssassin

[omb]

No, it is NOT impossible to review; it is very easy,

Just say what happens; the commercial products fail

to detect

mis-detect OK mail
prevent people working by canning MS project files ...

till now I have to send all attachments as Base64
encoded PGP encrypted files
and all the M$-l'admins are to stupid to understand

or do anything about it.

Greylisting

[pjt33]

I discovered today that the reason one of my friends keeps getting bounce messages from my address is that the company I have an account with uses exim's sender verification, and the organisation he uses has graylisting enabled. He sends an e-mail, exim tries to check back, the graylister drops the connection, and exim concludes that the MAIL FROM is forged. Be nice if anti-spam solutions co-operated, wouldn't it?

Re:Greylisting

[taobill]

Someone's MTA is misconfigured.

Probes are supposed to use the empty sender .

Greylisting systems, when presented with the empty sender, are supposed to accept all recipients, and only TEMPREJECT after the end of DATA.

That way Probers and Greylisters can coexist happily.

that's all well and good

[zogger]

and it's an accurate assessment, but not everyone out there is an ISP, and if theirs don't deal with the Spam problem, the users are stuck trying to cobjob their own automagical miracle multiple software apps complex IT spam solution at home, OR, use something as simple as tbirds or mozs spam filter, which works good enough to at least keep it down to a manageable size. Or is spam filtering only for the "IT elite"? How long do we poor non_ISP and non pro sysadmin plebians need to wait for ya'all to deal with the Spam then? How long has it been again?

If it is really hurting the ISPs, then it's in their best interest to do something about it, but they seem to not be doing that very much. Or would you rather all those millions of regular ole surfin folks just eat the spam until such a time in the mysterious future as the web "professionals" actually do something about it? Speaking as joe internet consumer, I am tired of waiting for the "IT Network ISP professionals" to "handle" it, because they haven't "handled it", not in the general sense.

As such it's NOT "useless" at all to run a personal spam filter, it's the only thing the millions of spam deluged people have currently,and at least we can use some end user app that's easy to set up and configure. But becauae it's not the single magic silver bullet, we shouldn't use it?

Some ISPs have made an attempt to "stop spam", or their upstreams, but most haven't, and the overall results are still dismal, else we wouldn't be having these spam-problem discussions every other day, and it wouldn't be a global annoyance and cost and complexity headache problem.

Now if the poor ISPs want to pay their users, take a penny off what they pay for an account per spam, something like that, maybe that will get their attention, but most ISPs just let the slop through. Why? Don't ask me, I ain't one of them guys, but spam filters have been around awhile now, no idea why they aren't more widely used at every point on the internet. Spam shouldn't make it past the first hop, IMO, or at least most of it.

Re:SpamAssassin?

[ThrobbingGristle]

I set up SA for some users on our "departmental" mail server. I switched myself over to DSPAM as a test and it works better IMHO, although I haven't upgraded SA recently, and I think it's improved as well.

DSPAM works quite well once trained... the only problem I have is that there were still occasional false positives. I get so much SPAM now though that I can't afford the time to dig through my SPAM folder looking for FP's.

Of course, plans are in the works for IT to move us onto the Exchange server so I may not bother upgrading SA again. I'll probably continue to run DSPAM/cyrus for my own use after the big day. Hooray for an all MS IT shop!

Re:SpamAssassin?

[omb]

So to correctly interpret your Astroturfing

the purpose of a review is to help inform the less informed readers, not just to collect cash from suppliers

Spamassassin is NON intrusive, better than 99.9%
accurate and free

since I run all my machines on Linux or Solaris I dont
have to worry much about viri, and
thanks to Spamassassin,
no longer have to read 500 solicitations to buy Viagra or have my breasts or penis enlarged!

Re:Stop complaining about spam

[Supertroll]

What if the post office were to start a service where an advertiser could give them one copy of a piece of junk mail along with a list of recipients. The post office would then duplicate the mail, stuff them in envelopes and then send them out to the recipients postage due. Eventually you would need a wheel barrel to get your mail everyday.

That's how spam works and that's why people complain.

Re:SpamAssassin?

[stupidfoo]

Uhmm... did you mean to reply to me?
If you did, wtf? I just quoted the article. That's in no way astroturfing and I in no way denegrated Spamassassin.

Apache Software Foundation

[PhYrE2k2]

"The short answer is that no one submitted it, but of course there's more to it than that. This year we reached out to the SpamAssassin community and asked them to participate. Although a few well-meaning souls volunteered to be the contacts for SpamAssassin, when it came time to test no one would step up to the plate and represent the product at a level that would make it competitive to the other enterprise-focused vendors."

How about the Apache Software Foundation who now develops the product? ApacheCon happened about a month ago and I'm sure was swarming with all the folks who work on these Apache projects.

While I realize what they are saying (They'll actually have to take an hour, search google, and get directions with _might_ work as expected, rather than clicking NEXT>NEXT>NEXT>FINISH>), I still disagree with it.

Most of the other products are commercial and would cost money. If they paid for them (doubtful), of course there's an 800 number. If they got them free, I'm sure there is a big flag labeled PRIORITY next to their serial number.

So with Apache, there's no 800 number or authoritative contact. Ask any Guru and they'll tell you all about ./configure && make && make install and then any final configuration on a system-wide or per-mailbox basis.

Bug a developer and you'll probably get great features like Razor and RBL activity in there.


All I'm saying is this happens all too often. Because there isn't a support@[opensourceproduct], reviewers look no further.

I would have been more impressed if they didn't mention it at all, as at least then they'd be ignorant rather than lazy/unresourceful

Did they look at the SpamAssassin home page?

[Per Abrahamsen]

The project home page (number one hit on a google seach for SpamAssassin) has a link to a list of commercial support solutions written with a large font at the top of the page.

Re:SpamPal

[Joffi]

Indeed.
SpamPal is best Win32 antispam product for the following reasons:
- ease of use and configuration
- multiple plugins to extend abilities
- active plugin developers
- active core development
- Open Source (see http://www.spampal.org/license-src.txt)

Re:wtf? Somebody mod parent down.

[theskeptic]

Argh. Who modded up that comment? Is this some kind of a troll? You want spamgourmet to be included in the test? Are you fscking nuts? Jesus Christ, did you even read the damn review? How can Spamgourmet EVEN be a part of the test?

Spamgourmet is NOT a software. It is a WEB service. You CANNOT install it on your network gateway. So it cannot even be a part of the test! For a company with an enterprise mailing system for 400-1000 employees, you expect them to use spamgourmet?

Re:wtf? Somebody mod parent down.

[SysKoll]

If you weren't so busy insulting everyone, you would have noticed that anyone can install the spamgourmet software (which is open-source) on their own intranet web server. This allows large and small companies alike to deploy their own spamgourmet instance.

Just because you don't understand something doesn't make it false. Somtimes, people more clever than you have actually already looked at the problem.

Missed the boat on 0spam.net...

[zerospamnet]

A copy of what we sent to the editor of NW:

Your magazine's analysis of 0Spam.Net completely missed the boat as to our service's accuracy level by reporting our false positive rate at 5% of message traffic. Clearly we would not have any customers if that were the case. Your results are statistically irreconcilable with the fact that for the last 18 months our service's false positive rate has been less than 1 in 2,000,000 false positives. You made no attempt to determine why your findings on a small sample of 10,000 messages differed with our production results of hundreds of millions of messages. As such, your reporting of our service as having a "dismal 5% false-positive rate" is not only inaccurate, but disserves the needs of your reader base. Your analysis was far from a realistic test of our service.

The problem with the test analysis is simple: it primarily boils down to your attempt to review 36 products simultaneously. While Mr. Snyder is to be commended for his efforts and did as well as he could with such an unrealistic task, he could not spend the time with each vendor that would be expected by a company's executive management for acceptance testing of a vendor chosen to solve such an important business problem as spam.

0Spam.Net is a service offering with real customer support personnel behind it and numerous feedback opportunities, NONE of which Mr. Snyder chose to explore. Perhaps no one expected a product to offer real service levels and direct interaction with the customer. While many vendors don't offer high service levels to their customers, our practice of doing so has shown with real production results that it leads to phenomenal quality, accuracy, and security levels.

To be specific as to how the testing was unrealistic, our normal acceptance testing process for new customers involves a 30-day period during which time auditors and trial account coordinators work closely with customer staff to collect feedback and adjust filters appropriately. The "tuning" period offered by Network World involved no interaction with Mr. Snyder ("the customer") and was considerable shorter in length. Further, we were not able to "touch" the service settings once the test period started; auditing and customer interaction go on 24x7 (as needed) with our service because, well, it's a service - not a piece of software or a box. Sadly, while there is not much need to have interaction after the acceptance testing, it is critical at the start of the acceptance process and was not possible given the test methodology.

Mr. Snyder also stated that our product "has no knobs" to make adjustments; it is unfortunate that he appears to have had so little time to read and follow the most basic of the end user documentation available for our service. There is no need for end users (or an administrator) to tune lots of knobs - most whitelisting, blacklisting and other tuning operations are easily done with an existing interface they are already familiar with: their email client.

In summary, we are dismally disappointed that your magazine spent such a small amount of effort understanding and testing the products as compared to what would have reasonably been expected by an enterprise IT staff in evaluating a product for actual acceptance testing. At a minimum, one would have expected you to seek to understand why your statistically tiny test sample of 10,000 messages might differ so much from the results of a much statistically larger body of production results with real customers. While we understand the pressure Mr. Snyder was under to try to evaluate 36 products simultaneously, your methodology came up short in our case and allowed a product with a customer track record far better than any of the other products in your review to be greatly shortchanged.

Poor assumption

[lorcha]

I use RBLs because I absolutely, positively care about every single legitimate e-mail. The problem with content-based filters is that the sender never knows whether his e-mail was received or not, or put into some spam bin that the user may never look at.

This is not a valid assumption. I run every incoming message through SpamAssassin and ClamAV and if ClamAV says "virus" or SpamAssassin returns a score of 10+, I reject during the SMTP session by sending a 554.

That means no misdirected bounce messages and anyone whose mail is rejected is notified by his/her MTA.

By the way, I have never had anyone tell me his/her legitamate email was rejected by my server. What false-positive rate do you consider to be acceptable?

Re:Poor assumption

[mabu]

This is not a valid assumption. I run every incoming message through SpamAssassin and ClamAV and if ClamAV says "virus" or SpamAssassin returns a score of 10+, I reject during the SMTP session by sending a 554.

That means no misdirected bounce messages and anyone whose mail is rejected is notified by his/her MTA.

By the way, I have never had anyone tell me his/her legitamate email was rejected by my server. What false-positive rate do you consider to be acceptable?

I contend there is no way you're running an effective spam filter if you've never had false positives.

In any case, the operative issue here is the reasonable balance between resources required to maintain reliable service and what's necessary to stop spammers. I have no doubt that one can spend an inordinate amount of time and resources to squeeze an extra 1-5% efficiency, but even that is fleeting, and ironically this plays in favor of the spammers, who profit by exploiting an unfair amount of resources via their efforts.

All spam "solutions" essentially fall into one of two categories: efforts that inhibit the spammer's ability to steal resources, and those that don't. RBLs directly affect spammers' ability to disproportionately exploit resources; content-based systems DO NOT.

When you fight spam, you have two objectives depending upon whether you want to temporarily or permanently address the problem: stop spam from getting to your inbox, and/or stop spammers from being able to steal your resources. When spamming requires you to install extra software and beef up systems to handle spam, that's still theft. You might have less spam in your inbox, but it's still costing you time and money. In that case, you still lose.

Content-based filters do encourage spammers to get creative and try to thwart the filters, but it's tit-for-tat. You have to do as much work as them to maintain the integrity of your system, so you're still having your resources bled dry.

RBLs are different. They put much more pressure on spammers and less pressure on innocent servers; they require less resources and time and shift the burdon to spammers. My approach not only cuts down on spam, but unlike yours, it requires less time and money and system resources on my part to implement. This is all about increasing the cost and liability exposure of spamming.

Personally I doubt that your system is more effective than mine. But even if it was, it is moot. Most of the top ISPs routinely lose legitimate mail because they depend too much on content-based filtering, which might offer a short-term improvement, but ultimately doesn't put pressure on SMTP sources to be responsible, and that's the only way to really make a difference. RBLs do that, very effectively.

Re:Poor assumption

[joel_snyder]

>By the way, I have never had anyone tell me
>his/her legitamate email was rejected by my
>server. What false-positive rate do you consider
>to be acceptable?

In our test, we had a bunch of vendors claim that because they were not aware of false positives that somehow there were vanishingly few. This is even true of systems that don't quarantine mail, although this is patently ridiculous---how are you even going to know about false positives if your mail disappears into a black hole? The answer is their marketing people pull numbers out of some occluded place normally covered by pants and claim those as false positive rates.

This, of course, is complete crap. False positives happen and mail has become so 'lossy' that we never even hear about some of them. For example, during our testing, I discovered a false positive in one of the RBLs that I was looking at. I found out because someone pointed it out (they got a 5xx or 4xx and had the presence of mind to send it in). Once I KNEW about the problem, I could go back in the logs and see a half-dozen other people who were affected by the same RBL error---but never bothered to complain or say anything.

This problem gets worse when you consider content-based systems, because they tend to false on the mail that you care the least about---bulk mail, mailing lists, etc. If you subscribe to a mailing list and it receives 50 messages a day, will you miss one? Almost certainly not... but that's likely where the false positives are happening.

InfoWorld, for all their faults, had a good idea to somehow separate out false positives from bulk mail as opposed to person-to-person mail. It's an interesting concept and I considered putting it into our test as well, but eventually dropped it. The reason is that some mailing lists are, to me, MORE valuable than person-to-person mail. For example, I'm on some software development project mailing lists, and a message there is likely to be more important to me than a message from a clerk saying "there's a package for you at the front desk from Amazon."

With all of this being said, I think that rejecting mail at SMTP time is really the way to do it. Whether you do it RBL or content or both, the fact that you can refuse to accept responsibility for the mail wins in so many ways. I depend on email heavily, and I fall into the "every sperm is sacred" school of thought with email. Quarantines are OK, but if I could know that either I got the mail or it got 5xx-rejected to the upstream, I'd prefer that solution by far. (Actually, the REAL preference is to let the network manager decide whether they want to accept/tag-and-deliver-quarantine/reject the mail)

Re:Poor assumption

[lorcha]

I contend there is no way you're running an effective spam filter if you've never had false positives.

I contend that "effective" can mean a lot of different things. I consider my filter to be effective. The fact that you do not has little bearing on me.

In any case, the operative issue here is the reasonable balance between resources required to maintain reliable service and what's necessary to stop spammers. I have no doubt that one can spend an inordinate amount of time and resources to squeeze an extra 1-5% efficiency, but even that is fleeting, and ironically this plays in favor of the spammers, who profit by exploiting an unfair amount of resources via their efforts.

You are never going to win the resources balance game. You've got spammers out there leveraging (in the true sense of the word "leverage") zombie desktop machines doing nothing but sending out SMTP. The spammer is using zero of his own resources to accomplish that. Are you using more than zero resources, mabu?

The fact of the matter is that CPU and RAM are cheap. I've got an old desktop filtering spam and it has no difficulty keeping up. And besides, I'm doing content based filtering NOT to filter out that last 1-5% of spam, but rather to ensure that I am not getting any false positives!

Of course, with any system it is theoretically possible to get a false positive, but realistically, if your email got scored 10+ by spamassassin, then your email was spam. I analyzed 2 months of email traffic and no hams were scored above an 8. So I set it at 10 and don't worry too much.

When spamming requires you to install extra software and beef up systems to handle spam, that's still theft. You might have less spam in your inbox, but it's still costing you time and money. In that case, you still lose.

Welcome to the real world, complete with economic externalities. I hope you enjoy your stay.

The fact of the matter is that many things that shouldn't cost you resources, do. When someone walks by and disposes of a beer bottle in your yard, it costs you resources to pick it up. When your neighbor does foundation work on his house and all the rodents that were living in his crawl space are displaced to your basement, it takes your resources to get an exterminator to come remove them. Sure, you can try to ask your neighbor to take care of the problem he caused, but that still takes your time resources.

If a spammer wants to steal my idle CPU cycles, tell me why I should care? In the meantime, I'll be enjoying my spam-free inbox.

RBLs are different. They put much more pressure on spammers and less pressure on innocent servers

That's a load of crap. You're rejecting innocent mail, so you are putting pressure on innocent servers and innocent emailers. I just looked at my current Ham corpus and out of 1505 messages, a full 15 of them would have been rejected if I used RBLs alone. Well, guess what. I don't want to reject 1% of valid email, so an RBL-only solution is totally unacceptable to me. What if a client emailed me and his email got 554ed as spam? 1 in 100, man. Don't forget that.

Personally I doubt that your system is more effective than mine. But even if it was, it is moot. Most of the top ISPs routinely lose legitimate mail because they depend too much on content-based filtering, which might offer a short-term improvement, but ultimately doesn't put pressure on SMTP sources to be responsible, and that's the only way to really make a difference. RBLs do that, very effectively.

Personally, I could not live with your system, let alone consider it effective. You are rejecting legitimate email, and that is unacceptable to me.

You also think that you are somehow putting pressure on spammers. You are seriously mistaken in that regard. Do you really think your grandmother's zombified computer cares if the spam she is sending you is being blocked by an RBL or SA+Clam? Do you think she or the spammer knows the difference?

I'm confused as to why you accuse ISPs of losing email due to content-based filtering yet you routinely lose legitimate email due to false-positives.

Re:Poor assumption

[mabu]

Zombied computers, aka DUL IP space is RBL'd in most well-tweaked RBL-based mail systems and it doesn't affect legitimate mail. These systems shouldn't be running their own SMTP gateway so it's practical to block them. If they want to run their own SMTP, my rejection messages give them a way to request permission. Most content-based systems involve just as many, if not more hoops for someone to jump through to be validated. In the meantime, I don't have to dedicate nearly as much resources to fight spam as you do. If you have unlimited bandwidth and computing power, congrats. I don't need to spend more money on a content-based system that requires a LOT more maintenance, that delays mail, and is even more problemmatic than RBL-based systems.

Re:Poor assumption

[lorcha]

I think I've spent more time explaining my system to you than I did actually setting it up. As far as maintenance, I just let apt handle that. I certainly don't maintain any DUL permission requests or anything ridiculous like that.

My system just works, it is maintenance free, and it is more accurate than yours. You are willing to sacrifice accuracy to gain CPU, whereas I am willing to sacrifice CPU to gain accuracy. Not everybody has the same requirements, and it seems we are both happy with our respective solutions.

Have a nice day.

Re:Copycat, clueless cat

[richi]

Yes, the parent post is misguided (and yes, Joel is doing a great job with his replies here on /.), but there's a big problem with this testing methodology.

Basically (please correct me if I'm wrong, Joel), they replay a bunch of incoming messages at the product under test "in real time." This has the advantage of making the tests repeatable. However, it takes no account for the newer spam filtering methods that look at "out of band" information to see if the sender is a spamtool or a zombie.

The methodology was OK a year ago, but I have serious reservations about it now. I'm guessing this is why several of the big names declined to participate this year.

Of course, what's Joel to do? The alternative methodology is to give each product a real live inbound stream to work on, but that's hardly a repeatable test, is it?

r.

Re:Copycat, clueless cat

[joel_snyder]

Yes, that's the methodology. Exactly. We replay them "immediately" for some value of "immediately" which can be translated as "if they're up, within a second or two, otherwise we queue." Not "real time" in the strict definition of the word, of course, but fast enough that we can claim that all products saw all messages at the same time as they showed up on our Internet pipes. I wouldn't say that it makes it repeatable, but it makes it supportable across 40 products.

And the issue of spamtool/zombie identification is a real one; thanks for bringing that up. As is the much more important issue of the 'sending' IP address. The 'sending' IP address problem in this methodology is one that can be dealt with by a good product (and several products can and do deal with it just fine); some others are so restricted that they cannot work effectively except as first hop. To me, that's a bug. I had one VP of Marketing scream at me "no one ever puts our product anywhere except as first hop," and I barely held back from saying "yeah, that's because your product is such a piece of crap it can't go anywhere except for first hop."

However, the "looking at the SMTP conversation" part is impossible to really deal with. But, I'll note that the majority of the products sit on top of an MTA. Either they install for themselves or they use tools like sendmail, qmail, or (for Windows boxes) the MS SMTP MTA. So those products don't actually know anything about the SMTP stream. There are a few products that specifically brought up this issue because they DO look at the SMTP stream, and they probably did not do as well. How "not well?" Maybe a couple of percentage points in spam catch rate. Maybe less.

In the review, I wrote a short side-bar where I admitted this up front: (http://www.nwfusion.com/reviews/2004/122004spamsi de5.html) "You may notice our numbers are not as optimistic as the marketing literature from vendors' products. There are four reasons for this:

1. Side effects from our test bed probably shaved a few points off of each product's ability to identify spam. ..."

I also brought that issue out when I wrote : "The false-positive and false-negative rates we found are useful for comparing products but a real installation will likely have a lower false-positive rate and higher spam-catch rate." (and mention things like the SMTP catch rate) in http://www.nwfusion.com/reviews/2004/122004spamsid e.html

OK, so now that I'm done defending myself, what's the point? Well, one of the vendors told me "you know, all these products basically have no false positives and catch all spam." But that's completely wrong. We discovered a bunch of products that are still dark-age when it comes to catching spam, ones which have enormous false positive rates (in particular).

However, if you look at the top 10 or 12 products, you can see that while there are differences, they are not showing a huge variation in behavior.

What this means is that you can take a test like mine and use the spam catch rate/false positive rate as a "first cut." Because I believe that where you want to make your buying (or implemention, in the case of open source) decision is based on things besides just spam catch rate/false positive rate.

We have to do the FN/FP rate tests just to say "you must be this high to attack this problem." But from then, there are huge differences in the products, and that is what is important. I don't want to seem like I'm lashing out at the people who say "oh, I use (insert product here) and it never falses and never misses," but those folks just don't get it. It's not the spam catch rate that differentiates the products; it's everything else.

An easy example is CloudMark. Talk about Zen. This product doesn't do

Re:Copycat, clueless cat

[richi]

Yes, you reminded me of my other methodology comment. This thing about putting AV in front of the spam filter.

I think this is likely to significantly reduce the effectiveness (ie, increase FNs) of *some* of the products. That very fact makes your effectiveness comparisons dubious, because you're no longer comparing apples with apples.

"Shaving a few points off" the effectiveness of all the products would be fine, but I don't see that you're shaving off equal amounts across the board. How can you be? The products are different, using different methodologies.

In order to achieve 95%+ with vanishingly-small FP rates, these guys have to make some incredibly subtle judgements. By insulating them from the out-of-band data, you're going to reduce their ability to make those judgements.

That doesn't necessarily make those products a "piece of cr@p". On the plus side, it does make them less susceptible to content-mangling tricks, which is a "zero-hour" benefit, at the very least.

I hear your point about FN/FP rates not being the only criteria, but it's absolutely Job #1 to get this right. Especially with FPs, otherwise the cure is worse than the disease.

richi.

BULLSHIT

[theskeptic]

Seriously, break out of your bubble. Or somebody is going to pop it.

Spamgourmet is open source software. And its free. However, there is NO VENDOR SUPPORT for this software. You get that? Who is going to support it once it is in place?

Secondly, have you ever ever come across corp email id's in the format- bestbuy.5.linda@xyzcorp.com ?

Where do you get the faintest idea that companies will think of using that kind of email addresses?
Have you come across one medium sized company using such a solution? NOPE.

You think any company is going to use bestbuy.5.linda@xyzcorp.com?

That sounds and looks like shit.
Nobody would like to be caught using such email addresses.

The problem the author of the review stated with spamassasin is ALSO the problem with spamgourmet. Nobody is selling it, so nobody except volunteers are supporting it.

If something goes wrong, then you better not be the one who implemented such a system because your ass is definitely going to be on the line.

Email is sacred to companies. Why the heck do you think anybody is going to use such a system? The author of the review did not EVEN consider

So as I stated in the grandparent, you are a troll. Don't bother replying to this message.

Re:Copycat, clueless cat

[joel_snyder]

I'm not sure that I agree that putting AV in front of AS is going to change much. What we do with AV is what people should be doing with AV today: delete that crap. So what that means is that the AS products didn't see viruses, or cleaned messages. Back before mass-mailing worms, we used to see a few viruses a day that could be cleaned and such, but now we see a constant flow of about 1000/day (about 10% of our flow) where they are ALL worm traffic. I can't find a non-worm message in this month's log, in fact.

So what would the effect on AS be? Well, some AS products DO detect mass-mailing worms (sure, why not); those guys didn't see them. Some AS products do NOT detect worms; those guys didn't see them. So I am not sure that we're going to see much of a change in the behavior, given that whether a worm is spam or not is something that varies from vendor-to-vendor. (It tends to be spam if the vendor only makes AS tools; it tends to be a virus if the vendor makes both and packages them in a single device).

In general, our detaching the product from first-hop-SMTP had to have some effects, as you & I have both just noted. Intuitively, what I think you would see is a higher spam catch rate (i.e., lower FN) but no change in the FP rate. For example, a common technique we are now seeing is that spammers will hit an open HTTP proxy, and do a POST to smtp-server:25 with a bunch of crap that happens to also contain a valid SMTP transaction. Many SMTP servers will accept what is effectively pipelined SMTP (even if it is not negotiated). So, if you were in the SMTP engine, you could look for "POST / HTTP" before the HELO/EHLO and say "this guy is a spammer, screw 'em, reject."

That would increase your spam catch rate if the spam would not otherwise have gotten caught. It should not increase your FP rate.

One thing I thought would be true is that people would have lots of subtle tuning that they wanted to do. So we opened up this review to let them do that---everyone could SSH or RDC into their boxes and try and tune things up. What we observed is that only a tiny number wanted to do that. My thinking is that the vendors did not have very subtle dials to turn regarding spam catch/false positives that might have been affected by this insulation we were artificially creating. So, I am inferring that FN might have gone up by some small number but FP not at all.

I regard that as A Good Thing. I think that 1st generation (whatever that means) anti-spam was VERY 'person' intensive in terms of tuning, management, etc. Actually, I KNOW that they were intensive; I wrote an anti-spam product back in 1995 that effectively implemented what we call "Graylisting" now, as well as put in rate-based controls. That was great for its day, but you had to watch it all the time to make sure that big senders didn't get blocked.

However, I believe that most email admins are sick and tired of playing with their anti-spam software. I talked to a bunch of them when I was constructing this test because I thought that the antispam market must basically be "done." Don't we already have nearly 100% adoption of anti-spam products? The answer is "yes, but..." Most of the people I talked to were looking for a better solution. They were early adopters and were dissatisfied with the amount of effort that went into maintaining their anti-spam solution. (Or they were looking for features, such as quarantine, that might not have been in the original product they bought.) They want it to work, and they want it to work well without a lot of screwing around. This is not an unreasonable thing to ask. We used to screw with O/S tuning all the time; then the O/S guys figured out that they could do it better than we, and now we rarely have to do that. Same thing for databases, etc. Now, the anti-spam guys are raising the state of the art as well.

This is why a lot of new products (and services) are coming out which are essentially untunable or are only tunable at a very coarse level. The