Spamassassin Beats CRM-114 In Anti-Spam Shootout

Simon Lyall writes "A new study of antispam software shows that Spamassassin performed well in various configurations along with Spamprobe , Bogofilter and Spambayes also came out good while CRM-114 failed to live up to its previous claims . The study shows: 'The best-performing filters reduced the volume of incoming spam from about 150 messages per day to about 2 messages per day.'"

Comments

Re:in related news

[bigberk]

Content-based spam filtering is a waste of time. . . RBLs WORK

But content-based filters can very accurately determine what is spam and what's not, and so they can feed RBLs/DNSBLs. Let real spam to real user accounts form the blocklist! One such project is WPBL.

Isn't Human Accuracy always 100%

[PetoskeyGuy]

From the CRM-114 site...

News Flash: As of Feb 1 through March 1, 2004, 8738 messages (4240 spam, 4498 nonspam), and my total error rate was ONE. That translates to better than 99.984% accuracy, which is over ten times more accurate than human accuracy

Maybe I'm missing something human accuracy always going to be 100%? I tell the computer what is spam, it learns. I may decide that regardless of what it thinks, this last message is OK. So aside from clicking too fast or changing your mind (which is a common thing to do) how can a filter ever suggest it is be better then people at deciding what people want to see?

Re:Isn't Human Accuracy always 100%

[sholden]

People make mistakes.

Yes, given one message to classify as spam or ham you are going to get it right 100% of the time.

Given 8000 messages to classify the wonders of boredom is going to mean you make a mistake every so often (not an "oops I clicked the wrong button" mistake, but an "oops I put it in the wrong folder because the subject looked spammy and I couldn't be bothered checking the body" mistake).

In practice though, those stats on human accuracy are provided by having one person classify email that has been classified by others - which of course means some of the mistakes in fact be disagreements...

DSPAM

[More Trouble]

In real world deploys of statistical filters, something like DSPAM's "global user" feature is necessary. The ability to begin with a relatively mature dictionary is critical to the user experience. Personally, DSPAM is filtering around 200 SPAMs per day for me, allowing one through every few days. It's 99.985% effective for me.

:w

I'm running SpamAssassin at work.

[khasim]

People LOVE it.

There are some false positives and some false negatives.

But I have it set to delete anything 12+. That gets rid of the worst of the worst spam. So far, not a single complaint of any email being deleted.

Everything else has the subject re-written so people can run their own rule set against it.

In the past 8 hours
1867 messages received
375 messages deleted
1266 messages flagged as spam

So, only a few hundred actual, good emails.

Of course, that's only 4 hours during the regular work day (and 4 hours after work). But you can see the proportions. It saves people a TON of time.

And it makes them happier when they don't have to constantly dig through crap to see if any real messages have arrived.

Now, those spam messages are NOT distributed evenly. Our HR manager had her email address posted on the website. So she gets about 20-25% of the spam.

It's not exactly Big Brother 'cause no human sees the deleted spam.

Re:Why don't people use catch-all accounts?

[FrenZon]

Why don't people use catch-all accounts?

Because you will always have one main 'obvious' address - be it something that goes on your business card, or something you tell to people you meet. For example, I use glen at glenmurphy.com.

Now all it takes is one slip - someone you know to get a virus, whatever, and your address is 'out there' for the taking. Your only possible recourse then is to stop using that address, but for some people that's just not an option, and it's a just bit defeatist to sit there surrendering email address after email address.

Re:Why am I so Blessed?

[dasmegabyte]

Because you don't put it into wierd text boxes, you don't use newsgroups, you don't have any enemies, you don't have any domains, and you don't have it in plaintext on your website.

I do all 4. I get my share of spam. It's not a HUGE deal, but it made it worth my while to get a spam filter.

Re:The Mozilla ThunderBird SPAM filter

[norton_I]

Better to do spam filtering with your MTA/MDA anyway, if possible. That way, the same filter is used no matter which email client you use from which computer. Plus, it means you don't have to download spams to your MUA when on a slow connection.

Now if only I could get the rest of my mail configuration to be shared between evolution, mutt, and squirrelmail.