Slashdot Mirror
Spamassassin Beats CRM-114 In Anti-Spam Shootout
Simon Lyall writes "A new study of antispam software shows that Spamassassin performed well in various configurations along with Spamprobe , Bogofilter and Spambayes also came out good while CRM-114
failed to live up to its previous claims . The study shows: 'The best-performing filters reduced the volume of incoming spam from about 150 messages per day to about 2 messages per day.'"
the mozilla spam filter does a very good job too, when it learns enough it becomes over 95% acurate. i dropped evolution for it , and never looked back
Putting a windows cd backwards, plays evil messages, but it gets worse, putting it right, installs windows.
The best-performing filters reduced the volume of incoming spam from about 150 messages per day to about 2 messages per day.
How many false positives though?
John Kerry is a Joke!
I use Spamassassin. Surviving mail then goes through CRM-114. At least in my case, it works better than each of the filters on its own.
I wonder how Mozilla Messenger/Thunderbird's spam filtering stacks up against these filters? I've heard some negative comments about the Mozilla filtering system, but it's worked wonders for me.
Is to do away with current email protocols and go with new ones with verification.
That should take care of the problems. The gov is now concentrating on this.
- I only use my main email address with friends and family, and never post it online.
- Whenever I post an email address or register for anything online, I put thatsite@mydomain.com as my email address.
- All email is received by one account, but each message can have a different "to:" header. I set my filters to filter mail to different boxes. Email sent to amazon@mydomain.com goes to the amazon folder. Same with ebay, slashdot, whatever.
- Any time I start receiving spam, I just set my mail server to disregard email sent to whatever email address is getting the spam, and I can stop doing business with the company that sold my email address.
I receive on average 0 spams per day.Albuquerque PC
OSX's built in mail seems to be pretty close to the accuracy numbers listed in the above summary. I tend to have one to three pieces of spam slip through which are almost always entirely image based with some poetry or equivalent attached.
I must say I've been pleasantly surprised with the spam filtering it provides and it has been a lot easier than the hoops I used to utilize to clean out my inbox.
--- I do not moderate.
It's unforchunately that DSPAM was left out of this very good quality report. I have personally used SpamAssassin, SpamProbe and DSPAM
After using each for a couple months at a time, I found DSPAM to be by far the most effective (after it was properly trained)
DSPAMS claim "DSPAM (as in De-Spam) is an extremely scalable, open-source statistical hybrid anti-spam filter. While most commercial solutions only provide a mere 95% accuracy (1 error in 20), a majority of DSPAM users frequently see between 99.95% (1 error in 2000) all the way up to 99.991% (2 errors in 22,786). DSPAM is currently effective as both a server-side agent for UNIX email servers and a developer's library for mail clients, other anti-spam tools, and similar projects requiring drop-in spam filtering. DSPAM has been implemented on many large and small scale systems with the largest systems being reported at about 125,000 mailboxes." was quite accurate for me
Also check out some priceless photos Priceless Photos
Gamblers Forum
users. those silly, silly users. i was in charge of spam for my company for the greater part of a year. using an outdated KEYWORD based system > I was forced to read every.caught.message to look for false positives. ...
did you catch that? yeah...i had to go through EVERY 'spam' tagged e-mail that went through the company.
needless to say, after the first week i was ready to gouge my eyes out. but hey, at least i earned that 'i read your e-mail' sticker!
anyways, the point that i'm failing to make here is the cause of the spam...
the damn users.
whether it be responding to spam, putting their e-mail address in every single webform they encounter while surfing instead of working, signing up for spam voluntarily, or whatever the cause may be..
i ran some numbers on the logs, and came to an astounding find.
a few people were getting literally a thousand messages blocked, per month.
i, on the other hand, had maybe one or two a month.
and i'm not a nazi with my e-mail address....but i do take some care in what places i type it in.
an ounce of prevention goes a long way folks.
SpamAssassin used to be super-good for me, but recently it has become a nightmare... even with Bayes filters on and training it with about almost 2000 spam messages that have escaped it before, I STILL get an enourmous amount of spam every day... maybe I'm doing something wrong with the config, I admit that I haven't spent that much time on that, but it seems like it should be working better :-((.
Spam sucks. Everyone stop buying the products advertised and it'll be over. But then again, people will always be too dumb for an easy solution like that (reminds me of the gooback southpark...)
I haven't read everything in detail yet, but one of the things that stands out is that their 'gold standard' representing the best result consists of 9,038 ham messages (18.4%) 40,048 spams (81.6%). While large, the dataset is unbalanced. One of the things that is recommended by many of the filters is training on equal proportions of ham/spam in order to prevent biasing (overfitting).
Their train on errors approach may simulate what goes on with some filters it doesn't reflect the scenario where there is a initial dataset to be trained on _before_ new messages are processed. Instead, each message is in essence 'new'. So in their tests the machine learning filters start out knowing nothing, but SpamAssassin starts out with its inbuilt ruleset. Not exactly fair.
-Greg
just my humble opinion...
i use email for business and receive many letters from clients. i just afraid to loose any of these because of a spam filter. therefore even when i used one, i checked all the emails anyway.
I can certainly see how waiting on our government will decrease the number of messages transmitted through my mail servers daily.
It's reassuring to know that the "authorities" have effectively reduced the number of messages through my server by 10-14k per day......What great guys, those 'authorities', aren't they thoughtful and quick to respond. We've only been waiting for a spam-relief law for....10 years and they finally gave one to us. Oh wait....SpamAssassin is what reduced those messages.
The reason we don't wait for the gov to step in and take care of business is that THEY'VE DONE NOTHING SO FAR. You expect me to believe the government will solve my spam problems? I'm not holding my breath.
A combination of RBLs, DNSBLs, F-Prot, and SpamAssassin is what reduced the number of messages sent through my servers. I'm interested in results NOW, not legislation tomorrow.
"Lame" - Galaxar
And it has just now learned to filter out almost all the spam. IIRC, SpamAssassin said it would learn what to mark as spam after a couple hundred obvious spams and the same number of obvious non-spams. I still get the occasional false positive.
It's getting just plain rediculous. When I started keeping track about a year ago, the email filtering system I set up was blocking about 10,000 spams per week for just under 1500 users. Last week, it blocked over 170,000. That is an average of over 100 spams per user and the vast majority of my users don't get any at all. There are a couple dozen that get the vast majority of it. Of course, these are addresses that would be a major pain in the ass to change because of all the people that would have to be notified, and only if I could convince the user they want to. Of course, with this many users, I can't get a good grasp on the number of spams that make it through, but I do know it's enough to have several people continually complaining about it. It's just plain sickening all the resources and bandwidth that gets wasted. I use three different black-hole lists, so about 110,000 of those don't get any further than initial helos, but still. Disgusting. Bring on the protocol change. I've told everyone that I would be willing to work 24 hours a day for an entire weekend to implement a server and/or gateway that uses a new email protocol if it meant most spam would disappear.
But why is the rum gone?
Here's how you assuade false positives:
You keep one account for people who don't know you. You spam check that one. You put that on business cards, use it to sign up for porn sites, and post it on slashdot.
You keep another account for responding to email. You set that as your reply-to. You do not spam check it.
This way, there is a way to reach you for customers, clients and friends that will ALWAYS work. Call it the direct line. And, there's a way for people to introduce themselves to you. Call it the "front desk." Anyhow, with SpamAssassin (which includes a bayesian filter, btw, which can be autotrained to learn spam-like language from other mail it sets up), most of the bullshit calls will be correctly tagged and most of the incoming calls will get to you. I haven't had a false positive in months. But I train the thing like Rocky Balboa.
Hey freaks: now you're ju
For an INDIVIDUAL, Bayesian filter works far better than just the regular SpamAssassin rulesets.
That's because the Bayesian system will LEARN from you what you consider to be spam and ham.
I use SpamAssassin with Bayesian filtering turned on and it catches over 90% of the spam. But then I've fed it a decent sized corpus.
Would be interesting to see how that message sample reacts against more spam filtering technologies, or even webmails with spam protection integration.
Me too. I couldn't check email for about a week and grew 4200 or so spam messages and 300 ham ones. 1 spam misclassified...(but some false positives also).
I try to teach the program the least possible (if a message doesn't look like spam for me, even if it is though, I do not teach it).
I also delete de ADV: (prefix) in the subject and the crm114 spam metadata (TAG) and fix it in general so it doesnt get confused when learning spam.
Bad teaching at the beggining leads to lower quality filtering (I did this at the beggining, not cleaning tags amongh other mistaques).
I tryed spamassasing and got fed up. The rules system made Spamassassin pass as ham everything that spooed a PINE filter. WTF...I deleted the entry, then one day upgraded and voila, lots and lots of spam again. And accuracy was much lower (the PINE problem reproduced with a lot of other "whitlisting rules" that I never needed).
After a week with CRM114, I deleted spamassain preprocessing for my account.
unfinished: (adj.)
I'd like to second SpamSieve. If more than one piece of spam gets through in a day (where each day I receive > 500 pieces of email), I am truly surprised. My stats for June are:
Works for me. Oh, the false positive was a list that I just signed up for. They sent a confirmation mail, I checked to see if it was caught (it was), and marked it as "good". Piece of cake.
I live ze unknown. I love ze unknown. I am ze unknown.
Some guy a few stories back mentioned he was getting 3000 ad impressions and 15 clicks a day or so with AdSense. Which is terrible. At first I assumed he was just oversaturating his visitors with ads. But his ad placement is also terrible. It's at the very bottom of the page where few are going to see it. But he is also over saturating. His pages are very busy with information and the ads are on every single page.
What happens when you constantly shove something in someone's face is that they learn to ignore it. Either consciously or subconsciously. In the case of advertising if someone is shown an ad and they aren't interested and another ad is shown there's a very good chance they won't even notice it. Even if they would have been interested in what it was offering. This is because they were annoyed by the first ad so they just mentally block any additional ads.
This is why the response rate to spam is so terrible. People for the most part just subconsciously ignore it. It's just noise.
Advertisers like radio stations because it tends to be a captive audience. People are very unlikely to turn the station when ads come on. However there is one local station that I've learned to turn the channel on when the ads start because I know I'm going to get to my destination before another song comes on. There are other stations that I don't change the channel on because I know it's just a short break.
Just like the guy pumping out 2985 ads that no one clicks on, spammers would benefit immensly by pulling a large chunk of the ads. People are more likely to notice when they aren't bombarded by ads and the response percentage goes up.
It seems counterintuitive that less advertising means a greater response but that's actually the case.
I normally notice the ad banners on Slashdot because that's pretty much all the advertising there is. I rarely ever notice the text ads. Even though they're placed on the left side in the best position as anyone who scrolls the page is probably going to see them. Slashdot's problem is that the ads blend in with the web-site's color scheme too well so they're pretty much invisible to anyone with a scroll wheel.
On GameDev the site is so littered with advertising that I never notice it anymore. By the time I close the stupid popup ads that circumvent Google's pop up blocker using evil little tricks I'm too annoyed to even look at the other ads.
Web-sites get desperate and think more ads == more money. And the actual result is less valuable ad space because the click thru rate is so low and fewer clicks because users tune the ads out which results in less money than if they had focused on the click thru percentage rather than the number of impressions. If you have a web-site with a high click thru rate advertisers are more likely to pay more because they know that if they show an ad there's a very good chance they'll get a click thru.
But then I'm guess spammers have never taken a course in marketing or bothered to think about things from their potential customer's perspective.
Keeping ineffective ads visible hurts the effectiveness of the better ads. Spammers are in effect destroying themselves in that area. As are ad happy web-sites.
Ben
Work Safe Porn
I've been using DSPAM for nearly a year now, and it's just kept on getting better. I can't imagine life without it now.
I have 17 DNS-based blacklists in front of it, because I would rather block the messages at the network interface than filter them with my own resources, but those that slip through don't stand much of a chance of reaching my inbox. I have had my current email address out there on the web and in Usenet for six years, so I see a lot of junk -- DSPAM stops all but one or two per month. SpamAssassin can't even come close to that.
Warning: This signature may offend some viewers.