Slashdot Mirror
Seven Spam Filters Compared
Goo.cc writes "Those wondering how their spam filtering software performs in comparison to other's may want to read this article on Freshmeat, where Sam Holden performs comparative testing of various popular e-mail filters. The filters tested includes Bayesian Mail Filter, Bogofilter, dbacl, Quick Spam Filter, SpamAssassin, SpamProbe, and SPASTIC."
Sounds great, but until I hear about software products like these in my morning mailbox, I don't really trust that they're any good.
people/editors need to learn the a tag
clicky
Spam Filters
http://freshmeat.net/articles/view/964/
Click here.
silly editors. Yay for SpamAssassin..
Ahh--I have to copy the URL to the clipboard and paste it in the URL bar. Seems to me like I'm browsing text files.
The author makes a good attempt at comparing these products, but I don't think his samples are indepth enough to come up with real-world results.
For Bayes testing, he used 68 spam and 68 ham messages. Spamassassin for one won't even activate bayes until it's learned from 200 messages; it's not uncommon for those who regularly deal with spam management on the server side to use 5000-10,000 message corpuses to test new rule additions and to train spam.
The low number might have a slight effect if most of your mail contains similar characteristics, but I'd much rather have seen bigger numbers of samples.
-Barkeep, a draft of your most hazardous brew, for the world is slowly stepping into focus, and I don't like what I see.
IMO, the best way to go with spam is to combine a heuristic filter with a text/baysian filter, in my case SpamAssassin and SpamProbe. I run them both, and it does a noticably better job than either running alone.
SpamProbe can be fooled by clever spammers who insert lots of common words in non-visible html. A Baysian filter can't really catch that, but a heuristic filter can be written to notice the pattern.
Also, set up your Baysian filter to re-learn regularly from your spam folder. SpamProbe adds a unique ID to each message, so it won't process a message twice. Therefore, you can just manually move any false negative spams into the folder, and they'll be learned from.
I have seen at least two of these comparisons and no one seems to want to roll Mozilla's spam filter into the mix and compare it. Therefore, the comparisons are kind of useless to me. I am guessing I am not the only person using Moz either, for specifically this reason (ease of use for Bayesian filtering).
What's up with that? I know it's not a proxy, so the methodology is different than most of the products in the comparison. I'm very interested in how well the filter works however, compared to these other products.
HBI's Law: Frequency of calling others Nazis is directly correlated with the likelihood of the accuser being Communist.
Don't complain, they're trying to keep from slashdotting the server.
Ok, probably not, never attribute to benevolence what can be explained as well by stupidity.
The address is there though, before you complain about them being too stupid to make a link you might ask yourself if you're really too stupid to cut and paste. Cuts both ways.
Article isn't really anything new, but a decent quick rundown on the current state of the field I think.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Does anyone find it disturbing that --
a. Spam Filter software company is now a "viable business."
b. Spam Filer is needed AT ALL?
ELOI, ELOI, LAMA SABACHTHANI!?
As was noted earlier, the set of messages given to the filters for learning was terribly small. Furthermore, SpamAssassin wasn't tested in a way useful to most as the tests in this article didn't take into account SA's Bayesian filter nor it's network-based tests (Razor, etc).
Also, what's with keeping the spam threshhold score secret?
How the heck could Active Spam Killer be left out? I used to get about 150 spams a day and now I get ZERO. No false positives, no false negatives.
It is an autoresponder that checks the sender against a whitelist and a blacklist. If a new e-mail is in neither, then it bounces back an e-mail asking for a confirmation that the sender is a human. Simple!
I think OSDN can handle being slashdotted because they ARE slashdot
I noticed immediately that the author turned off SpamAssasin's Bayesnian filter, claiming "it already has 5 points, that's enough". WTF does that mean? The whole point of SpamAssasin is to do a lot of tests, and add the scores together- and then set the threshold you want(something he also doesn't modify- I changed my threshold after looking at the scores spams were getting and such.)
I trained SA's bayesnian filter off of about 3 years of spam and legitimate email sent directly to me. SA as a whole is working nearly flawlessly- the only messages it has tagged as spam were those from users with improperly configured email clients AND suspicious email addresses AND using only HTML. Ie, a message that would damn well look like spam. However, like I said, I lowered SA's threshold by 2 points because I was having too many false positives(that was before I had properly trained the Bayesnian filter, so perhaps I'll kick it up a point now.)
One important note- when you get a falsely classified message, it's REALLY important to tell Spamassasin's bayesnian filter about it. It's as easy as cut+paste if you do sa-learn --spam/--ham --single, hit enter, paste the message, hit control D. Done!
Please help metamoderate.
- Contains my initials. I simply ask my friends to insert my initials in the subject line. They're all happy to comply.
- If I opt-in to something, like
/. updates, I allow *that* domain (*@slashdot.org, for example). No third party co-brands are accepted.
Fair enough?Instead of going after Spammers, why not go after the companies that hire them to send us Viagra/Penis Enlargement/etc mails? Without them, no Spam. Also, I'd like to know who the fucktards are that repsond to these mails and buy their products.
Maybe the site is too valuable to DoS?
They started off by quoting John-Graham Cumming, et they didn't include his brainchild PopFile.
Check it out Here.
GeekWares - Buy and Download Today!
What about PopFile? I've tried SpamAssassin and a few others, and I like PopFile the best. After a little training it's EXTREEMLY accurate. It survived the deluge of mail I've gotten in the last few days (due to virii) with flying colors.
According it it's internal statistics, it has classified 2821 messages as of the time I type this. It has made only 95 errors (often close calls, so I don't blame it). That puts it at an accuracy of 96.63%. For the record, of the e-mail I've gotten, it's 308 messages of ham, 2513 spam.
I have only been using PopFile since June 7th of this year, but it's working fantastic. The only thing I've used that's this good was Cloudmark's SpamNet, who stabbed the community in the back, so I switched to something else. I'm glad I've found PopFile, and I suggest you try it too if you're looking for something good.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
See our PSAM project site for a refereed paper evaluating several machine learning spam filtering techniques (although not specific filters). This site also contains large standardized corpora for evaluation. The paper contains a number of tips on evaluating ML spam filters.
The /.-referenced article has some good ideas about evaluation. I particularly liked the explicit discussion of the false positives. The recommendations at the end are excellent. On the other hand, the evaluation isn't across a broad or obviously representative corpus, many of the tests are a bit odd, the ROC tradeoffs are not discussed. In particular, the evaluation set for the tests did not include enough ham to be able to accurately estimate the false positive rate: consider what would happen to the precision estimates if 0.5 were added to each of the numbers in the false positive table.
Overall, though, this was an interesting evaluation, and I'm glad that the author published it.
Whoops! The real url is
http://popfile.sourceforge.net/
GeekWares - Buy and Download Today!
forgot to mention the last part of my address is @notsohotmail.com and that isnt hard to remember. Havent ever gotten spam.
The Television Wiki
That's right! Our company has found a high-tech way to use various anti-spam tools to enlarge your penis. My pennis is noww sso lrage that i Cannnot type curretcly. Itt gtes in teh way.
Please visit www.spamfilters2enlarge.com
Act before midnight and get a $30 discount.
Table-ized A.I.
Of couse your baysian filter will QUICKLY learn that html tags that create invisible text are VERY common in spam and nowhere else-> problem solved
Dont forget that the filter sees more than the eye...
HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
If you decide to try out spamprobe or another bayesian filter, try this web interface which lets you easily reclassify mail, even those marked as spam. I found that "training" the bayesian filters was the hardest part; this definitely simplifies the process.
It wasn't mentioned in the article, but I really must plug popfile. It filters out my spam yes, but it is also a general mail categorizer. It sorts ten yahoo groups for me, personal, work, and school related emails. I know you think you could do this with rules for the emails, but for example, I get several hundred emails a day from the Harry Potter for Grownups List. Popfile can sort them into 'probably interesting' and 'probably not' for me. Very nice.
An interesting thread here about how TMDA, a C/R filter, used in conjunction with SpamAssassin, can provide the best of both worlds. While TMDA is by itself effective, there seem to be some humanistic issues involving the assumption that all e-mailers are spammers unless they prove otherwise. The thread explains how Bayesian filtering can be improved by using a decent C/R filter like TMDA without alienating people that send legitimate e-mail.
Personally, I figure anyone thin-skinned enough to be insulted by my C/R filter probably isn't worth talking to anyways, but I digress...
The quickest way to stop spam in the U.S. would be to have a respected person such as the Surgeon General of the United States say that
1) There is no way to increase the size of your body parts,
2) The cheap Viagra is not Viagra,
3) and so on.
We can help by telling everyone we know not to buy anything from spam. Next time you are at a party or family gathering, make that point.
Spam would disappear if there were no buyers. We need to make it culturally unacceptable to buy anything that is advertised through spam.
I don't know if thunderbird uses the same filter as mozilla, but for me, thunderbird is horrible at spam recognition. I have an account that gets about 50 spams a day, and one legit email from 'word a day'. It consistently screws up even after weeks of training. Thunderbird couldn't find a spam labeled 'young virgin sluts selling herbal viagra from the Congo'. But then again, it's only 0.1, so I'm more than willing to cut it some slack.
Please don't bother your Congressmen or Senators proposing legialation that might not work 100%. Just keep on filtering the spam I send you, I know you would have never bought from me anyway. That you can filter ligitimizes my business and my waste of your bandwidth.
P.S. To be sure of not getting a false positive , be sure to send all filtered mail to a special folder. Waste your storage space storing the mail until you manually go through every piece to be sure you didn't accidentally filter something important. Of course, this will take exactly as much effort as it would have to just check the e-mail when it first came in, not to mention the extra effort spent in setting up the filters and the extra space for storing your incoming spam folder, but what the heck. You geeks enjoy wasting time this way, and I certainly appreciate it. It makes the work of all us spammers much easier.
I'm using the standalone Thunderbird and it catchs everything that passes by Spamassassin. Spam is marked but never deleted, so I can go back and check. Some spam programs will delete email, which could delete a good email, unacceptable.
Basically, I'm using a mandrake linux box, imap, procmail, fetchmail and spamassassin. Easy, and I can send/receive email from my linux box, and port 25 is blocked from the Net so nobody can use me as a bouncer.
Only problem I had was, there was no complete document to set this up, I had to piece each part together.
So for anyone who wants to know, heres the quick steps.
1. I'm using mandrake, but had to update SA for the sa-learn utils. (Gotta train SpamAssassin)
2. Setup fetchmail in your personal account.
3. Setup
DROPPRIVS=YES
VERBOSE=ON
LOGFILE=/home/userac
|
4. Setup your user_prefs in your local directory for SA. (mine, but im no SA expert, but it works)
required_hits 5
rewrite_subject 0
use_terse_report 1
report_safe 1
use_bayes 1
auto_learn 1
ok_locales en
use_pyzor 1
pyzor_max 9
pyzor_add_header 1
use_razor2 1
always_add_headers 1
always_add_report 1
spam_level_stars 1
pyzor_add_header 1
skip_rbl_checks 0
#timelog_path
5. As root make sure Imap,Spamassassin is running.
6. Load Thunderbird, use Imap, use filters on x-headers.
Neither has I, but then I do not publish my address
on the net. When I create "register accounts"
on hotmail.com or mail.com they get flooded right away.
Anyone care to point out a decent way to use SA's bayesian filter with this setup:
.forward and .procmailrc to do this). I'm currently deleting spam (score = 5)
I have a linux box running as my web/mail server that has spamassassin on it for anyone who wants to use it (setup
The problem is how to get spam and ham from Outlook back to the linux box correctly. To my knowledge, outlook doesn't export mail in any way that's readable by the sa-learn script. I'd like to setup a bayesian filter, but it seems like a lot of effort to get rid of the 4 or 5 spams that SA actually does let through each day.
"It is seldom that liberty of any kind is lost all at once." -David Hume
I use SpamBayes (free) with Outlook on my W2K machine. I trained it with over 400 SPAM and over 1000 non-SPAM emails. I get about 45 SPAM each day and my ISP, attglobal, filters out about 40 of them. The SPAM that gets to my mailbox are the ones that pass through the attglobal filter and that filter has NEVER given me a false positive for more than 2000 SPAM. Those SPAM are put in special folder on the server for inspection but I now just delete them en-mass every week or so.
That means that SpamBayes is filtering only the hardest emails to classify and so far it has only given me one false positive. I got one false negative after training it for the first time. SpamBayes also has a folder for messages that it is not sure of and so far they have all been SPAM. I seldom have to do more than inspect the sender and subject to confirm that they are SPAM.
Each time a message is automatically moved to the SPAM folder (or moved back to the Incoming folder) the training set is adjusted for that email so I don't have to re-train.
To sum up I'm really impressed by well designed Bayesian filters and this one in particular. I think it's worth while to take the time to build up a corpus of SPAM and "good" messages as I can then evaluate competing filters.
Nate
I am not sure about getting spam with such an addres ssaf4502@E8Hkl3.biz . I AM certain , however, that i would not receive regular mail.
You can not put it in a bussiness card, people will always type it wrong. You definately cannot pronounce it over the phone.
In fact, most would give up on contacting me through e-mail just looking at this monster.
Slashdot Sig. version 0.1alpha. Use at your own risk.
This is a pretty bogus "fix". It might work if you set up such an account and never use it, but if it's used and gets into a spam database the computers can proprigate this e-mail address just like they can any other. The spam database computers simply don't care if the name is "joe" or "saf4502", they deal with both exactly the same. All you'll really do is make it harder for you to pass along an e-mail address verbally to someone.
Spammers get these addresses any number of ways. Many are harvested tens of thousands at a time. If you ever use that e-mail address in a usenet news group, for example, it will get harvested. Of course, you can munge it and give instructions in the post for how someone wanting to reply should unmunge it (replace the number in my name with the square root of the number) but realistically few are going to bother to go to extra work to unmunge an e-mail address, so if you made a post to really try to get some information back rather than to just hear yourself talk, that's a big waste.
Same if you want to post a contact e-mail on your website.
Businesses you deal with are even less likely to unmunge your e-mail address, and if they do you certainly have no protection that they are not the ones about to sell their mailing list database to a spammer.
And even if you just keep your e-mail adderess for close personal contacts, one of them may eventually come across what they think is a "cute" electronic greeting card site on the web and give them your address to send some damn picture of a dancing bunny, or use your e-mail address on some site with an "e-mail to a friend" link for a story they think you would be interested in, or even just let their computer get infested with some worm that goes through address books, and your adddress is in some spam database, soon to be in thousands. Having a hard to remember e-mail address is no more protection than having an easy to use one is.
I even created a dummy e-mail address one time on Mindspring, with a very uncommon name and numbers. Never used it. It started getting spam after a while. Either Mindspring sold the names, or they had a bad security system and some employee sold the names, or they had a really bad security system and someone hacked in and harvested the names.
I'm an American. I love this country and the freedoms that we used to have.
since the filters do better after being trained with lots of spam, anyone think of gathering up a huge collection of spam to give to other people? i mean exporting a corpus of spam from outlook, sticking it up for download somewhere, and letting other people import it into a spam folder. then other people could run their filter of choice and train it!
you could even make it all official-like, and somehow guarantee that the spam that's up for downloading is "official" and "virus-free" and "safe for your computer." you know, do geek stuff like check hashes or whatever it takes to verify that the spam collection is legit. whatever it takes to ensure that someone else hasn't filled it with a ton of virus/trojan/etc. attachments. or whatever. i dunno. you know, somehow guarantee it's safe.
imagine it! download spambayes, get spambayes to connect to the official spambayes spamcorpus server, and download the latest 2000 spams! instant training.
anyway. just an idea. mod me down as -1, herrd0kt0r. 8P
SAProxy for Windows (Based on SpamAssassin) got the highest marks.
Simply adding random text to a message is not enough to get it past SpamAssassin.
I run SpamAssassin, I know that it catches that stuff.
The reason it does catch it is because it used a WEIGHTED system for classification. If the message has the characteristics of spam, but has random words in it, it will still be considered spam UNLESS those random words have been used previously in ham messages that it has learned.
Now, the odds of the spammer hitting upon words that my version of SpamAssassin has learned as ham are very slim.
And if he did manage it, those same words would most likely not be in someone else's ham list.
So spam that can get through to me will not get through to 90% of the other SpamAssassin users.
I'm running SpamAssassin at work and it is catching over 1,000 spam messages for every false positive or false negative that it lets through. Despite the spammers including random words and random text and all of their other tricks.
i use apple's mail.app with bayesian filtering. i have received maybe 4 or 5 true spam emails in over a year. i haven't yet missed any real emails either. i would have to say that's pretty good. otoh, our groupwise system at work is fscking horrible. i get tons of fscking spam. i have had to set dozens of rules, and it still doesn't matter.
My problem? I was perfectly gruntled, until some numbnuts came by and dissed me.
I use Pine when I'm at work (ssh into my box at home), but generally speaking, I use Outlook when I'm in winders or Moz when I'm booted into linux on my desktop.
I suppose (as the other poster) mentioned, that I could turn on IMAP, but like I said before, it sure seems like a gigantic pain in the ass to do nothing more than filter out a few extra emails a day.
"It is seldom that liberty of any kind is lost all at once." -David Hume
I dont think you get the point. Notsohotmail.com is easy to pronounce and remember. You are purposly ignoring my second post to prove your point. I am surprised that this was modded up for insightful when i clarified my point already. I was kind of joking and made a mistake about that.
The Television Wiki
If you don't worship your tinfoil hat, you should check out mailinator.com. You don't need to set anything up, you just check the mailbox that you just made up.
Nerd rage is the funniest rage.
Comment removed based on user account deletion
If you reread the slightly ambiguous sentence in context you will realise he meant he had evaluated five baysian filters and felt that was enough. Nothing to do with Spamassassins point system...
My time and the time of 100,000 users is not.
And since the stuff like the spam filters are getting pretty generic, they can be configured and replicated to numpty users reducing spamming effectiveness by several orders of magnitude.
Poor attempt at irony BTW.
Government of the people, by corporate executives, for corporate profits.
I'm not disagreeing with the posters that stated that he has low sample size. It might be one of the problems why he doesn't have a higher catch or recall rate.
The main problem I see with bayesian filters is that they are complicated and nontrivial to set up. I've been playing with Bogofilter for several months. And even with sub 1000 corpuses, I get a very high catch rate (greater than 90-some %, though I don't have exact numbers).
The method that I've employed is start with a small set of three hundred or so ham and spam corpuses, then to train on error over time. It's a pain in the ass because I still have to continually inspect the results and tweak the databases.
In addition to that, there are at least a half a dozen parameters that contribute to the success or error rates. So much so that bogofilter actually comes with bogotune to analyze the corpuses to suggest optimal parameters.
So give the guy a break. I wouldn't say his results are robust enough for an academic publication, but it isn't worthless. It's interesting enough for a read. It's more work than many of us are willing to do.
Also an interesting read is Comparing Bayes Chain Rule with Fisher's Method for Combining Probabilities.
I second the comment about using IMAP. I have been using it very successfully and it makes it easy to move spam inbox messages from whatever email program I'm using into a spam mailbox. I then have a script called learn-spam.sh that could be set to run each night to reclassify spam / ham.
I use IMAP to read my mail, mostly because that makes it easy to read from both work and home, and occasionally when I'm on the road. Right now I'm using the bayesian filter in Mozilla. It's great, but since it's client-based that means I have three seperate filters I need to train. Sometimes I'll run into weird problems where two of the filters think an email is good but the third thinks its spam. If I accidentally left the third one running at home when I went to work, it will sometimes decide to re-classify my inbox and make messages "magically" pop in to the junk mail folder behind my back. Not good.
What I'd love is a filter that I could run on my server box at home and point at the IMAP mailboxes at my ISP. I'd want it to filter the messages and move the spam to the Junk IMAP folder rather than a local one. That way all of my mail clients would be seeing the same thing and using the same training data. I'm not sure what the UI to this would be -- there would need to be some way to train the filter in both bulk (this folder is all spam) and individual (this one message is spam) modes.
I've done a bit of looking for a tool like this, but I haven't found anything that looks ideal yet. Some of the filters mention that they support IMAP, but it's unclear whether they're optimized for a multiple-client setup like this. For example, the IMAP-aware Outlook plugins (in SpamBayes?) wouldn't do the trick.
Does anyone know if such a thing exists? I'd prefer one that ran on Windows, since that's what my server runs right now. (I know, I know. But it was very easy to set up, and I'd rather spend time improving my programming skills instead of leaning to be a Linux admin. I was a 4.3BSD admin way back in the day, but it's been a while.) If there were a great solution that only ran on Linux, that might motivate me to switch, though.
Any advice?
Messages classified: 6,116
:)
Classification errors: 88
---
Accuracy: 98.56%
And THAT is with 8, yes EIGHT, different buckets for sorting my mail. Of course 79% of my mail is spam so
Your hair look like poop, Bob! - Wanker.
Surely this article should have been written by Spam Holden?
sig:- (wit >= sarcasm)
When, of course, most spam has forged senders.
Whee, looks like another idiotic pattern I have to bock.
If corporations are people, aren't stockholders guilty of slavery?
Sam's article was a very interesting read, but his results need to be taken with a grain of salt.
To show that one piece of software outperforms another, you need to prove statistical significance. This can be done in two ways:
The first method is called the pairwise t-test. What you need to do is to run k tests using different training and test data. For each of these tests, you find the accuracy of the classifier (#success/#trials). The, you form the "t-statistic," t = d/sqrt(sigma_d^2 / k), where d is the difference of the means of the two classifiers, sigma_d^2 is the variance of the difference samples and k is the number of samples. Then, you compare your t-statistic to the Student's distribution with k-1 degrees of freedom. Typically, you want a confidence level of 90% or 95% so you find the number of standard deviations away from the mean for the specific t-test (e.g. the 90% statistic 9-degree of freedom t-test is 1.38). If your t-statistic is greater than the number of standard deviations, then the difference between the two classifiers is statistically significant with X% confidence. Read more about this in Witten and Frank's Data Mining book.
The other method is called Analysis of Variance (ANOVA). I'm not familiar enough with this method to explain it here, but it allows you to choose from a set of experiments which ones really are above the average. Dig around in your statistics books or on the web for more information.
Sam should have made use of either of these techniques when doing his analysis. Since he only ran one experiment per configuration of his classifier, you can draw no real conclusions from the data presented (it's a Student's distribution with 0-degree of freedom... essentially flat!).
Since most of us only have a small number of corpora kicking around (maybe even only one!), you can use a method called "cross validation" to give yourself a larger number of data sets than you actually have. When doing a cross validation, you divide your corpus up into k "folds" and then perform k experiments. In each experiment, you set aside one fold of your data for testing and train on the other k-1 folds. Since you're using different test data each time, each experiment can be considered to be different and then you can use a pairwise t-test to prove statistical significance. There are other methods that you can use such as "leave one out" where you have as many folds as you do pieces of training data and "bootstrapping" where you sample your training data with replacement and test with whatever wasn't sampled for training.
However, cross validation may not be appropriate for incremental learning algorithms if your data is on a timeline (such as e-mail). You can break your corpus up into pieces and do your evaluation on that.
Proving statistical significance is very easy and allows you to be confident in the conclusions that you make in your publications. It's the scientific method!
Good luck!
Henry
Personally, I wish that he has included DSPAM and CRM114 in his testing. Otherwise, I thought that it was an enjoyable review.
Could you please forward that email to me... I have a friend from Nigeria that would probably like to become a business partner with this group from the Congo.
--Kevin
/joeyo
2^5
I use bogofilter, and it seems to me it would take far too much of my time to manually feed my own spam to it for training purposes. What I do instead is this:
/home/bogofilter/spamlist.db
We have several spamtrap addresses on our sendmail server. They were not intentionally set up as spamtraps, but in looking at my mail logs I noticed that there were many email addresses receiving spam attempts that are not and never were valid addresses on our system. These invalid addresses somehow got into spammers' email databases and they receive nothing but spam.
So I set up entries in my aliases file to automatically redirect all mail for these accounts to bogofilter's spam database. Here is a sample...
nikola: "|/usr/local/bin/bogofilter -s "
cal: "|/usr/local/bin/bogofilter -s "
bwilson: "|/usr/local/bin/bogofilter -s "
fayre: "|/usr/local/bin/bogofilter -s "
(If you are also using sendmails access.db to filter mail based on the source IP address, you may want to set up the spamtrap addresses as "spam friends" so that spam directed to them is not filtered out by your IP address filters.)
To keep the spam database fresh and to keep it from growing to an excessive size, I use a daily cron job that automatically deletes spam entries older than 30 days...
# remove records older than 30 days from spamlist.db
/usr/local/bin/bogoutil -a30 -m
This gives me an 8 Megabyte spamlist.db with about 14,000 emails in it which is constantly refreshed to keep up with the latest spam trends.
Maintaining the non-spam database isn't quite as easy. I use bogofilter's -u option on my own incoming email, which tells Bogofilter to update its databases with my incoming mail based on its classification of the message as spam or non-spam. I never get a false positive, but I do occasionally get a false negative which requires me to make a correcting entry in the database.
I ignored your second post because it does not compute. You mean your address is: saf4502@E8Hkl3.biz@notsohotmail.com ?
especially when you put two "@" characters. Postfix, interprets this as
saf4502@E8Hkl3.biz.notsohotmail.com
(replaces all @ after the first one with dots.)
So there is something wrong with this.
Slashdot Sig. version 0.1alpha. Use at your own risk.
Maybe you don't have this problem. I never did until becoming the list manager for my cycling team, then the bounces and spam (to a list alias that is NOT advertised anywhere) started flooding in.
Mod parent up
I'm an American. I love this country and the freedoms that we used to have.
Jeez, what a saddo. He or she just set up a /. account to post this crap again...
However, I need to take exception with this bit:
Just go through lots more work to set up special filers on your computer
I have just set up a Network Appliance F840 filer for NAS on our network, and it we very easy indeed!
Call me old fashioned, but I like a dump to be as memorable as it is devastating - Bender
Comment removed based on user account deletion
Complete BS.
Geeks are ones that set up the spam filters for everyone else. End users will no more have to install spam filters than they have to install DNS entries, multi-peered lines ot the backbone, etc. (In fact, the problem is that often ISPs don't tell you they are filter, or give you the chance to turn it off.)
Your filters really help cut down on the complaints to the Internet service providers I do business with, and as long as not too many complaints come in their marketing people assure me we can do business.
Sorry, but my delete key is tied to your ISP's abuse box.
Ok, I actually have a separate "this is spam" key that send the spam off to spamcop. I also use the following procmail script to report anything that scores too high on spamassassin:
The spam_report script is very simple, it just encodes the spam and sends it off to spamcop. It can be found on http://spamcop.net/reporter.pl. I modify the number of stars (spamassassin score) depending on how much time I have on my hand right now. If too many reports get sent to spamcop for me to deal with, I increase the number of stars, when a spammer pisses me off, I decrease the score.Even a small number of vindictive anit-spammers reporting spam will get the spammer's IP address onto spamcop's DNSBL, which feeds back into things like spamassassin.
The amount of spam that reaches my inbox in the last 6 months has been far lower than any time since the mid 1990s. Even with the reporting to spamcop, I'm spending less time dealing with spam now that two or three years ago. Over the last year or so, I've come to believe that Spammer's days are numbered.
Oh, one final note. The original article complained about the fact that spamassassin mine-defangs the spam and then says that it is hard to get the original email back. This isn't true at all. On older versions, you just run it through "spamassassin -d". While you can still do that with newer versions (as per my scripts above), they now create an attachment so you can just click on it if you want to see it.
SPF support for most open source mail servers can be found at libspf2.
I recently switched from bogofilter to SpamBayes. While it still shows the minor issues a young project always has (incompatibility with the dumbdb in Python 2.2.2 of SuSE 8.2 so you have to use gdbm as the internal db driver etc), i consider it one of the most promising spamfilters around.
Sure, it's only one data point, and next week will be different, but i think i'll stick with SpamBayes for now.
It's a combination of Bayesian filtering and whitelists based on your address book. When you first start the application, it goes into pure training mode in which junk mail is flagged but not filtered out of your inbox automatically. You train it for a while, labeling junk yourself and correcting false positives. After the training mode is sufficient (no more false positives at all for a set period of time, though there usually aren't any as anyone in your address book is whitelisted and everyone you hold a correspondence with is added automatically) the filter then prompts you to go into automatic mode, in which it separates junk into its own box. After 10 days or so in the junk box (you can set the exact time, including never), the messages get deleted. And for those annoying people who forward jokes to you but are whitelisted anyway, enough training can actually selectively overcome the whitelist, it's really very cool. For the occasional piece of SPAM that makes its way into your mailbox, you can select it and press the junk button and it immediately banishes it to the junk box and learns for the mistake.
I understand that it wouldn't have worked out considering the methodology behind the tests but I'd be interested to see how Apple's Mail.app compares.
SpamAssassin is used in quite a few of the commercial offerings, it can also filter before passing on for internal delivery. I'd guess that one or two of the others can too, it's not difficult
Custom Rules For SpamAssassin
It seems that the worst part about spam is wasted bandwidth and processing power. Wasted electricity from undesired messages being shoved through fiber optic cable seems like a waste, then even more power to process and discover if it is spam or not, then you have false positives. I think a better solution would be to weed out all the spammers, maybe take the internet away from countries that allow spammers or somthing?
Sig: I stole this sig.
So the results aren't quite up to date. I've trained it on a couple months of spam and non-spam and it seems to significantly improve its classification.
Messages classified: 3,545
Classification errors: 110
Accuracy: 96.89%
This is with 4 buckets. My spam bucket received 2,561 ( 72.24%) of those e-mails, with 7 false positives and 9 false negatives.
Oh yeah, POPFile is cross-platform...Windows, Linux, anything that will run Perl (Windows users, don't be afraid. The installer installs an interpreter for you - you'll never know it's there!)
the crippled SpamAssassin did pretty damn good though.
I don't think it's totally unfair to run SpamAssassin with the bayes disabled in these tests, a lot of people run it that way in the real world, especially on mail gateways where no provision has been made for training & retraining.
We just need to remember that on every score for SpamAssassin in those tests, it can do a lot better. I've heard good things about a few of the others, but SpamAssassin's nothing short of a miracle here, 2 false negatives and one false positive last month on approx 54K messages.
Custom Rules For SpamAssassin
How many times are we going to re-review the same old crap over and over again?
:)
btw I agree with most readers here, the comparison is useless.
this is aside the fact its pointless for windoze users (the generators of most spam). Where is a review of Popfile ?
Ohh, I love the BS line about (Paraphrased!) "we turned bayesian filters off for spamassassin because 5 other filters were good enough" - wtf ?
With low data-sets like that, the article is useless, plus this is not a valid method of dealing with spam anyhow.
Has anyone else noticed how this topic keeps get regurgitated over and over ad-naseum?
blatant plug - anyone who wants to discuss anti-spam in real terms contact me (I'm in the process of setting up a sourceforge page too!)
Actually, unless he/she really is a spammer (which I doubt), he/she is just role playing. But the analysis has a very good point. Spam filters really do have the effect of helping spammers focus their mailings to those people who aren't going to complain, and especially might even buy something as a result of the spam. The trouble is that this help is paid for not by the spammer, but the victims of the spammer. I'll dismiss, for the moment, that the buyers of spamvertized products/services are victims. The rest of us incur costs as a result of spam, ranging from the time it takes to press delete, write complaints to whoever, use more disk space because the spam folder doesn't get cleaned out, process each message with the latest craze in content analysis, and just handle each incoming message, or connection, or SYN packet on our bandwidth. Spam filters don't decrease these costs; they just shift them around. They're doing nothing more than sweeping spam under the rug ... for us. And they will help justify even more people and businesses becoming spammers. The oft heard argument is that if spammers have no market to sell to, they will quit and find something else to do. But that presumes 100% coverage of spam filters. It will never happen, and for the most part, the targets of the spammers won't be covered at all, while it will still be cheaper to not clean the lists.
now we need to go OSS in diesel cars
I've been thinking lately that SpamAssassin might have the best Bayesian implementation, with only a slight change.
AFAIK, most/all Bayesian scanners out there simply tokenize the mail and then use the tokens as the basis of the rating system.
However, SpamAssassin adds an X-Spam-Status header to all mails (by default), which contains a list of the various tests (regex, network, or Bayesian) that the mail triggered. If SA were to move the Bayesian scan to after all other tests have completed, then this list of tests passed could be (or might already be) considered by the tokenizer for the Bayesian algorithm.
The benefit to this is that regex's can discern more patterns in the code (or more correctly, equate patterns) and the network tests are fairly reliable. In a large sense, this is using Bayesian techniques to develop a self-adjusting rating scheme the tests. Using this, one could assess, for instance, how much having a host in the relay chain in an RBL influences the spamminess of an email (for instance, a large amount of email originating from SPEWS-listed IPs is not spam; this would imply that SPEWS would have a lower confidence rating in picking out a spam).
the external filtering stages of Sendmail, postfix, qmail or Exchange's SMTP engine. You know, the place where you can run an external program on the email message. Since ALL of the reviewed spam classifiers were chosen because they run from the command line with only the message as standard input and a classification as the output, I'm sure you can write a quick perl script to use it in that context and acheive the mail accept/reject feature you need.
Maybe you add in an extra header: (X-Int-Spam: Yes) to let downstream clients deal with delivery options.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
The only spam filter that is acceptable is one that utilizes challenge-response. There are NO false positives with filters based on this. I cannot tolerate even one false positive with my email. And the only spam that gets through are those with actual real-life return addresses. These are rare, and since there's a live address on the other end I have the luxury of sending the spammer a bitch-o-gram.
POPFile rocks. It is incredibly easy to use, and very, very accurate. I initially started using it for spam reduction purposes but now I find it's best use is actually sorting my mail... waaaaay better that pre-defined mail filters.
I strongly recommend people check it out if they want a very effective solution that is easy to use and configure.
Read Pynchon.
Hey, it's not my fault that you can't appreciate sarcasm.
You're not the same Timbo of "Timbo's goals" fame are you? If so, any predictions for the season? I can't believe we didn't try to get Mendieta if he was available for free.
"Accept that some days you are the pigeon, and some days you are the statue." - David Brent, Wernham Hogg
Try cleaning your training file (\training.dat) and retraining it, there's probably something wrong with it, and it can't "unlearn" whatever is screwed. It should be able to do a LOT better than that.
And yes, it uses the same filter.
that was supposed to be \training.dat, but /. ate it.
My current spam filter is SpamOracle. It's a simple procmail filter based on Bayes' formula. It's really efficient, I haven't had a spam mail in my inbox for a week. The only bad thing is that it's written in ocaml which might not be on everybody's machine. Mandrake users can install a contribs package and don't need ocaml at all.
...and when is FreshDeadMeat going to use fonts that are so boogeringly fsckin' BIG...? Must be composed and viewed under Linux or somethin'...