Slashdot Mirror
Two Spam Filters 10 Times As Accurate As Humans
Nuclear Elephant writes "The authors of two spam filters, CRM114 and DSPAM, announced recently
that their filters have achieved accuracy rates ten times better than a human is capable of. Based on a study by Bill Yerazunis of CRM114, the average human is only 99.84% accurate. Both filters are reporting to have reached accuracy levels between 99.983% and 99.984% (1 misclassification in 6250 messages) using completely different approaches (CRM114 touts Markovan, while DSPAM implements a Dolby-type noise reduction algorithm called Dobly). If you're looking for a way to rid spam from your inbox, roll on over to one of these authors' websites."
I haven't been 100% accurate.
I received an email from my sister-in-law from her work, and the address looked suspicious (one of those weird-looking "letter and number" jumbles.
I deleted it. It happens.
Human=99.84
.16% and the machine only missues .016% hence the machine is 10 times better.
New proggie=99.984
So the human misses
No, you are just bad at math .9984 = .0016 .99984 = .00016
1 -
1 -
A factor of 10 in reduced error rates
160 errors per 10 thousand vs 16.
[*Bing* -- mail from VP of sales pops into my inbox. Subject: "Making money fast!"]
[*Bam* -- I hit delete, thinking "Stupid Spam!"]
Ahh, shit! Lookie, a human screwed up.
The filter would have actually examined the message and probably decided that it was legitimate.
A CRM114 plugin for SA is available, thanks to Devin Nate:
d =2 301
http://bugzilla.spamassassin.org/show_bug.cgi?i
If you don't control the mail server to create aliases for yourself, you can also employ RFC-compiliant suffixes to your e-mail address. For example:
foobar+dellorders@mydomain.com.
Unfortunately, even though it's RFC-compliant, I've found probably half the sites I have to give my email address to won't grok the username+filtername@mydomain.com syntax. It's convenient when it works, but it doesn't work enough to rely on. No, throw-away spam-bait email addresses that you use for 6 months at a time for all online ordering and the like, then eventually trash when they get too spam-ridden are the best solution I know of.
ObKubrick: In 2001: A Space Odyssey, one of the pods was marked with the designation CRM-114. And in Clockwork Orange, Alex is injected with serum 114. I suppose CRM-114 is to Kubrick as THX1138 is to Lucas.
Dobly, on the other hand, is from This is Spinal Tap , a mispronounciation of "Dolby" by David St. Hubbins's girlfriend:
Not to mention that it probably avoids trademark infringement (though I wouldn't put it past Dolby Labs or Thomas Dolby to raise a stink).
Maj. Kong
Shoot, a fella' could have a pretty good weekend in Vegas with all that stuff.
Good question! We're working on this problem, among other things, at the PSAM project. We have a project to produce high-quality benchmark corpora for spam filter testing. Watch that space for ongoing work, or e-mail us an offer to pitch in and help---we could use it!
When these factors are considered, I think it's quite possible to write software that in the long run has a higher success rate than a human who has better things to do than filter his mail all day.
Umm, Fetchmail + procmail on your local machine?
Not sure exactly why you need a pop3 proxy involved, just use Fetchmail to deliver locally, run things through procmail.
Set your local mailserver (sendmail/qmail/postfix/exim/whatever) to use your ISP's SMTP server as a smarthost, and it'll send everything it doesn't recognize as local off to them to handle.
I write spam filters for a living, and I promise you that they can eliminate many of the spams just by looking at the subject too.
Of course, so can I. Now, since I write the filter based on my human judgement of what constitutes spam, which is more accurate?
username+filtername@domain.com should go to username@domain.com as per the RFC (the +filtername is carried but not used by servers, or at least it shouldn't be). Some email clients will allow you to use this for such things as folder sorting (i.e. username+foldername goes into foldername automatically). If this worked consistently, it would be good for people who don't have the ability to make more usernames.
AFAIK, username-filtername will still just go to username-filtername, i.e. you have to configure your mail server to handle username-filtername separately from username. This works great when you can specify as many usernames as you want (i.e. if you manage your own server or have a catch-all on your domain).
Maybe you are talking about something different than the original poster?
One reason why the - would work when the + does not is that the - can appear multiple times, so it just another valid character (like a letter, number, or underscore). The + can only appear once, so many servers can ignore it, drop it, or puke on it.
Interestingly enough, while the (optional) challenge/response system is what gets the press, the main purpose of TMDA is to create aliases like username-filter (and then filter based on them). Thus the name: *Tagged* Message Delivery Agent. The -filter is the tag of Tagged.