Slashdot Mirror
DSPAM v2.10 Released
Nuclear Elephant writes "DSPAM v2.10 is finally available, after four months of development. This is the first stable release to include Bayesian Noise Reduction which was recently mentioned on Slashdot and in Wired News as an algorithm providing accuracy levels as high as 10x that of a human. Some other new features include Neural Networking - which finds nodes in a network that are contextually similar to form a decision matrix, Global Filtering - which provides SpamAssassin-like out-of-the-box type filtering for new users until they build up their own wordlist, Automatic Whitelisting - which automatically learns who your trusted senders are, and many other optimizations and enhancements. Head on over and download the latest tar ball."
I tried several incarnations of dspam over a period of about 6 months. It was a pain in the butt to install, required a massive amount of training, and required you run a web server in order to have the point and click training capability.
I eventually gave up and tried the CRM114 Discriminator:
http://crm114.sourceforge.net/
It was MUCH easier to install, MUCH easier to maintain, and has the same or better level of accuracy. I used to get 100+ spam messages a day and now I'll get maybe 1 or 2 a week that sneak through (after only a few weeks of training on errors only).
FYI, modern MRI scanners use bayesian noise reduction during image processing. I used to work in a MRI research laboratory, and our director had pioneered the application of Bayesian noise-filtering algorithms in post-processing of image data.
Oddly enough, our director of research was notoriously difficult person to schedule a meeting with. Makes me wonder about 'unsupervised learning'...
DSPAM is one of these statistical filters (like spamprobe and CRM114) that can perform virtually perfect filtering of spam/non-spam you receive.
Now that you are free of spam yourself, may I suggest that you take it one step further and share your data with the anti-spam community; the WPBL project lets many users report the IPs sending them spam and non-spam in realtime using a couple simple scripts installed in procmail.
Our central database then publishes a real-time list of spam sources (the IP blocklist). Unlike spamcop, WPBL is entirely based upon automatic decisions made by statistical filters, 24/7. The resulting blocklist is already used by many ISPs; and you can also use it to block spamming IPs at your own server.
No matter what technology it uses, neural nets, b-trees, recursion, tinkertoy logic, smell-emitting diode, leaky junction zener transistor, steam-powered aeolipiles, it only automagically presses delete, which is a pretty lame way of fighting spam.
It's a lame way of fighting spam, because, we STILL have to pay for the fucking spam bandwitdh; we STILL have to pay for the goddammed disk space used by the spam; we STILL have to pay for the bloody time lost transmitting the spam; we STILL have to pay for the extra ISP infrastructure to carry those spams.
Naaah. Spammers should be eradicated from the Internet, and the best way to do so is to completely BLOCK networks who host spammers (no matter what service), in order to force the collateral damage to whine to the ISP or simply vote with their feet.
The authentication is useless even if implemented - you want to receive email from strangers, that's what all businesses are doing. If you are not one of them and only converse with your buddies, make a whitelist and be done - no spammer will guess your friends' emails.
Permissions to send email are also troublesome. If they are automated, then spam robots will be written to ask for permission first. If they are not automated... but how would you know if some random "John X. Frisby" <jfrisby@big.provider.net> is really who he is, and the matter he wants to discuss with you is not a bug in your Loafizer 0.99 script for your bread making machine, but a placebo enlargement pill. Additionally, permissions delay the mail exchange, which is bad for business.
There are ways to block anyone you don't want, and all other senders are legit (until they spam you, that is.)
So the problem is quite different, as you can see. There is a free channel of marketing, and spammers will be using it until it remains a) free and b) channel. Remove any one of those two, and they will close up the shop.
I don't get SPAM. I don't have SPAM filters. How is this possible? Simple. I create a different e-mail address for any new untrusted entity that I have to provide one for. In the beginning I took advantage of being able to alias all e-mail for non-existent mailboxes (basically, *) at my domain to my primary account. It seemed to me an obvious and simple approach. Whenever I needed to provide an e-mail address, I just made one up, and it was forwarded to my regular Inbox. In my opinion, at that time my ISP was more "sophisticated" than most. Since then I have moved to hosting all of my domains on my own co-located server which runs Exchange 2000, thus complicating things. Now I have to actually add any new aliases that I want to use into my user account. I know of at least one product out there that can handle non-existent addresses and forward them to a specific account, but it is rather expensive for a feature that should have been built-in from the beginning (althought I'm not aware if the new Exchange can do this out of the box). Not to mention that someone with the proper knowledge and skills could make a similar add-on in relatively short order, but who ever has the time? The point is that you have to consider when and where you give your e-mail address out, and the possible consequences therein. It's not altogether different from giving out your phone number (especially if you are unlisted) or even your SSN.