Slashdot Mirror
DSPAM v3.0 RC1 Spam Filter Released
Nuclear Elephant writes "DSPAM v3.0 RC1 is now available for download, with a stable release scheduled for June 13. DSPAM has appeared on Slashdot and in Wired News in the past for its high levels of accurate spam filtering. v3.0 is the product of three solid months of work. Some of the highlights include a very sleek redesigned interface, PostgreSQL support, many mathematical enhancements, and support for many of Gary Robinson's algorithms (such as Chi-Square, Geometric Mean Test, and Robinson's technique for combining P-Values)."
I am using this filter and after some training it is very effective. Especially useful is the inoculation feature, which you can use to register a spam only address to spam sending sites so that it trains faster.
My heart is pure, but make no mistake, it's pure evil
I'm all for throwing technology at the problem, but I hope people still realise that having a complex (and effective) spam filter does not take away the millions of megabits of traffic wasted on UCE when it's in transit.
But will it find out who sent the SPAM and hurl them into the Sun? Until I get this feature, I don't think it'll be perfect :)
DSPAM has a strong focus on providing better data to already existing algorithms (Bayesian, Chi-Square, etcetera) Combination algorithms work inherently well, but depend on the quality of data. Some of the approaches deployed in DSPAM towards this goal include Chained Tokens, Inoculation Groups, Classification Groups, advanced de-obfuscation techniques, and a new noise reduction algorithm called Bayesian Noise Reduction. The goal is to incorporate processing algorithms that can withstand the long haul of ever increasing message complexity. So far we're doing a great job.
The idea of combining more than one anti-spam heuristic is not new. But one thing that cant be denied is that all methods are just complementar to Bayesian analysis, that can reach up to 95% precision by itself. Chi-Square, itself, can reach up to 85% precision
Look! We came out with this great filter so nobody else gets spam! This solves the problem of spam once and for all! Even though spam is still clogging our networks and wasting bandwidth, this filter will solve all of our problems.
With all the time spent on making spam filters, why don't we spend that time working out a new protocol for email transfers, one that would not be able to spoofed, or spend that time installing server side programs that put a small time delay between messages as well as bandwidth restrictions for all outgoing mail?
unless mail sending protocol is redesigned(for example,in a way you have to have your fingerprints recognized when you type it) we will have to face the fact SPAM will be in our daily news. Soon slashdot will put an article where the best 3 spam filters are compared, like a normal review.
"The quality of life is inversely proportional to the number of keys on your keyring."
I tried to setup spamassasin a couple of months back and I found it to be too much of a hassle to setup. Could someone who used both spamassasin and dspam comment on easy or difficult it is to setup dspam?
Do not read this
Warning, it seems to be designed more for high volume use than individual sites. I've fed dspam almost 3000 spams and it is still only catching 80%, does seem to be getting better though.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
When you run your own mail server, or administrate a mail server for a large number of people, server-side anti-spam filters and countermeasures start making a lot more sense. Do the math on a company with 100 employees (at $25/hr) who check mail twice a day and spend 5 minutes each time hassling with anti-spam measures in client-side mail apps. In this scenario, a seamless anti-spam solution is worth conservatively $400 per day, or $100k/year not counting bandwidth savings. There are definitely cases when client-side filtering makes sense, but if you can handle it at the server, email-based business methods scale better.
http://tinyurl.com/4ny52
would be to publicly humiliate/boycott the companies that use the spammers services. Like drug dealers, as long as there is a market, the spammers will be around. Remove the demand, and the suppliers will eventually move onto selling something else.
If you can't kill the leeches because the water is too murky, then boil off the pond!
CodeTrap (www.codetrap.net)
As far as I know, the main difference is DSPAM does not use weighted filter rules at all like SpamAssassin's hybrid approach does - DSPAM is designed to purely rely on analysis of spam's properties (Bayesian, etc).
The other cool thing about DPAM is that it is designed to let users add/modify their own spam database - every email DPAM processes is tagged with an identifier, and is logged in a server-side database. If a delivered email is in fact spam but wasn't tagged as such, the user can then forward the email to the designated spam-sorting address, and DSPAM will automatically update that user's spam corpus (eg, because it's tagged with an identifier, you don't have to worry about the user forwarding the full headers, as the server already has that info on file).
AFAIK you can't do that with SpamAssassin.
You can configure DSPAM to not use the ID, but this requires users to "bounce" the incorrect e-mails instead of forwarding them (as forwarding strips the headers).
Is the ID really that inconvenient?
I've been using DSPAM for about three months. A few criticisms:
First, by default DSPAM wants to run as the "root" user and usurp delivery of e-mails. (With Exim, they actually want it to recursively reinvoke the mail server for actual delivery!) It took quite a bit of configuring to get it to work like SpamAssassin from procmail.
This software is somewhat buggy, so running DSPAM as root would also introduce security concerns. For example, I'm using 2.10.6 because the 3.0.0 compiled and installed with no problems, but failed to classify anything. (Even with several hours of gdb tracing I was unable to determine why). Another bug is that if I run the "--falsepositive" on an e-mail that's lacking the "!DSPAM" signatures, the message should be ignored, but apparently this is not the case because the statistics counters are incremented.
From the FAQ:
"Q. Does DSPAM support whitelists?
A. DSPAM doesn't have a whitelist manager, rather whitelisting is an automatic function of DSPAM's Bayesian filtering mechanism."
This is crazy -- the whole point of whitelists is for when the Bayesian filtering fails! And DSPAM does fail. Twice now I've had to reset my database because the classifications were wrong and training wasn't helping. All I can say is I'm glad I've got procmail to rescue the important e-mails.
I think one source of my problems was that the default training mode ("train on everything") causes incorrect learning when you fail to report a false positive. This was a big problem for me, since I get around 700-800 spams/day. While false negatives are easily caught, the false positives go unnoticed unless I happen to wonder why someone never responded, and invest some time to search my spam folders. (I'm still trying to figure out exactly how to deal with this problem. E.g. maybe I could have it challenge the sender with Turing Test or something.)
I will say that DSPAM's basic technology is quite good. It's just that the software still has a "prototype" feel, and I'd caution you to do some experiments before unleashing it on your users. (For example, there's no manpage, and there isn't even a command-line option to print out the current version number!)
-Gonz
Your post advocates a
( ) technical ( ) legislative ( ) market-based (*) vigilante ( ) lack of an
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
(*) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
(*) The police will not put up with it
(*) Requires too much cooperation from spammers
(*) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
(*) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
(*) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(*) Asshats
(*) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(*) Eternal arms race involved in all filtering approaches
(*) Extreme profitability of spam
(*) Joe jobs and/or identity theft
( ) Technically illiterate politicians
(*) Extreme stupidity on the part of people who do business with spammers
(*) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(*) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) No-lists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
(*) Countermeasures should not involve sabotage of public networks
(*) Countermeasures must work if phased in gradually
( ) Sending email should be free
(*) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(*) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!