Slashdot Mirror
Critical Eye on SpamAssassin
ErrorBase writes "In this Infoworld article, Logan G. Harbaugh makes a great deal about an ancient (2.44) version of SpamAssassin comparing it with newer comercial variants.
Quote : You get what you pay for. [...] However, it took more than 10 times as long to install and configure SpamAssassin as it did any of the other products. "
Why did he not ask Kevin Railsback who had the whole thing working some while ago?)"
You never know most places would have not looked for an OO solution.
A psychopath can't tell the difference between right and wrong. A sociopath knows the difference - he just doesn't care.
not using Outlook? Seriusly most good anti spam filters are server side.
A psychopath can't tell the difference between right and wrong. A sociopath knows the difference - he just doesn't care.
Seems like this guy did not verbalize it but that was his problem. If you know what you are doing hacking a conf file from vi is easier than a gui for sure. However, his low performance and configuration woes would have probably been handled with a easy to use graphical interface.
Aren't there tools that do this?
ACK
This was just a setup to make commercial software look better or just a incompetent reviewer. Next.
"We compare a collection of recent operating systems: Windows XP Professional, Mac OS X Panther, Debian GNU/Linux 0.91".
Seriously, InfoWorld, SpamAssassin 2.44 was released in February, all the other vendors you compared were constantly updating their products to cope with the ever changing nature of spam.
John.
This is likely funded by un-named virus vendors who has integrated SapmAssassin into their appliaces. Away on a vacation, I came back to find our people unaware SpamAssassin was open source. The vendor quietly forgot to mention that.
In the end, any company is going to have to put people and tools together to get a spam solutution, or outsource it. But DIY needs people time.
Don't pay vendors for SpamAssassin, it runs quite nicely on left over PCs reloaded with Linux.
well, on the first page the author already makes it pretty obvious why SpamAssassin had to come out at the bottom of the list. He is comparing version 2.44, which was included in RH9 and is thus at least 8 months old, to the latest antispam software that is regularly updated. How on earth is that an unbiased comparison? In a world where spam patters change every week, if not every day, 8 months is a generation... he even says so in his article. I'd be interested to see the results of a similar test, but with SpamAssassin 2.60 and of course with bayesian filtering and some of the other optional features enabled...
Sounds to me like Infoworld has an advertising contract with (at least) one of these companies. At the very least he should have checked the site for an update before he started his "tests". For a while there, I got every one of those "IT industry" hype mags (always free). While there was some good information here and there, you had to wade through a lot of advertising pretending to be articles.
I love SpamAssassin and would not consider email hosting without it. It has made my email account useable again ! For the record, it seems to catch about 80-90% of my spam, and I have never seen a 'false positive' (I do check my 'spam' folder, but less and less)
The grass is only greener, if you don't take care of your own lawn.
It is not a fair comparison to compare the open source solutions against commercial variants, especially in the spam war. Yes, it is nice to have a small army working against your spam (like in the commercial products), but you have rendered your control to someone else. That is the beauty of open source. You are the captain of your own ship. Maybe that is the problem, if it sinks, you have to go down with it. With a commercial product there is someone else to lay the blame. Spamassassin is very easy to configure and tweak. I change settings as the flow of spam changes. We recieve a lot of e-mail from over seas and Spamassassin does a wonderful job of sorting out the unwanted mail.
Look at where the article is from!!
Infoworld.com Do you think there going to put their advertisers products down? I could tell after the first three paragraphs that the article was a sales brochure.
I take no responsibility for what I say. Even though I'm never wrong
Seriously:
- The Spamassassin installation documentation could be better written IMHO.
- Why doesn't RedHat's update service offer constand updates to the current version of SpamAssassin?
- Why doesn't it (as mentioned in another post) have the most important configuratoin setups included in their overall configuration GUI?
I really wish distributions would support SA better.First thing, the user has to be at least as smart as the tool they are wielding. No, actually just smart enough to follow directions and go beyond clicking on "help" to get help. Just another case of wannabe administrator arrogance: "If the tool doesn't configure itself or have cool looking icons, it must suck."
The bias apparent in this article and the crappy comparison chart aside this review doesn't even begin to touch base as a throughly researched opinion ion piece and ends up look like an advert for Brightmail.
However we do in the OS community face a UI problem. The missing rung on the ladder to mass acceptance is the absence of high quality UI that give users and indeed administrators of the point and drool variety a interface with the service they are seeking to use.
Before the Highly polished phpmyadmin I met serious resistance from admins for MySQL over msSQL based mostly on interface. The same goes for CUPS which has a web interface that I think has come of age if not achieve adult hood. The Webmin's are OK as long as you don't tinker to much or do anything slightly non-standard. I dislike Swat and am now so used to editing smb.conf I haven't even checked it;s working. I think that a lot of these services, apache, Spamassassin and X11 for example, could bare providing embedded configuration UI's if they aim to capture wider markets. Mandrakes X11 confugulator is very good.
I was going to mention the difficulty presented for admins with widely deployed Outlook when looking at these kind of solutions but then I though no only have sympathy where it is due. An I know that SpamAssassin could work seamlessly with Outlook but if users want a front end for white-listing then SpamAssassin isn't going to be your toy just yet.
Though we love the text based config file you may have to put a lot of working into configuration UI's if you want to enter the area as far as that reviewer and many sysadmins are concerned.
I knew nothing about filtering spam until I installed SpamAssassin 2.6 in a multi-user environment last week. Here are my responses:
I wouldn't recommend that my grandmother install SpamAssassin, but if you have any admin skills whatsoever, it's quite easy to use it to set up effective and useful filters. Furthermore, there are enough factual errors in the article that I'm tempted to dismiss it outright.
Of course, it's possible that it got a lot better between 2.44 and 2.6, but that begs the question, why did he install 2.44?
"The obvious mathematical breakthrough would be . . . an easy way to factor large prime numbers"
Bill Gates, 1995
Can you imagine someone writing "Oracle, Sybase and Postgres were compared. While the data and workloads were different, all products performed enough work to assess thier capabilities."
...and it ain't that hard.
A very large sample of mail would negate almost all of the differences caused by using a different set of mail, but I get the feeling that each of these servers ran for about a day and the results were gleaned from that.
I don't know anything about Brightmail. Spamassassin end user whitelists entries can be set up in a number of ways.
As aluded to in the summary, this is false with modern versions of Spamassassin, which uses Baysian filtering. (The author later says he couldn't get it working.)
Maybe I'm missing something or taking things that I consider basic for granted, but Bayesian filtering with SA is about as straightforward as it gets, except that instead of clicking a few buttons, you run one short command.
While it is true that one must be comfortable with a text editor to configure Spamassassin, thus perhaps putting it out of reach of point-and-click admins and technical journalists, I also wouldn't be prone to put my mail servers in the hands of either of those groups of people.
I think we've all known these types, and unfortunately they're more widespread than we'd like to think. Many simple solutions such as SA are ruled out because the admin doesn't have the skill to implement them. Note to any managers reading this: hire people with a solid background in the field, not those who list single-platform applications on their resume as "skills." Software changes, but a good administrator has the ability to adapt.
-- "Complacency is a far more dangerous attitude than outrage." -Naomi Littlebear
I can install Spamassassin and six other applications via CPAN in the time it takes to get the syntax right for one license key.
I also like the characterization of Spamassassin as "first generation" without any supporting evidence to the fact. First generation was adding spam senders to your e-mail client's blocklist. Bayesian filtering is well beyond first generation, but spammers have learned to defeat Bayesian filtering with poison data in non-eyeball space and text obfuscation. The next generation in spam detection is to detect the Bayesian evasion features - and guess what does that!? Spamassassin (2.60).
who are those slashdot people? they swept over like Mongol-Tartars.
Here's a nice example of a commercial guarantee. See if you can determine where it's from:
...
11. LIMITED WARRANTY FOR PRODUCT ACQUIRED IN THE US AND CANADA.
Microsoft warrants that the Product will perform substantially in accordance with the accompanying materials for a period of ninety days from the date of receipt.
YOUR EXCLUSIVE REMEDY. Microsoft's and its suppliers' entire liability and your exclusive remedy shall be, at Microsoft's option from time to time exercised subject to applicable law, (a) return of the price paid (if any) for the Product, or (b) repair or replacement of the uct, that does not meet this Limited Warranty and that is returned to Microsoft with a copy of your receipt.
Note that a) no updates or fixes are guaranteed, b) your only remedy is media replacement or a refund, and c) this choice of remedy is up to Microsoft.
I love it when people claim that you're taking a huge risk with open source software without guarantees. Microsoft says their software will work, but isn't saying that if their software doesn't work, they have to fix it.
It's better to vote for what you want and not get it than to vote for what you don't want and get it.
- E. Debs
TrollAssasin would be nice, imagine seeing posts subjects as *****TROLL***** heh
Seriously, wasn't that one of the ideas behind moderation?
"I installed the software on Red Hat Linux 9, with help from one of Proofpoint's systems engineers. She talked me through getting the Linux system configured properly, getting sendmail set up, and installing and configuring the Protection Server, which includes the MySQL database server for storing quarantined e-mail."
who needs a gui?
no wonder he gave spamassassin a low score. he couldnt have someone handhold him
I think the poster was creating an implicit comparison between various types of admins. Installation, configuration and maintenence of Spamassassin is simple for a skilled admin, while it may not be for an inexperienced one. It is a simple solution because well, it is, if you know what you're doing. If you don't, perhaps you shouldn't be trying to solve the problem.
There are easy comparisons to other fields. For instance, changing the brakes in a modern car is simple. It happens thousands of times every day, and there are entire franchise operations set up to do it. And yet, if I were to sit down with a random 2003 model car, it would be hard for me, perhaps beyond me (I dunno, I used to change my brakes on my 1984 Civic with no problem, but I suspect the braking systems are as overengineered as the rest of the car these days.).
See the distinction?
I forget what 8 was for.
> I don't understand why he's so critical of a free product.
Why is there this attitude that if your project is free, then it does not matter if it is garbage. Furthermore, you are not allowed to say it is garbage, because, after all, you don't look a gift horse in the mouth. Perhaps that is why Linux is still not on the desktop. There are plenty of people who spend days configuring theirs and then post "it works for me" comments, while the rest of us silently wonder why anyone would want to spend so much time on such garbage.
This is very true, of course. But has the guy considerered that this is 1:1 the case with commercial software too?
Even support providers for enterprise level software (i.e database vendors, which may charge hundreds of thousands of $, depending on the installation and support level) will never guarantee that they provide you with a solution.
Of course their sales reps have the flashier presentations though, which is a part of what you pay for.
ich bin der musikant
mit taschenrechner in der hand
kraftwerk
To moderators. When you mod something "informative", please check the facts first. Spamassasin in RH 9 is 2.44.
Save the bandwidth. Don't use sigs!
Yep, linux is ready for the desktop
Larry Seltzer did a similar job with a review of disposable email address services in
PC Magazine.
Spamgourmet (open source and free to use) was lined up against several commercial offerings, and was rated the lowest. It was clear from the review that he didn't spend much time learning about how spamgourmet works -- he wound up faulting it for perceived problems that were addressed by features that he ignored in the review.
Not to be cynical, but if I were a tech reviewer, I might be afraid of lawsuits resulting from my reviews -- open source projects have no revenue, and therefore can't prove up any damages in court. This might make me more likely to choose the open source alternative to get the shaft. Hopefully that's not what's going on here, but you've got to wonder...
who's moderating the meta-moderators?
If you have your mail on a POP server (ISP, hosting provider, etc.) try PrismEmail. It filters between your server and you so there is effectively no time or load on your computer, plus it works with virtually any mail client with nothing to install on the server or on the client.
I'm at 99.9% accuracy so far this month.
Hmm. Brightmail Anti-Spam - Enterprise Edition is $14,000 a year for up to 1000 users ($1500 for up to 50 users). Hiring a professional consultant to install Spamassassin (about an hour or two of work) would surely cost much less. And you wouldn't have to worry about the company going out of business or raising prices. So even if your administrator's time is worth more than $7,000 (or $750) an hour, there's an alternative solution, pay someone to install the damn thing.
> Messages classified: 1,440
> Classification errors: 19
> Accuracy: 98.68%
That's nice, but it's really important to break it down between false positives and negatives. I get over 200 spams a day (before filtering), and while it's quite tolerable for 2 or 3 of those to get through, missing that many legitimate messages a day is not.
This guy's article was a joke. Not only did he use an ancient version (in the spam world) of SpamAssassin but he either flat out lied in his article or was too lazy to seek out the truth. Hard to configure? Can't find docs? Doesn't support A B C D or E? If this guy had spent 5 minutes of his precious time doing to research on SA he wouldn't have made these flagrant lies. I don't get these people. I really don't. I CCd the Editor-in-Chief at InfoWorld, Mr. Steve Fox, as well.
Mr. Harbaugh,
This letter is in response to your InfoWorld article titled "Commercial solutions win, spam loses." In that article you portray all commercial spam solutions as winners and you portray the only open-source spam solution you reviewed as a dismal failure. I must say that as a professional in the anti-spam field I'm am truly disappointed by your incomplete and inaccurate assessment.
You start the article off quite well. Your introduction regarding two of the possible types of spam filtering is in terms that the average reader can understand. The introduction is also technically accurate, although it doesn't mention the other ways to filter spam.
You quickly take an opportunity to kick dirt on SpamAssassin by claiming it filters a fraction of the amount of spam all the commercial solutions filter. You hint at something during that statement when you said that SpamAssassin's "age showed in my tests," yet you fail to actually make it apparent to the user what the real truth is. I must ask, why did you choose to compare such an ancient version of SpamAssassin to the current versions of the four commercial products? Version 2.44 is over 9 months old. Spam filtering techniques are constantly evolving to filter a continually changing target. Comparing a 9.5 month old copy of SpamAssassin to the current version of BrightMail is like comparing a 1990 Chevy Silverado to a brand-new 2004 model. As an author and professional in the IT industry writing a column for InfoWorld, one of your goals is accuracy and fairness in reporting, is it not?
You make numerous false statements regarding SpamAssassin in your article:
1) "All the products except Brightmail and SpamAssassin allow end-users to add senders to the domain whitelist themselves... SpamAssassin allows only the administrator to add to the whitelist, with no direct access for users."
This is simply not true. SpamAssassin allows its users to add whitelist or blacklist entries to the personal preferences. It also allows its users to control the scoring for each individual ruleset with SpamAssassin's arsenal. Even the ancient version of SpamAssassin you chose to use had that simple feature. SpamAssassin also has the ability to automatically whitelist senders.
2) "Delegation of specific administrative functions is possible with all the products except SpamAssassin..."
This too is not true. As I said in response to number 1, SpamAssassin allows its users to control the scoring for each individual ruleset. This gives them the ability to disable certain rules, lessen the scores of others, and increase the scores of rules they wish had more weight. For example a user could disable the MAPS RBL DNS blacklist checks, whitelist joe@mydomain.tld, blacklist annoying-spammer@spamdomain.biz, and increase the score of the rule ALL_CAP_PORN to 2. The users can also create their own rulesets. SpamAssassin gives its users a high level of control over their spam filtering.
3) "Finally, in addition to stopping spam, all four commercial products provide content-filtering features, allowing the administrator to block incoming or outgoing e-mail that contains proprietary data, audio or video files, executables, sexually explicit words, or racial slurs. They also provide protection against DoS attacks and directory harvesting attacks."
This one baffled me at first. I'm honestly not sure why you want to compare features that have nothing to do with filtering spam. Filtering racial slurs from an email is