Slashdot Mirror
Spam Filtering For Small/Medium Business?
or_is_it writes "The company I work for has been growing dramatically and I've been charged with the task of being the gatekeeper for our GFI Spam filters. This involves manually inspecting the subject line/to/from for all caught messages in each filter rule folder. For a company of about 50 people, in one day the number of spam messages can exceed 2,000. Neglect it for a day and you end up with quite a task on your hands. I've made the rules lax enough so important messages can go through, along with a few stray spams, for which I get bitched at. Tighten the rules up and then maybe an important time-sensitive email never gets to its intended recipient, and I get bitched at. Manually reading through all those subject lines is supposed to prevent that, but I'm only human and genuine messages can easily get overlooked. How do larger organizations deal with the spam issue? I can't imagine having one centralized person manually inspecting everyone's junk-mail header is the optimal solution. Purchasing a different commercial mail filter product is a possibility, but I'd like to hear some anecdotal evidence before jumping ship."
I've had to send emails to recipients within the Australian Defence Force (specifically, the Army), and every email sent from a civilian must include a keyword within the subject line. The keyword is to do with whether or not the information is classified or unclassified. Sure, getting all the clients to send all their emails with [companyname] in the subject line is a little annoying, and may not be possible depending on your circumstances, but the chances of spam having that keyword within it is virtually impossible.
Set up an automated filter whereby anything that doesn't have the keyword in the subject gets dumped into a spam box to be sorted later. If the senders do the right thing, it assures their emails will be directed to the correct person.
This is just one example of active spam filtering as opposed to the passive spam filtering used in IT today.
You moved your mouse. Please restart Windows for changes to take effect.
How do larger organizations deal with the spam issue?
I used to work for a mining company you've heard of. Our department had responsibility for managing the email vendor, who used Spamshark to filter spam coming into the organisation. From my limited knowledge of the setup, Spamshark does basic blacklisting etc. but also does selective blacklisting on specific IPs when an email is flagged by a user. So Alice flags a message as spam, Spamshark figures out the message id, grabs the IP address it came from (it knows because it previously handled the email), and then blacklists that IP for a certain amount of time. Now this internal blacklist is then shared to all the other customers who use Spamshark, so they are now protected too; resulting in a 5 nines hit rate on spam.
Like I said we just handled vendor relations, and the above description might not be totally accurate, but this is what I gathered when we dealt with them. I also remember getting about 10 complaints of spam a month for an organisation with 10's of thousands of email addresses - so it was very effective.
"And then I visited Wikipedia
Ya, i rolled a baracuda out in a similar environment back in 04, and the users couldn't stop singing the praises compared to the filtering our mx offered + my manual filtering. I strongly recommend baracuda for this size roll-out.
How Jaded Are You?
I can't imagine having one centralized person manually inspecting everyone's junk-mail header is the optimal solution
Actually, that strikes me as a good solution; it's certainly better than having other employees dealing with spam as part of their daily routine and losing 30 minutes/day for everybody in the company. And by centralizing it, you have the ability to pick the tools to make your work more efficient, as opposed to having 50 employees each fiddle with their own spam filters.
This is just a simple guide compiled from my experience:
:)
1. Do what you can on the server. I like to use SpamAssassin to add spam scores to beginning of subject lines, so they sort by score in my inbox (I use "/*_SCORE(0)_*/"). I also automatically delete anything over a score of 11, since the highest I've ever seen a legitimate email score has been "10.something". Realistically, anything above an 8 is the sender's fault and they need to do something about it and anything above an 11 you can safely blame the sender (you won't be the only spam filter deleting their emails).
2. Provide the tools on the client. ThunderBird's "spam marker" is a must, and because it learns from what you mark, you aren't just marking them in vain. Also, to deal with spam in real-time, instead of using the junk folder, I like using the "delete junk!" button from the "Buttons!" add-on. Incoming junk gets marked and marked as read, and after marking the spam the filter missed, I hit "delete junk". Very easy and quick. Pre-configure Thunderbird for everyone.
3. Educate and support. If you have 1 and 2 in place, then make sure everyone knows what you are doing and why you chose to do it. Write a short manual or something. Educate them about their tools. They also need to know NOT to publish their addresses.
The idea is to make spam highly visible, and to make it *quick and easy* to deal with. Knowing you've facilitated these two goals should be enough to impress your employer and earn the respect you deserve from everyone you serve
I spent a few days migrating 100,000 emails from Windows Mail, because it was horrible. Thunderbird is a godsend and the add-ons make all the difference. If there is something you dislike or want, chances are someone made an add-on for it.
btw 2000 messages is *not* a lot of spam. It will get far worse with time.
I'm not really "in the know" of what's good or bad when it comes to spam filtering packages, but in the years I've been using gmail, I'd estimate maybe less than 20 emails that have hit my inbox have been spam. It only happens to me once every couple of months and I get around 100 pieces of spam a day, so I'd say that's pretty good.
As for the "false positives", only the most dubious of mailing lists seems to get caught (I still regularly check my spam just in case) and when I report them as "not spam", they never get mistaken for spam again, so I can't really complain either.
I'm not disagreeing with you, I'm simply just curious as to what makes it bad? Have I just been fortunate enough to not have any major problems or is there something that it should (or shouldn't) do when it comes to corporate use?
+1 IDisagreeSoHeMustBeATrollOrAnAstroturferOrAShill
Businesses shouldn't be using those for internal communications anyway. Set up a jabber or irc server internally for that.
I'm listed as the technical support contact for my employer's listings on eBay, and our PayPal account links to me as well. No spam filter on God's green earth is going to cull the spam from the ham for me.
Gah they are so expensive. And to keep them up to date is ridiculously expensive. I prefer free with ASSP.
Additionally I have a serious problem with the backscatter they cause. They should reject mail at SMTP time and not bounce them.
But Barracuda support is very very good. Very responsive and timely and overall a good people orgaization which can make the difference for wanting to deal with them.
Chuck
I was faced with exactly this problem myself around October/November last year.
You've basically got three options:
1. Go for a completely outsourced service.
Pros: It's someone else's problem to look after.
Cons: A company of 50 staff will never be terribly important to such a service provider. Unless they provide an extremely good control panel and logs, sooner or later someone's going to ask where an email is and your answer is going to be "er... let me get back to you on that.... er... I don't know".
2. Go for an appliance - either in the form of a prebuilt lump of tin like the Barracuda system mentioned elsewhere or in the form of a precooked Linux installation which is literally just a matter of "insert CD, boot, tell it what it's IP address is and what domain it's providing email for".
Pros: Dead easy to set up. Most also provide a nice web-based UI.
Cons: The decent ones are almost universally commercial and you have to pay licensing fees on a per-active-email-address basis, which can get very expensive - particularly when the vendor won't tell you how their system decides how many email addresses are regularly active and the first you know that you're exceeding the license is when suddenly all the spam filtering is disabled.
If you look closely, expect to find that many of them are architected around a number of single points of failure. And in the real world, nobody is likely to check a web-based UI on the offchance that they find an email misclassified as spam sat there.
3. Roll your own. If you take this route, I can strongly recommend rolling it around an existing framework rather than following a bunch of complicated instructions to configure Postfix that you have to re-learn every time anything needs tweaking. This is the route I took, and I based it around MailScanner. MailScanner provides a framework for plugging in spam and virus filters and allows you to divide spam according to its score. Delete high scoring spam, let low scoring spam through with a note in the subject line that it's suspected spam and let non-spam straight through.
Pros: You get to keep a close eye on all the configuration, can keep close track of the logs and respond quickly to any issues. Your users can easily set up filters for spam (for that matter, so can you) and their "potential-spam" where misclassified mail may wind up is in their email client rather than a separate web-based system.
Cons: You need to become intimately familiar with every aspect of your email system in order to manage it effectively. I would argue that any self-respecting sysadmin should be intimately familiar with his email system anyway, but YMMV.
Consider as well that the Barracuda appliances consist of (a) an open-source operating system (b) an open-source MTA (c) an open-source web server (d) an open-source spam scanner (e) an open-source virus scanner (f) other pieces of open-source software and (g) use community-mintained DNSBLs and RHSBLs. This is all held together with proprietary (closed-source) code, mostly for the purpose of providing a poorly-designed GUI interface. Any competent email system administrator should be able to create their own near-equivalent in an afternoon; it's not difficult. Such homebrewed creations have repeatedly been shown to vastly outperform Barracudas on multiple metrics, including cost, scalability, customization, security, and perhaps most importantly -- adaptability to new spammer techniques. (Barracuda is years behind the times and falling further back.)
It's very tempting to "just buy an appliance" and consider the problem solved, but it doesn't work. There's no substitute for expertise -- and given that much of that expertise is available for free, for the asking, on lists such as spam-l and spamtools and so on, it's difficult to understand why anyone would choose not to avail themselves of it.
It's really not that good. GMail is a viable option now that it has IMAP support. My small business uses it for several reasons:
1) It is a hell of a lot easier to maintain for me (sysadmin)
2) It comes with a webmail interface.
3) I don't have to have redundant mail servers
4) Even our marketing guy can set it up. It's that easy
5) I get a lot of spam on my personal account, it filters it like a charm. I never get false positives and only once or twice will a spam message get through to my inbox.
6) It is free (as in beer) for businesses with fewer than 50 employees
That is why GMail is a viable alternative for small business.
As you may know, it used to be that Postini was considered, by those of us in the anti-spam industry, something of a black hole, and not a service we would recommend.
However, having been in touch with their executive team in recent years, I had inside knowledge as to how that was changing - how they *wanted* that to change.
Recently, we decided to take our own spam filtering outside, to let someone else's servers do the heavy lifting. We tried several solutions, and finally, almost in desparation, I gave the 'ok' for us to try Postini (which of course is now owned by Google, but the exec team is still in place).
Let me tell you that we were *extremely* pleasantly surprised - the service really has been *very* good, it was relatively easy to set up (you do need to be familiar with how to set up your MX records, etc., but if you are already adminning a server, you should already be fairly comfortable with that).
The price is good, and the end user UI is excellent in that it's pretty easy for an end user to understand how to scan their "spam folder", how to get something delivered out of the spam folder, how to whitelist a sender, etc..
Honestly, it's one of the easiest-to-use of the offsite systems out there - and one bonus is that it gets the user support *off* internal admins.
And, the false positive rate is low, as is the false negative rate - which really is the bottom line test for spam filtering services.
We have a formal review for our corporate blog (http://www.TheInternetPatrol.com/) in the works, but in the meantime consider this an endorsement of Postini from the Institute for Spam and Internet Public Policy (http://www.isipp.com/)
Anne
Anne P. Mitchell, Esq
CEO/President
Institute for Spam and Internet Public Policy
Professor of Law, Lincoln Law School of SJ
Author, "The Email Deliverability Handbook"
2)
3)
Given the dictionary definition for "junk", this is not an unreasonable mistake.
- RG>
Hey pal, this isn't a pleasantforest, so don't waste my time with pleasantries!
Pardon me, but I just don't see the "size". I personally (and professionally) receive well over 3'000 spam e-mails each and every day. I take about three to five minutes to run through them. For 6'000 in two days, I take four to seven minutes.
." which get selected and deleted in a block of one hundred at a time.
I do it without a spam filter of any kind. I have only two technique.
First, simple rule-based filters throw clients and friends into their own folders by from: line alone. That covers everyone I know in advance.
The second set of rules simply looks for my full name, my company name, my e-mail signature, my telephone number, or my mailing address. These into the "it's damn likely a legitimate e-mail" folder. This folder gets about 2 spam e-mails per week.
The remaining I simply run through, in outlook express of all clients. Sorting wins the day. The greatest trick? Sort by the to: field. It doesn't take long to see that 75 messages went to moocow@mydomain.com, 75sevens@mydomain.com, or some other horribly malformed address to that doesn't exist. Sorting by subject does similar things -- like give you "70% off . .
Your spam has very simple patterns to look for. Sort by them, click the first, shift-click the last, and hit delete.
Last year, I was contracted by Viagra's H.R. department to do some quick work, I made it through unscathed.
I've seen Postini-filtered mailboxes. Don't bother.
Only solution that I know works is my own: Postfix with amavisd-new, spamassassin, clamav, postgrey, along with FuzzyOCR on smaller installs, though setting that up on a separate system to filter through might cover a large organization. Don't forget to include things like Spamhaus' Zen list, any of the *.countries.dk.net blocklists to filter out any geographical areas from which you don't expect legitimate mail, and also helo filtering--if the connecting mail server can't say helo/ehlo with something that resolves in DNS, it can just bugger right off.
Tell your boss that expecting not to lose email with spam filters in place is unreasonable, and that tasking one human to eyeball all the rejects is a serious misapplication of time and money.
Best of all, you should educate your boss to realize that email is not a reliable messaging system. There are far too many points of failure that could cause a message to be lost, most of them being outside of your own or your company's control. There exist many better ways to send time-sensitive material, like fax, overnight mail, and telephone calls. If a severe amount of money is to be lost because an email didn't make it on time or made it not at all, then the message should have been sent over a more reliable medium in addition to being emailed.
Only the severely clueless would rely on a system like the one you have set up. You have to allow for a certain failure rate in any system. That's a basic principle of quality control methods that have been in use for decades.
The business I work would qualify as a middle-sized corporation.
We run into the EXACT same issue you're running into.
The dilemma is if we don't tighten the spam filter enough, we'll get complaints from employees (who are not shy about sending EVERY LAST PIECE OF SPAM THEY GET to us.)
However, if they tighten the filter too much, then important emails that may seem spam-like begin to get blocked, and we get just as much heat for that.
The answer - do your best to block what spam you can, and if you get complaints about some spam slipping through, tell them to delete it. We'll often add that we're working with the spam filter vendor to try and resolve the issue, but it's not that easily resolved.
And no - we don't go through each message looking for spam - it's not practical due to the number of employees we have. We DO give them the power block spam from specific addresses on their own, though. The benefit of this is the email is sent to a junk mail folder they can still access, which is useful should something legitimate end up there.)
searching on google did find this Google Apps Administrator Help page. So it looks like because the
Google has one of the best spam blockers in the business, and it's integrated into Google Apps. Spam is purged every 30 days. We have built in virus checking, and we enforce checking of documents before allowing a user to download any message. Most computer viruses are contained in executable files, so standard virus detectors scan messages for executable files that appear to be viruses. Google blocks viruses in the most direct possible way: by not allowing users to receive executable files (such as files ending in
There's no way that I'll ever configure my server for any anti-spam technology based on the destination server requiring more of the source server after successful receipt.
If I mail a letter to you, and you don't like the return address on the back of the envelope, you can do with it whatever you please, but it's not my responsibility to ensure that you'll open your mail. It is my responsibilty to deliver your mail. If you don't like the colour of the envelope, that's your problem.
I have a lot of clients who routinely call me saying that one of their messages bounced from some server that says their message won't be delivered for one reason or another. The server received it, and then chose to request that I reconfigure mine. That's just not going to happen. I'm not going to reconfigure my server because another server admin wants me to make his life easier. My server is configured for my reliability, not his ease.
As for my being a spam processing machine, it's actually a lot easier when you get a lot more spam. If I received only 1'000 per day, it would be difficult. But by the time you cross 2'000 spam messages per day, it becomes a lot easier. And by 4'000, it's just funny.
For example, you may find it hard to tell if an e-mail is real by the subject. But if you've received the same subject three times, at three different addresses, it's spam. So when you sort 1'000 messages, how many are duplicate subjects? When I sort four thousand, there are loads of duplicates.
So I've actually got a bunch of extra addresses that I use loosely enough to be spammed thoroughly. It adds to the bulk, and makes sorting easier.