Slashdot Mirror
Ask Slashdot: Why Are Major Companies Exiting the Spam Filtering Business? (slashdot.org)
broswell writes: For years we used Postini for spam filtering. Google bought Postini in 2007, operated it for 5 years and then began shutting it down. Then we moved to MX Logic. McAfee bought MX Logic, and McAfee was purchased by Intel. Now Intel is shutting down the service. Neither company chose to raise prices, or spin off the division. Anyone want to speculate on the reasons?
Maybe it's not profitable?
Because Gmail does a better job anyway, and outlook is for old people.
Guessing Google integrated the parts of Postini they wanted into Gmail's spam filtering and has no desire to help improve other email providers spam filtering.
McAfee recommends migrating from MXLogic to Proofpoint -- an exclusive partner. In this pdf they call it a more feature rich product. And I'd guess they are getting some sort of financial incentive to recommend Proofpoint. http://www.mcafee.com/resources/faqs/faq-eol-email-security.pdf
Google had no need for Postini. Google's own spam filtering in Gmail is pretty good. Probably as best as spam filtering could be, under the circumstances. So that's one elephant in the room.
The other elephant in the room is Microsoft, with Hotmail, or Office 365, or whatever it's called these days. I don't have any firsthand exposure to that service, but from what I hear its built-in spam filtering is also fairly good.
Big email providers like that have no need to use an external, third party spam filtering service, since they have the technology, and the scale, to implement it in house. Organizations that outsource their email service to these elephants get spam filtering as part of their service and, again, have little need for a third party service.
About the only likely market for third party spam filtering services would be small to mid-range ISPs or organizations that want to run their E-mail in house. They wouldn't typically have the in-house technology to implement spam filtering, and would rely on a third party. Seems like a fairly small market to me, and with E-mail generally on a slow, steady decline there doesn't seem to be a lot of market opportunities here, for third party spam filtering services.
Far too many people in this world dedicate themselves to profit rather than what would serve their fellow human beings the best. Never will understand why though. As a species, human kind has depended on the help of others to advance.
Time is what keeps everything from happening all at once.
Because pretty much everyone uses gmail, yahoo, Office365 or some other mail service which already does spam filtering. The only person in recent history that I know of running a private e-mail server was Hillary Clinton and see how much good it did her...
Why? Because the Oracle has spoken.
The vast majority of users don't give a damn about security. Their only concern is about what is happening to Aunt J who lives a couple thousands of miles away. She might leave them a hundred gazillion dollars to the most doting offspring after all. So curious here, what is the price of privacy?
Time is what keeps everything from happening all at once.
At the top end, the big tech companies like Google or Microsoft have their own spam-filtering systems in-house. At the bottom, individuals and entities too small to run their own mail servers either depend on Bayesian filtering in their e-mail clients or get email from one of the big tech companies. And in the middle, they either outsource their email to one of the big tech companies or can put together their own spam-filtering solution readily enough using available tools like SpamAssassin that're mostly open-source. End result: there's no market for spam filtering except as part of a complete email provider business on the scale of Google or Microsoft.
I think Spamhaus has taken this market and anyone else will have a hard time competing and still be profitable to make it worth while.
It isn't profitable. It's enormously expensive to pay so many employees to read EACH AND EVERY email to determine if it's spam.
I run a private email server on a hosted VM with Spam Assassin for me and my family. It's secure, private, and inexpensive. The biggest cost a a few minutes every couple months to keep it patched. Well worth it. My friends with gmail and yahoo addresses keep getting spammed, slammed, and pwned, but can't be bothered.
COE
In the past, spam was mass-flung with no real power. Filtering it was a relatively easy task, with an acceptable false positive rate and an even more acceptable false negative rate.
Today, while those spams still exist, between e-mail client junk folders and greylisting, the mass-flung spam is little more than an annoyance -- it doesn't have any real negative effect in term of dollars. Virus scanners catch those attachments pretty well too.
But now we have spear phishing -- real-world big-business, hand-crafted, artisan spamming. No spam filter is ever able to catch any of those. And they do real damage creating real monetary losses for big and small business alike.
So if your spam filtering business can catch the easy ones that do no real damage, and can't catch the hard ones that do the real big damage, then who's your paying market?
The time to have put a stop to spam email was long before Arpanet was even invented, let alone the Internet, or the Internet being opened up for access by the general public. The time to stop spam was way way back when the first bulk advertising mail to 'Resident' first occurred. If the U.S. Postal Service had said 'Hell, no!' to bulk mail, back in the day, we probably wouldn't have spam email now. As the situation stands right now, it's more or less impossible to stop, I'm sorry to say, and as such it's no mystery to me that any company that offers an anti-spam service would think twice about continuing to offer that service. Trying to herd ferrets on Adderall would be easier at this point than trying to stop the tsunami that is spam.
Are YOU using the TOOL, or is the TOOL using YOU? Think about it!
On the commercial side, there is Barracuda. On the free side, there is Spamassasin. That doesn't leave much room for others.
https://www.roaringpenguin.com... they provide and support CANIT PRO, which is basically mimedefang and spamassassin on a debian base, with dynamically updated blacklists and filtering rules. It works really well. David is one of the guys behind behind mimedefang, so you are also helping open source by going with these guys. The pricing for us was really decent.
They usually work with appliances, but we managed to use our own configuration to do some sweet stuff: we put the mail filtering cluster in the DMZ, along with the DB. but we put the customization interface is on an internal network. That way there is no firewall exception for the DMZ (ok except SMTP... can't avoid that one.) and the DMZ gateway doesn't need access to internal credentials at all (Active Directory in our case) It just knows that the interface machine on the inside is trustworthy. Even though the DB has no access to authentication services, the users can still customize their filtering to their desire.
I think for big companies, one concern is that I have never heard anyone rave about spam filtering. In terms of brand-awareness it is a completely one way street, Either people are satisfied with it, in which case they shrug, or they get irrationally violently abusive of the service, and have un-realistic expectations. It is a risk for any major brand to operate spam filtering, with literally no upside (ok, aside form revenue, but if it is a small part of a business, the reputation risk might outweigh the revenues.) Touching people's email brings out all the consipacy buffs you can imagine, and for some small but vociferous group they always have their own solution, and whatever the email admin does is crap. That's a thing that was great about Roaring Penguin's CanIT PRO when we rolled it out, it gave each user the ability to turn off the filtering entirely, if that's what they wanted.
It worked like a charm. Whenever we got some idiot (the truth hurts!) who thought they could do better, we just said fine, here is how to turn it off. Out of 6000 boxes, we had about 200 opt-out right away, most of them turned it back on within a few days, after a year it was down to 60 or so, and then when there were some malware infection episodes, it came out that their 'custom' solutions were not actually working that well, and everyone came back into the fold. Being able to let people opt-out saved us literally months of pointless arguments while letting us deploy good service for the co-operative many.
This was for about 7000 mailboxes, which is small as far as mail installations go these days. The real clients for this stuff is hosting providers and outsourcing companies (cloud based) I think the reason for large companies exiting the business is the huge trend of small companies to cloud, there just isn't much of a market for small email installs anymore... People are using huge hosted configurations. It's gradually getting dismantled now because of some organization move to a single outsourced solution with many hundreds of thousands of mailboxes...
The anti-spam market is small, mature and shrinking as more and more companies outsource their email to Microsoft or Google. While it can be profitable, the actual numbers are way too small to interest behemoths like Intel or Google.
I happen to run a small anti-spam company. We're doing extremely well, but that's because we have low overhead and can survive quite nicely on the little slice of market share we have. But I have no illusions that my company will be the next Facebook or Google or whatever... we'll chug along steadily for as long as we want to, and we'll make a very nice living at it, but that's about it.
put your mail server on a aaaa record only and you will see so little spam that you can filter it manually.
Thanks for the vote of confidence! (I'm from Roaring Penguin and am the MIMEDefang gal). But actually I go by DIanne now.
There is no such thing as a free lunch. When there's no competition left then we will suddenly see such services being an extra cost.
The problem with Office 365 is also that you have no ability to control the spam filter - legitimate mails gets junked without your knowledge.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
I had no idea! It shows how long it has been since we needed to talk! Thanks for the great product, Dianne !
Can't answer any questions about why the spam filtering services you mention are being discontinued, but the providers I work with that use a spam filter for their customers are mainly using Edgewave, with a smaller amount using Barracuda.
Palaces, barricades, threats, meet promises
Spam filters need to go away, as they only pass the buck along in the war on spam. They cost dramatically more than the sticker price (especially when they are "free") as the email is still sent, parsed, and quarantined. After that the filters need their rules updated regularly to catch the latest spammer tricks. Meanwhile as the spammers' botnets get bigger and more sophisticated it just gets that much easier and less expensive to send out spam.
If you want to end spam, you need to acknowledge that spam is an economic problem and spammers send out spam because they make money doing it. There is one and only one way to end spam, and that is to prevent spammers from making money off of it. No legal - or extralegal - action will slow it down by any meaningful amount. Interrupt the money flow and the spammers will find other work.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
Big companies like Google, Apple, Intel, etc. buy companies for access to their technology, for their patents and sometimes for their personnel. It is almost the normal thing for the purchased companies to be absorbed and their services to wound down. Google wants you to use nearly spam free G-Mail, not your own mail server or local ISP's mail service, so why would they continue offering a standalone anti-spam service? McAfee sells anti-spam as part of their security suites, why on earth would they sell the anti-spam service they've bought as an independent service under someone else's name?
Intel? Well who knows what goes on at Intel. My guess is that one management committee doesn't know what the other management committee is doing and haven't for decades now. Ditto Microsoft, IBM, etc.
legitimate mails gets junked without your knowledge.
That's a good start.... But how many 10s of dollars extra per month per mailbox, are you willing to spend, to have more control?
You might take a look at appriver secure tide (https://www.appriver.com/services/spam-and-virus-protection/) email filtering. It's SaaS email filtering that you put in front of your smtp server. It has reasonably good controls and does a better than average job. It's reasonably priced.We use it and have been happy with the filtering quality, price and support.
Barracuda networks also sells a SaaS spam filtering service, haven't used it, but have heard good things about it.
I'm not saying it's the Freemasons, but it's the Freemasons.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
What is the price of a lost important mail?
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
What is the price of a lost important mail?
It's pretty much zero, because if the email is important enough to actually matter, you will be asking the recipient to let them know, and contacting them over another channel, if they didn't get it.
So if the important mail is "lost", the actual price is attributable not to the loss of the E-mail, but human error in not anticipating the possibility of a lost e-mail and making sure the message gets through.....
was that most companies running exchange were just using Microsoft's spam filtering. This is like when people tell me the market for Mac users is huge. Yes, there are a lot of Mac users and they _do_ have lots of money, but the trouble when writing business software for them is you have to take out all the users who aren't willing to go out and buy a $300 PC to run your software :(...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Spammers would be prospective clients for Google advertising services
and be targets for Google to spam.
Go well
It is a tough nut to crack unless you have access to the complete mailboxes for the following reason:
- Any sort of AI/neural net/bayesian net is going to be only as good as the sample you train the system on. In most cases, it is easy to accumulate spam mails (honeypots etc), but it is hard to get hams (good mails). No enterprise customer would donate his "good mails" for research purposes.
- Running any sort of optimized neural network on customer box (via some sort of toolbar etc) doesn't help, because that is the first thing they disable.
- People are more likely to delete a mail rather than report a spam mail. Without access to usability data from their mail client, this causes more spam to more or less leak through.
- Spams are generally targeted regionally. A spam received by a person in USA is very different from the spam received by a person in China. This further restricts the accuracy of spam filters.
(Now these are not a problem for Google/Microsoft etc who have access to all these data)
Which leaves only secondary ways of detection: .. these are reactive rather than predictive, so some spams will always get through.
- Black list/pink lists/grey lists
- Rule based (regex/strings): Needs to be updated constantly, is less scalable, and needs a lot of multilingual people to stay up to date. Not very scalable.
- Reliance on the likes of libspf, which is still not as widespread as we'd like it to be.
Most email spam engines to my knowledge can easily catch upto 95% of spam.. may be 99% on a good day, but that remaining 1-5% earns them the ire of their customers. It seems to be just a labor intensive job, which is just not as rewarding as we'd like.
That doesn't work. It's not always apparent what's important, or even who's important, until much later.
Probably because so many companies are moving to cloud based email, where its on the provider to filter spam rather than the Exchange administrator, that the market's just not there anymore. Besides, there's freebies like spam assassin that can help a lot for many organizations.
Alas, yes, slack is partly replacing email where I work. One more medium to have to follow. BTW Google almost moved to Exchange in 2008 or so... IT wanted to, but upper management nixed it for obvious reasons. (And there was much rejoicing.)
Did Microsoft do the 5 year thingie with the Nokia purchase?
We've had good experience with Mailroute.net - actually replaced hardware Barracuda 300s with this, never looked back. Mailroute doesn't archive messages, just filters and forwards, so it won't help you if a you / a user deleted a message and you'd like to get a copy back from the 'backup'. But for basic filtering - really good, quite inexpensive. And for us the cost was really more about the $$ necessary to keep a Barracuda alive in a data center plus the yearly spam update subscriptions. It worked out to something like $2K / year for one Barracuda 300, and Mailroute is a fraction of that. YMMV / FWIW
Somewhat of a matter of approach. Most of them keep iterating on the same techniques in a constant arms race. Just so hard to make much more than an incremental progress and differentiate from everyone else.
I like the approach being used by Atriceps (http://www.atriceps.com)-- they're flipping the problem on it's head with a consent-based approach that looks for valid content and reputation, rather than exclusion.
Disclaimer: My company was in the Fall 2015 cohort of the Mach37 Cyber Accelerator with them.
Spamming is much older than ARPANET
http://blog.modernmechanix.com...
In those days bombing could solve the problem for real. But in all seriousness, this is why there are international laws about broadcasting into another country's territory. A law that Radio Free Europe conveniently ignored during the cold war.
The dangers of excessive individualism are nothing compared to the oppressiveness of excessive collectivism