Slashdot Mirror
5 Things the Boss Should Know About Spam Fighting
Esther Schindler writes "Sysadmins and email administrators were asked to identify the one thing they wish the CIO understood about their efforts to fight spam. The CIO website is now running their five most important tips, in an effort to educate the corporate brass. Recommendations are mostly along the lines of informing corporate management; letting bosses know that there is no 'silver bullet', and that the battle will never really end. There's also a suggestion to educate on technical matters, bringing executives into the loop on terms like SMTP and POP. Their first recommendation, though, is to make sure no mail is lost. 'This is a risk management practice, and you need to decide where you want to put your risk. Would you rather risk getting spam with lower risk of losing/delaying messages you actually wanted to get, or would you rather risk losing/delaying legitimate messages with lower risk of spam? You can't have both, no matter how loudly you scream.'"
Their first recommendation, though, is to make sure no mail is lost.
Nice goal, but you are going to lose mail. It is either going to get buried in the pile of spam or misclassified as spam by your software and pitched. What you need to do is pick an acceptable level -- it is all about trade-offs.
I like to REJECT (not bounce!) spam, so when you accidentally mark good stuff as spam, the sender has a chance to get the message to you later.
How does the CIO not understand what the IT deparment is doing and still become CIO? Can someone clue me in on the way a manager can know nothing of what they manage and still be a manager?
Inventions have long since reached their limit, and I see no hope for further development.-- Frontinus, 1st cent. AD
Trouble is how many CIO understand the technology they supervise enough to make a good business judgement?
The one thing I will tell them follows like this:
Trust your own I/T staff for maters of technical choice and direction, they have the most to gain, the most to lose and have to live with the consequences. Vendors know how to sell problems then the solutions, users know how to blame their lack of patience and personal issues on computers. I/T personnel often are the ones to eat the heat on organizational issues beyond their control. This includes the flawed systems we use today. Let I/T participate in business descisions, not to rule but nor to be a door mat for the next irrational business type having a conniption fit.
SMTP and POP
Now, nothing against educating management... but POP? POP doesn't belong in the enterprise. Even at home I have my own IMAP server. POP is a relic of the dialup-time where you only had access to your own computer and nobody else (seemed) to have one.
A shame that gmail doesn't support IMAP, I'd prefer it that way instead of that poor POP3 hack they use...
Was my spam filter installed backwards? It seems to let the ads through and trashes emails from my friends... Don't mind me, I am just auditioning for a CIO job. It pays a lot better.
Orignator of the Miserable Failure Googlebomb
Around 2000 there was legislation adopted in many states called the Uniform Electronic Transactions Act (UETA). Under UETA a legal notice sent by email is considered delivered to the recipient when it enters the recipient's ISP, regardless of whether the recipient ever sees the email. This was the UETA drafters' attempt to create the equivalent of something called the "mail box rule" for email. AFAIK, under the mail box rule, if you give a legal notice to the post office, it is considered delivered.
There are numerous examples of legitimate emails getting caught in spam filters, and there are ways to format a legal notice to raise the likelihood that it will be caught by a spam filter.
In addition to educating our corporate managements, we also need to educate legislators about this and to get UETA amended in the various states to recognize the realities of todays electronic commerce environment.
Forget CIOs... there are many system administrators who don't know the real issues regarding spam. Here are some things everyone needs to know:
1. Content filtering is not a solution.
I hate to say it, but it's the truth. Filtering mail based on what's in the e-mail message is a never-ending battle that does not work. It slows down mail service, causes legitimate mail to be blocked more often than using RBLs, and violates peoples privacy, costs more money to maintain and makes the mail system inherently less efficient and reliable.
E-mail used to be instantaneous. Now it isn't, because all the major ISPs toss their mail into big queues where they go over it and file it away or pass it on. If you send something to a Bellsouth users nowadays, they *might* get it 6+ hours later! Stupid, content filtering doesn't work and creates worse problems.
2. The Spam problem is mostly a law enforcement issue and not a technological issue.
99.9% of spammers break the law. The reason why spamming is such a problem is because national and international authorities won't get off their lazy asses and prosecute the spammers for the laws they break. In the end, you'll do more to reduce spam by petitioning your local district attorney to prosecute spammers than installing some obnoxious cpu-chewing filter that will become obsolete within two weeks. And no, the jurisdiction issue is bogus. Technology exists to track all these spammers right back to where they are. There are spammers all over the world and especially in the U.S. that can and should be in jail right now, but they're not because the Feds are more interested in going after people like Tommy Chong. Call your D.A. Call your Congressman. Complain that your reps aren't putting these guys in jail.
When I say "spam" I mean the big spam operations. The industry can easily police itself of low-level, incompetent opt-in schemes, but that's not the real "spam" problem we're talking about.
3. Don't listen to the anti-virus/anti-spyware software companies.
These companies make their living off of spam. There is an inherent conflict of interest in relying on Symantec or any other company to be trusted to help deal with the spam problem. They need spam and they'll never do what's necessary to stop spam from becoming more of a problem. This is analagous to why car manufacturers won't build more reliable/efficient cars when they are capable of doing so -- it's not profitable for them. Stop looking to McAffee or any of these other foxes to be trusted in helping you guard your henhouse.
4. Most anti-spam methods do nothing to stop spam, except relay blacklisting.
Spammers steal bandwidth, violate peoples' security, tamper with third-party computers and bog down the Internet. Content-based filtering does not hurt spammers. RBLs do. Relay blacklisting is the single most effective deterrent in the war on spam. PERIOD. No other method both stops spam, and makes it exponentially more expensive and troublesome for spammers to do their job.
Relay blacklisting works. If you don't like RBLs, chances are you just had a bad experience with a bad one. Try a different one or create your own. They work. They work exceptionally well and best of all, they save bandwidth and resources from the spammer's grimy hands. They also have the added benefit of stopping the propagation of worms and punishing irresponsible ISPs who allow their zombie users to pollute the Internet. There is NO BETTER THING CURRENTLY you can do to combat the spam war than by feeding and using RBLs (aside from following #2 and complaining that spammers aren't being prosecuted).
5. There are not that many spam operations. The spam epidemic is not unstoppable.
The amount of spam going around on the Internet has increased but only proportionally to the amount of user and bandwidth growth, and not due to more and more people getting into the spam business. A cursory examination of most spam clearly indicates that there are
I'm shutting down our lab mail server and migrating a large userbase to central university mail services because of all the problems we're experiencing with supporting an internal mail server. Everything from excessive spam (and it's well over 90% of all incoming connections), people using email as for storing files (as if it were a home directory), and recent rulings demanding that IT offices track email and IMs.
I worked out how much staff time we spend maintaining and supporting our mail server and was shocked. For a service that's commoditized and available for free from any number of vendors (never mind our uni's central IT service we're already paying for), and I worked out that last year we had spent ~100 hrs/yr of staff time. Looking back I realized that in years previous we had spent far less on a per year basis. IOW: staff consumption on mail service was growing while prices for commodity email service was plummeting (all the way down to near free).
Dumping email support is the only rational solution.
Where will this go? I think email (as in RFC822, etc) is doomed. The protocol is broken. It has no safeguards to confirm the legitimacy of the sender or recipient, no mechanism to secure the communication during transmission (like a real envelope), and as a result the protocol begs to be exploited by Internet fucktards. Which is exactly what's happening. Time to toss SMTP and start from scratch.
I really miss my Blue Frog. Just a promising little pet that never had a chance. Maybe Okopipi will make an appearance someday.
remember, Bill Gates said he would end spam. As a "trusting" MS user, I believe him. So, since spam has ended, I don't know what these "systems" guys are complaining about. Geeez.
General, you are listening to a machine! Do the world a favor and don't act like one.
Now try to implement your suggestions on a mail server that supports 500 users or more. Good luck.
..make sure it is clear to your boss that they might lose some legitimate email with porn because of spam filters.
1. Sending email gets infuriating as your machine slows to a crawl anytime someone hasnt whitelisted you.
2. Maintaining a Taint Free Whitelist gets to be a bit tricky.
3. How is this going to work for services like Gmail and Yahoo? A minute of chug time on a machine is expensive if your offering it for free. If you whitelist them it doesnt do much good because then spammers just use those accounts
4. How does this work for people in poor areas of the world using some antique machine (like a Pentium 200 mmx) where email would take 30 minutes a peice to send. Counter Proposal--- 0) No white list, however a digital signature (spam score/service type) from trusted trackers (like spamhouse, etc). 1) A proposed challenge is sent for (CLASS1 Trusted) 2) The client will either accept or decline the challenge to drop to the next level (CLASS2 Trusted) which would be a lighter challenge, but sorted accordingly. 3) After negotiating the terms of transmittal then the problem would be solved, and tagged appropriatly. With standard sendmail at the very bottom.. _________ some of the kickers... The spam score is a monster again.. as you need to have both a "start date" and a "most recent date" to classify the longevity of the account, and that it hasnt been used for spam lately. As well as having a "Diversity" score that keeps spammers from farming accounts for later use. The spam score keeps people from having to endure the longer wait. It also allows for the free-email systems to track individual accounts without so much work.
As far as the challenge goes I would go with an AES key that needs the last X digits solved.. But hash collision seems fine to me.
It is still a pretty weak solution for the people with low computer power..
Storm
Here's a stupid question? If 99% of email is spam now, why don't we all just switch to a protocol and servers that authenticate and force identities based on a distributed trusted service? Sounds like there is so much to gain by jumping off the SMTP ship.
You are checking your backups, aren't you?
"...about 200 employees with mailboxes."
That is such a small number of users, that you anecdotal evidence is meaningless.
You don't get spam because you don't have many users sending mail, your users are in a controlled corporate environment that (probably) keeps their machines virus/trojan/spyware free, your users probably are somewhat careful to only use their "work e-mail" for "work-related" stuff, and you have a domain that isn't very widely-known.
Try running an ISP with hundreds of thousands of users, a large percentage of which have viruses on their machines, and with a domain name that is a target of spammers (because you have so many users).
200 users is NOTHING. Until you are processing hundreds-of-thousands of messages per hour, you don't know how difficult it is to stop spam.
From my read on the anti-spam laws, the company would be an ISP for the employees. Given that, the company can sue the spammers that use deceptive headers and subject lines in their e-mails. Under California law, a recipient or ISP can get $1,000 per illegal e-mail.
When it starts costing spammers more money than they make, they will stop. In my experience, asking spammers to stop nicely does not work. Filing a lawsuit usually is the only way to get them to stop. I have one spammer that still spams after getting 6 figures yanked from their payment processing account. This time, I am asking for 7 figures in punitive damages.
Fight Spammers!
If CIOs instituted a policy of disqualifying any vendor of Internet, data or communication services that appears anywhere on Spamhaus's top 10 list from doing any business with the company, Varshavchik feels, "the spam problem will pretty much disappear, mostly overnight."
That list (http://www.spamhaus.org/statistics/networks.lasso ) has verizon.com, att.net, serverflo.com, xo.com in spots 1, 2, 3, 4. Should CIO's stop using Verizon, ATT and XO until they clean up their act?
It's amazing how quickly Slashdotters switch from quoting the Bill of Rights in order to defend freedom of speech that they want to ignoring the Bill of Rights in order to to condemn freedom of speech that they don't want. It's no wonder American lawyers earn so much!
Quidnam Latine loqui modo coepi?
Anonymous Coward said:
Translation:
Thanks for spreading the problem, idiot.
It hurts more if you leave it in the can.
Triv
The trick is to target the one vulnerability all spammers have: A website to sell their goods. All spam messages have a link where you click to buy the viagra, invest in Nigerian hedge funds, etc.
This vulnerability could be renlentessly attacked by ISPs, where each filtered spam generates an automatic "opt out" message to the website contained in the email. Kind of like bluefrog, with attitude. The beauty of it is, unlike bluefrog, there is no single point the spammers can attack, since individual ISPs would be generating the opt out requests instead of a single website.
Right now, a spammer only has to process the requests from the spam that actually gets through and is responded to. If this is implemented, the spammer would have to process (or ignore) every spam sent out by one of his zombies. Kind of a Self-Denial of Service attack.
When you have to process 18,000 requests a day, your hardware and bandwidth costs are minimal. If you had to process all 18,000,000 your zombies sent out, your costs would be considerably higher, and it might make spamming somewhat less profitable.
"Be grateful for what you have. You may never know when you may lose it."
A 200 meg attachment is nothing! A good 10 years ago, a client asked us if we could provide data in a database format similar to one she was using for another project. Boss asked for a sample. The next morning, I noticed the partition with the mail spool directory kept filling up, emptying, filling up, emptying. Finally figured out it was because someone was trying to send a message with a monster attachment. Moved the spool to a bigger partition. It kept growing and growing and growing and growing. Eventually, the entire 430 meg file came through and was delivered. It took a heck of a long time over our 128k ISDN connection. :)
I was kinda proud that the mail server I'd built from spare parts was able to handle it (once it had enough room to store the file).
Enforce one standard of encryption internal, for all employees and all clients that want to do email communication with the company. Bounce all messages that aren't encrypted.
Voila!
All Spam problems solved instantly.
Neat side effect: Your emails are safe and contract proof.
We suffer more in our imagination than in reality. - Seneca
>The trick is to target the one vulnerability all spammers have: A website to sell their goods.
Not any more. The stock scammers can get their money without any contact information whatever in the spam.
Problem: What happens if the spammers discover you doing this, and send new spam with a link to your website?
Even if people do it manually, this is going to sting legitimate people who have nothing to do with the spam.
Don't thank God, thank a doctor!
... email is not delivered by trucks driving through tubes.
One key area to consider is the root source of spam. Most spam comes from bot infected computers that circumvent inbound and outbound anti-spam techniques. The root solution is to remove the spam generating malware from infected computers. Although a long-run approach, spam malware cleanup is probably the only true way to reduce the world-wide volume of spam. Act locally, think globally.
The above rant is just a string of strawman arguments without an iota of evidence. It ascribes to filters disadvantages which do not exist, and to RBLs fantastic properties that also don't exist.
Maybe RBLs are useful in the fight against spam -- maybe not. To suggest that they obviate content filtering is preposterous.
Mod parent down.
Since the article is spread over three pages with ads, here is the list:
1. Lose No Mail.
2. There's No Silver Bullet.
3. It's a Continuous Battle. Budget Accordingly.
4. Understand the Basics of E-mail Technology.
5. People are Making Money on Spam. Respond Appropriately.
#4 is pretty funny: Boss? Understand basic technology? Buahahahahaha! That's a good one.
We spent most of 2006 looking for the best possible solution to our spam problems and had many meetings and spoke with many 3rd parties. At the end of that discovery, despite my strong distaste for it, we outsourced. I hate taking on additional periodic expenses, but in this case it just made too much sense. The spamassassin solution we had been working on constantly was costing us too much in manpower for not very good results.
We used an outfit called Red Condor. They offered external filtering by setting the MX to systems on their network, plus in-house filtering by way of an appliance that you can purchase and deploy. They allowed us a 60 day trial, which went extremely well. The bottom line is this, we now pay about ~$11k a year for ~10k mailboxes and get filtering every bit as good as what you get from the major email players like Gmail or Hotmail. The only downside is there are occasionally delays of up to 15 minutes. Hence it is almost, but not quite a Silver Bullet. These are issues that I expect can be somewhat resolved by purchase of additional appliances and load balancing.
This sounds like an ad, but I have no affiliation with Red Condor beyond being a customer. Spam and it's associated problems made 2006 the worst year of my 10+ year career and probably had contributed to more sleep deprived nights than any other thing for me. If you're like me and looking for a solution to what has become an epidemic, this is could be it.
Sigs are awesome huh?
To all of you people in here saying content filtering doesn't work:
How can you say that knowing that Yahoo, Gmail, Hotmail and AOL all do extremely effective content filtering? They aren't perfect but they're very very good with a low false positive rate.
Sigs are awesome huh?
Uh, from TFA: "Sequestering it is only slightly better than dropping it, because you have to look through the sequestered spam, and most people don't bother."
Can't find examples of evolution? No matter, neither could Dawkins
Both you and grasshoppa are right... for the subsets of the spam problem that you have to address.
My credentials? I run the mailfilters at a university with +50k student addresses, and around 3000 staff addresses. We typically reject a couple of hundred thousand messages daily. So, while our situation would probably turn grasshoppa's hair grey, it's a drop in the bucket compared to yours.
Spam is reasonably surmountable provided you have enough resources to throw at it. However, sometimes that doesn't scale too well
For example, I have two external relays load-balanced and ticking away running a whole lot of RBL checks, SpamAssassin checks and even using the Sane Security add-on signatures for CLAM while they do the regular virus scans. They presently have a whole lot of headroom, and if that ever diminished I could probably get a few thousand dollars for another box to share the load without too many questions given how well our solution works for us and how little it is costing otherwise. I imagine grasshoppa would have similar success with his or her solution. If you're talking hundreds of thosands of messages per hour, scaling my approach out to process that kind of message volume on a corporate budget dictated by the returns your bosses require could be interesting to say the least.
I'd add another item to the list, which some might see as a part of "No Magic Bullet", but I see it as sufficiently important to warrant its own entry - "There is no one-size-fits-all solution" . What works well-enough for you or me or grasshoppa will probably work well enough for similiarly sized organisations with similar resources and similar threats and similar values of "work well enough" - change any of that, though, and it's a whole new problem. The spammers are probably laughing their arses off over the kind of kind of religious wars people have over whose spamfiltering solution is best (it's mine, by the way, but then I also use vi to edit my conf files so of course it would be). And diversity of solutions is probably a good thing, as it means the spammers need to work harder to get past a whole lot of different approaches to get their garbage out.
A horrible solution, Challenge Response is... Let's assume, for a minute, that it's all handled server-side and the user doesn't have to deal with misdirected bounces. Realize that with the advent of botnets, bandwidth and computational power is something spammers have in spades-- far more so than legitimate mailers.
Let's also consider mailing lists. I manage a site that has tens of thousands of users, running on two MX boxes and one outbound SMTP box. I'd have to get a whole new RACK to handle the load you're suggesting...
Yes you can, its called dspam, and it works beautifully.
I, and none of my users, have seen an single spam email in over 3 years. I added graymilter and Project Zen from Spamhaus very recently, and its helped even more.
Sure, there are false positives that get caught and quarantined, but dspam has a nice webui that let's me retrain them and forward them on to my mailbox. The users have the same web interface and can manage their own false-positives in the same way. They can set it to catch more, or catch less with a few clicks in the interface. Some of my users love HTML email from online stores, and some do not. Everyone can tweak and train the heuristics for their own mail, however they wish.
I have no problem now making any of my email addresses visible on the Internet, on forums, wikis, mailing lists or webpages, because I simply do not get spam, so its not a problem anymore.
No, you're not getting both. You're just going for the risk of seeing something late, rather than the risk of losing something legitimate. Obviously, a quarantine means that you won't see the false positive until you specifically go check, but you won't lose it, unless you don't check for it before the quarantine's auto-delete timeout. Graylisting, by definition, introduces a delay in mail transmission.
Vintage computer games and RPG books available. Email me if you're interested.
There is no auto-delete timeout for the quarantine, not by default, and not that I can manually set without futzing in the code itself. I'm thankful for that, and so are my users.
A delay of 25 minutes is barely perceptable. Email is not IM, even though people assume the two to be interchangeable. They're not.
Besides, you could also use nolisting instead, if you so choose. I prefer to receive ALL of my mail, not potentially lose it without even knowing about it.
This was posted here previously, but it's a great idea. Annoying spammers with pf and spamd
The biggest problem is that it requires some OpenBSD knowledge. It'd be great if we could get a nice idiot-proof install ISO for a drop-in box.
If the SEC REALLY wants to enforce the law, all they need is a single email account to collect a bazillion pump and dump operations. It shouldn't be that hard to come up with a good list of suspects by watching what stocks get pumped, and then see who dumps. That alone could get rid of about 25% of the spam.