Slashdot Mirror
5 Things the Boss Should Know About Spam Fighting
Esther Schindler writes "Sysadmins and email administrators were asked to identify the one thing they wish the CIO understood about their efforts to fight spam. The CIO website is now running their five most important tips, in an effort to educate the corporate brass. Recommendations are mostly along the lines of informing corporate management; letting bosses know that there is no 'silver bullet', and that the battle will never really end. There's also a suggestion to educate on technical matters, bringing executives into the loop on terms like SMTP and POP. Their first recommendation, though, is to make sure no mail is lost. 'This is a risk management practice, and you need to decide where you want to put your risk. Would you rather risk getting spam with lower risk of losing/delaying messages you actually wanted to get, or would you rather risk losing/delaying legitimate messages with lower risk of spam? You can't have both, no matter how loudly you scream.'"
Their first recommendation, though, is to make sure no mail is lost.
Nice goal, but you are going to lose mail. It is either going to get buried in the pile of spam or misclassified as spam by your software and pitched. What you need to do is pick an acceptable level -- it is all about trade-offs.
I like to REJECT (not bounce!) spam, so when you accidentally mark good stuff as spam, the sender has a chance to get the message to you later.
How does the CIO not understand what the IT deparment is doing and still become CIO? Can someone clue me in on the way a manager can know nothing of what they manage and still be a manager?
Inventions have long since reached their limit, and I see no hope for further development.-- Frontinus, 1st cent. AD
Trouble is how many CIO understand the technology they supervise enough to make a good business judgement?
The one thing I will tell them follows like this:
Trust your own I/T staff for maters of technical choice and direction, they have the most to gain, the most to lose and have to live with the consequences. Vendors know how to sell problems then the solutions, users know how to blame their lack of patience and personal issues on computers. I/T personnel often are the ones to eat the heat on organizational issues beyond their control. This includes the flawed systems we use today. Let I/T participate in business descisions, not to rule but nor to be a door mat for the next irrational business type having a conniption fit.
Just routing what you suspect as spam into a separate folder. That way, if anything looks legit in there, I can double check before deleting it. You know, leave the power with the people. Why do you guys feel the need to protect us from ourselves? Oh, nanny IT. Rather than train your people properly, just protect them from themselves... it works so well for government after all.
How to eliminate spam:
0) Use a whitelist. Validate incoming messages against trusted keys using strong public-key encryption. This has been around for, oh, 30 years.
1) Don't bother with any blacklists.
2) Incoming messages not on the whitelist are automatically returned with a challenge. The user does not see the message until the challenge is solved, at which point the sender could be added to the whitelist.
One good challenge involves finding a partial collision for a strong hash. For example, find a 12 byte string which when appended to [some 4 random bytes] hashes to [some 4 random bytes plus the remainder padded with don't care bytes]. Using a partial collision allows the difficulty to be tweaked. For example, a challenge requiring one minute of average computation could only be solved 1440 times per day. Most email would be whitelisted and spammers would not be able to solve challenges economically.
3) No more spam!
SMTP and POP
Now, nothing against educating management... but POP? POP doesn't belong in the enterprise. Even at home I have my own IMAP server. POP is a relic of the dialup-time where you only had access to your own computer and nobody else (seemed) to have one.
A shame that gmail doesn't support IMAP, I'd prefer it that way instead of that poor POP3 hack they use...
Was my spam filter installed backwards? It seems to let the ads through and trashes emails from my friends... Don't mind me, I am just auditioning for a CIO job. It pays a lot better.
Orignator of the Miserable Failure Googlebomb
Around 2000 there was legislation adopted in many states called the Uniform Electronic Transactions Act (UETA). Under UETA a legal notice sent by email is considered delivered to the recipient when it enters the recipient's ISP, regardless of whether the recipient ever sees the email. This was the UETA drafters' attempt to create the equivalent of something called the "mail box rule" for email. AFAIK, under the mail box rule, if you give a legal notice to the post office, it is considered delivered.
There are numerous examples of legitimate emails getting caught in spam filters, and there are ways to format a legal notice to raise the likelihood that it will be caught by a spam filter.
In addition to educating our corporate managements, we also need to educate legislators about this and to get UETA amended in the various states to recognize the realities of todays electronic commerce environment.
Forget CIOs... there are many system administrators who don't know the real issues regarding spam. Here are some things everyone needs to know:
1. Content filtering is not a solution.
I hate to say it, but it's the truth. Filtering mail based on what's in the e-mail message is a never-ending battle that does not work. It slows down mail service, causes legitimate mail to be blocked more often than using RBLs, and violates peoples privacy, costs more money to maintain and makes the mail system inherently less efficient and reliable.
E-mail used to be instantaneous. Now it isn't, because all the major ISPs toss their mail into big queues where they go over it and file it away or pass it on. If you send something to a Bellsouth users nowadays, they *might* get it 6+ hours later! Stupid, content filtering doesn't work and creates worse problems.
2. The Spam problem is mostly a law enforcement issue and not a technological issue.
99.9% of spammers break the law. The reason why spamming is such a problem is because national and international authorities won't get off their lazy asses and prosecute the spammers for the laws they break. In the end, you'll do more to reduce spam by petitioning your local district attorney to prosecute spammers than installing some obnoxious cpu-chewing filter that will become obsolete within two weeks. And no, the jurisdiction issue is bogus. Technology exists to track all these spammers right back to where they are. There are spammers all over the world and especially in the U.S. that can and should be in jail right now, but they're not because the Feds are more interested in going after people like Tommy Chong. Call your D.A. Call your Congressman. Complain that your reps aren't putting these guys in jail.
When I say "spam" I mean the big spam operations. The industry can easily police itself of low-level, incompetent opt-in schemes, but that's not the real "spam" problem we're talking about.
3. Don't listen to the anti-virus/anti-spyware software companies.
These companies make their living off of spam. There is an inherent conflict of interest in relying on Symantec or any other company to be trusted to help deal with the spam problem. They need spam and they'll never do what's necessary to stop spam from becoming more of a problem. This is analagous to why car manufacturers won't build more reliable/efficient cars when they are capable of doing so -- it's not profitable for them. Stop looking to McAffee or any of these other foxes to be trusted in helping you guard your henhouse.
4. Most anti-spam methods do nothing to stop spam, except relay blacklisting.
Spammers steal bandwidth, violate peoples' security, tamper with third-party computers and bog down the Internet. Content-based filtering does not hurt spammers. RBLs do. Relay blacklisting is the single most effective deterrent in the war on spam. PERIOD. No other method both stops spam, and makes it exponentially more expensive and troublesome for spammers to do their job.
Relay blacklisting works. If you don't like RBLs, chances are you just had a bad experience with a bad one. Try a different one or create your own. They work. They work exceptionally well and best of all, they save bandwidth and resources from the spammer's grimy hands. They also have the added benefit of stopping the propagation of worms and punishing irresponsible ISPs who allow their zombie users to pollute the Internet. There is NO BETTER THING CURRENTLY you can do to combat the spam war than by feeding and using RBLs (aside from following #2 and complaining that spammers aren't being prosecuted).
5. There are not that many spam operations. The spam epidemic is not unstoppable.
The amount of spam going around on the Internet has increased but only proportionally to the amount of user and bandwidth growth, and not due to more and more people getting into the spam business. A cursory examination of most spam clearly indicates that there are
I'm shutting down our lab mail server and migrating a large userbase to central university mail services because of all the problems we're experiencing with supporting an internal mail server. Everything from excessive spam (and it's well over 90% of all incoming connections), people using email as for storing files (as if it were a home directory), and recent rulings demanding that IT offices track email and IMs.
I worked out how much staff time we spend maintaining and supporting our mail server and was shocked. For a service that's commoditized and available for free from any number of vendors (never mind our uni's central IT service we're already paying for), and I worked out that last year we had spent ~100 hrs/yr of staff time. Looking back I realized that in years previous we had spent far less on a per year basis. IOW: staff consumption on mail service was growing while prices for commodity email service was plummeting (all the way down to near free).
Dumping email support is the only rational solution.
Where will this go? I think email (as in RFC822, etc) is doomed. The protocol is broken. It has no safeguards to confirm the legitimacy of the sender or recipient, no mechanism to secure the communication during transmission (like a real envelope), and as a result the protocol begs to be exploited by Internet fucktards. Which is exactly what's happening. Time to toss SMTP and start from scratch.
I really miss my Blue Frog. Just a promising little pet that never had a chance. Maybe Okopipi will make an appearance someday.
remember, Bill Gates said he would end spam. As a "trusting" MS user, I believe him. So, since spam has ended, I don't know what these "systems" guys are complaining about. Geeez.
General, you are listening to a machine! Do the world a favor and don't act like one.
..make sure it is clear to your boss that they might lose some legitimate email with porn because of spam filters.
Sadly, I work for a commercial enterprise operating at a particular point along that spectrum of silver bullets. I say 'sadly', because we're terribly ethical and don't like employees embarrassing us by astroturfing, and that means I can't tell you what I think the silver bullets are...
1. Sending email gets infuriating as your machine slows to a crawl anytime someone hasnt whitelisted you.
2. Maintaining a Taint Free Whitelist gets to be a bit tricky.
3. How is this going to work for services like Gmail and Yahoo? A minute of chug time on a machine is expensive if your offering it for free. If you whitelist them it doesnt do much good because then spammers just use those accounts
4. How does this work for people in poor areas of the world using some antique machine (like a Pentium 200 mmx) where email would take 30 minutes a peice to send. Counter Proposal--- 0) No white list, however a digital signature (spam score/service type) from trusted trackers (like spamhouse, etc). 1) A proposed challenge is sent for (CLASS1 Trusted) 2) The client will either accept or decline the challenge to drop to the next level (CLASS2 Trusted) which would be a lighter challenge, but sorted accordingly. 3) After negotiating the terms of transmittal then the problem would be solved, and tagged appropriatly. With standard sendmail at the very bottom.. _________ some of the kickers... The spam score is a monster again.. as you need to have both a "start date" and a "most recent date" to classify the longevity of the account, and that it hasnt been used for spam lately. As well as having a "Diversity" score that keeps spammers from farming accounts for later use. The spam score keeps people from having to endure the longer wait. It also allows for the free-email systems to track individual accounts without so much work.
As far as the challenge goes I would go with an AES key that needs the last X digits solved.. But hash collision seems fine to me.
It is still a pretty weak solution for the people with low computer power..
Storm
You can't have a car that uses power, doesn't mess with the grid and uses no fossil fuel at once. You gotta have one, or the other, or a mix of both. You see, it's all a big tradeoff.
P.-S.: see this article.
Here's a stupid question? If 99% of email is spam now, why don't we all just switch to a protocol and servers that authenticate and force identities based on a distributed trusted service? Sounds like there is so much to gain by jumping off the SMTP ship.
You are checking your backups, aren't you?
You just have to explain the costs involved. By my estimates, if you paid someone to be an email secretary, they could accurately filter 3 to 4 persons emails a day. So, for only the cost of one more employee, the boss can his cake and eat it too.
Always remember that when the boss starts making outrageous demands, you can placate him by simply explaining how you can actually, well, placate him. The only obstacle, ever, is money. Ok, maybe money can't cure cancer, but it can certainly cure *this* issue.
A variety of comments posted already spout doom and gloom regarding email systems. When the truth of the matter is this; Spam is not that big of a deal if you use the proper techniques. I work for a small town city government, about 200 employees with mailboxes. Using about 4 different techniques ( from connection dropping based on connection metrics to content filtering of the actual message to tarpitting connections based on characteristics ), most of my female co-workers never see spam in their inbox, despite their damnedest attempts. Other admins will know what I'm talking about.
These five steps are good for your upper managers to know, but let's face it; They won't read that and understand it. Instead, use colorful graphs to highlight the work you have done to stop the spam, highlighting why certain online behavior is bad.
Mod me down with all of your hatred and your journey towards the dark side will be complete!
From my read on the anti-spam laws, the company would be an ISP for the employees. Given that, the company can sue the spammers that use deceptive headers and subject lines in their e-mails. Under California law, a recipient or ISP can get $1,000 per illegal e-mail.
When it starts costing spammers more money than they make, they will stop. In my experience, asking spammers to stop nicely does not work. Filing a lawsuit usually is the only way to get them to stop. I have one spammer that still spams after getting 6 figures yanked from their payment processing account. This time, I am asking for 7 figures in punitive damages.
Fight Spammers!
If CIOs instituted a policy of disqualifying any vendor of Internet, data or communication services that appears anywhere on Spamhaus's top 10 list from doing any business with the company, Varshavchik feels, "the spam problem will pretty much disappear, mostly overnight."
That list (http://www.spamhaus.org/statistics/networks.lasso ) has verizon.com, att.net, serverflo.com, xo.com in spots 1, 2, 3, 4. Should CIO's stop using Verizon, ATT and XO until they clean up their act?
It's amazing how quickly Slashdotters switch from quoting the Bill of Rights in order to defend freedom of speech that they want to ignoring the Bill of Rights in order to to condemn freedom of speech that they don't want. It's no wonder American lawyers earn so much!
Quidnam Latine loqui modo coepi?
Logically, because there was commercial value and because she admits that it could be argued the message was off-topic she must also admit that it could be argued the mesage was spam. Her response does much to convince she was motivated by commercial reasons.
Regardless, the two questions I have are: Does intention matter in determining whether something is spam? Does value (her article certainly has some value) matter in determining whether something is spam? I'm leaning towards "no" on both counts. Intentions ultimately cannot be known, so we can hardly use them as any kind of metric. Value-added emails are nice, but if the primary purpose is the value then the spam-like aspects can simply be removed (e.g. just share the article text, not a link). If we identify value-added emails as not spam, spam will simply include relevant or valuable information to mask its spaminess.
It hurts more if you leave it in the can.
Triv
The trick is to target the one vulnerability all spammers have: A website to sell their goods. All spam messages have a link where you click to buy the viagra, invest in Nigerian hedge funds, etc.
This vulnerability could be renlentessly attacked by ISPs, where each filtered spam generates an automatic "opt out" message to the website contained in the email. Kind of like bluefrog, with attitude. The beauty of it is, unlike bluefrog, there is no single point the spammers can attack, since individual ISPs would be generating the opt out requests instead of a single website.
Right now, a spammer only has to process the requests from the spam that actually gets through and is responded to. If this is implemented, the spammer would have to process (or ignore) every spam sent out by one of his zombies. Kind of a Self-Denial of Service attack.
When you have to process 18,000 requests a day, your hardware and bandwidth costs are minimal. If you had to process all 18,000,000 your zombies sent out, your costs would be considerably higher, and it might make spamming somewhat less profitable.
"Be grateful for what you have. You may never know when you may lose it."
A 200 meg attachment is nothing! A good 10 years ago, a client asked us if we could provide data in a database format similar to one she was using for another project. Boss asked for a sample. The next morning, I noticed the partition with the mail spool directory kept filling up, emptying, filling up, emptying. Finally figured out it was because someone was trying to send a message with a monster attachment. Moved the spool to a bigger partition. It kept growing and growing and growing and growing. Eventually, the entire 430 meg file came through and was delivered. It took a heck of a long time over our 128k ISDN connection. :)
I was kinda proud that the mail server I'd built from spare parts was able to handle it (once it had enough room to store the file).
They should leave it to the techs. That's what they pay them for.
Enforce one standard of encryption internal, for all employees and all clients that want to do email communication with the company. Bounce all messages that aren't encrypted.
Voila!
All Spam problems solved instantly.
Neat side effect: Your emails are safe and contract proof.
We suffer more in our imagination than in reality. - Seneca
>The trick is to target the one vulnerability all spammers have: A website to sell their goods.
Not any more. The stock scammers can get their money without any contact information whatever in the spam.
Problem: What happens if the spammers discover you doing this, and send new spam with a link to your website?
Even if people do it manually, this is going to sting legitimate people who have nothing to do with the spam.
Don't thank God, thank a doctor!
Incorrect - email generated by legitimate users is injected via the ISP's mail relay, which is not on the RBL.
The trick is to target the one vulnerability all spammers have: A website to sell their goods.
Here is the part you're missing.
There is no longer a direct connection between the entity sending spam and the company selling stuff:
- 1 A company gets suckered into thinking spamming might help them make money.
- 2 A spamming service takes their money and laughs because it doesn't matter if the company makes money.
- 3 A botnet provider contracts with the spamming service to send X million spams.
A new set of sucker companies are born everyday, the company that went broke yesterday isn't running around warning new companies to avoid spamming lest they, too, go out of business and even if they did the new company wouldn't believe them anyway.Money is made, and therefore spam will continue, regardless of whether any given company profits from any given spam campaign.
--
Ann (people call me Nan) E. Mouse
Allow all Spam at once so that all mail traffic (except Spam) comes to a grinding halt.
The resulting shock will create enough mass-consciousness of the problem that it will be taken care of on a world-political scale.
... email is not delivered by trucks driving through tubes.
One key area to consider is the root source of spam. Most spam comes from bot infected computers that circumvent inbound and outbound anti-spam techniques. The root solution is to remove the spam generating malware from infected computers. Although a long-run approach, spam malware cleanup is probably the only true way to reduce the world-wide volume of spam. Act locally, think globally.
The above rant is just a string of strawman arguments without an iota of evidence. It ascribes to filters disadvantages which do not exist, and to RBLs fantastic properties that also don't exist.
Maybe RBLs are useful in the fight against spam -- maybe not. To suggest that they obviate content filtering is preposterous.
Mod parent down.
Since the article is spread over three pages with ads, here is the list:
1. Lose No Mail.
2. There's No Silver Bullet.
3. It's a Continuous Battle. Budget Accordingly.
4. Understand the Basics of E-mail Technology.
5. People are Making Money on Spam. Respond Appropriately.
#4 is pretty funny: Boss? Understand basic technology? Buahahahahaha! That's a good one.
We spent most of 2006 looking for the best possible solution to our spam problems and had many meetings and spoke with many 3rd parties. At the end of that discovery, despite my strong distaste for it, we outsourced. I hate taking on additional periodic expenses, but in this case it just made too much sense. The spamassassin solution we had been working on constantly was costing us too much in manpower for not very good results.
We used an outfit called Red Condor. They offered external filtering by setting the MX to systems on their network, plus in-house filtering by way of an appliance that you can purchase and deploy. They allowed us a 60 day trial, which went extremely well. The bottom line is this, we now pay about ~$11k a year for ~10k mailboxes and get filtering every bit as good as what you get from the major email players like Gmail or Hotmail. The only downside is there are occasionally delays of up to 15 minutes. Hence it is almost, but not quite a Silver Bullet. These are issues that I expect can be somewhat resolved by purchase of additional appliances and load balancing.
This sounds like an ad, but I have no affiliation with Red Condor beyond being a customer. Spam and it's associated problems made 2006 the worst year of my 10+ year career and probably had contributed to more sleep deprived nights than any other thing for me. If you're like me and looking for a solution to what has become an epidemic, this is could be it.
Sigs are awesome huh?
To all of you people in here saying content filtering doesn't work:
How can you say that knowing that Yahoo, Gmail, Hotmail and AOL all do extremely effective content filtering? They aren't perfect but they're very very good with a low false positive rate.
Sigs are awesome huh?
Yes you can, its called dspam, and it works beautifully.
I, and none of my users, have seen an single spam email in over 3 years. I added graymilter and Project Zen from Spamhaus very recently, and its helped even more.
Sure, there are false positives that get caught and quarantined, but dspam has a nice webui that let's me retrain them and forward them on to my mailbox. The users have the same web interface and can manage their own false-positives in the same way. They can set it to catch more, or catch less with a few clicks in the interface. Some of my users love HTML email from online stores, and some do not. Everyone can tweak and train the heuristics for their own mail, however they wish.
I have no problem now making any of my email addresses visible on the Internet, on forums, wikis, mailing lists or webpages, because I simply do not get spam, so its not a problem anymore.
No, you're not getting both. You're just going for the risk of seeing something late, rather than the risk of losing something legitimate. Obviously, a quarantine means that you won't see the false positive until you specifically go check, but you won't lose it, unless you don't check for it before the quarantine's auto-delete timeout. Graylisting, by definition, introduces a delay in mail transmission.
Vintage computer games and RPG books available. Email me if you're interested.
There is no auto-delete timeout for the quarantine, not by default, and not that I can manually set without futzing in the code itself. I'm thankful for that, and so are my users.
A delay of 25 minutes is barely perceptable. Email is not IM, even though people assume the two to be interchangeable. They're not.
Besides, you could also use nolisting instead, if you so choose. I prefer to receive ALL of my mail, not potentially lose it without even knowing about it.
Interesting how Alan Ralsky's name pops up in weird places. Know someone who knows a woman who had a long term affair with Ralsky in the mid 80s to early 90s. She ended up having to do jail time in the UP of Michigan along with him due to one of his schemes. Hell of guy he is. This was verifiable via newspaper articles and court records. The Mrs. must be incredibly greedy or she would have thrown his cheating ass out long ago. More than likely he uses her to try to protect assets.
This was posted here previously, but it's a great idea. Annoying spammers with pf and spamd
The biggest problem is that it requires some OpenBSD knowledge. It'd be great if we could get a nice idiot-proof install ISO for a drop-in box.
If the SEC REALLY wants to enforce the law, all they need is a single email account to collect a bazillion pump and dump operations. It shouldn't be that hard to come up with a good list of suspects by watching what stocks get pumped, and then see who dumps. That alone could get rid of about 25% of the spam.