Slashdot Mirror
CAN-SPAM One Year Later?
BigPoppaT asks: "Computerworld has an article reviewing the effectiveness of CAN-SPAM one year after it passed. In the article several anti-spam companies cite spam as a huge (and increasing) percentage of the total e-mail load. Most state that it is more than 50%, and some are saying as much as 75%. (This matches what I see in other articles on the subject.) Are these figures reasonable? I do not work for an ISP or maintain a mail server, but speaking as an end-user, I do not have anywhere near this much spam - more like 5 to 10 items a week (out of a few hundred messages). This is in my personal email - I do not recall ever receiving any spam in my work inbox. If the numbers above are reasonable, I wonder why I get so little spam? I am on a number of mailing lists, and have purchased things online, so it is not as if I have gone too far out of the way to hide my email address. I am not complaining, mind you, I just think it would be useful for the Slashdot readers who deal with this in an administrative capacity to explain it to the rest of us. Are the spam numbers being inflated by these anti-spam groups as a marketing tool? (This is not a rhetorical question - I really am not in a position to evaluate this, so those who know, please fill the rest of us in.)"
No. "Users" like their free crap. So they are willing to give out their e-mail address because it seems inocent enough. Then 2-3 months later they get their "free" spam and still haven't learned their lessons.
By playing around with permutations of my email address, I find that a large chunk comes from infected colleagues' and students' computers. Relatively little comes from web crawlers. I also get a burst at around 8:00-8:15 when the staff members turn their machines on, and another burst a little later as faculty drift in. During the holidays, the rate goes way down.
Have you ever registered a domain? Nearly all the spam I get is to an address I only use for registering domains. I'm careful with my primary addresses, and receive nearly nothing on them.
A lot of spam that hits the system you'll never see as well. A big chunk of spam lists have bad or nonexistent addresses in them. There's usually some poor schmuck (here, that's me) that has to check and see if an Important Business Contact just can't type, or if all those emails to betty1@example.com, betty2@example.com, etc. are aimed at insecure men.
Other popular targets for spam are sales@, info@, support@....etc. so unless you're responsible for one of those, that's more spam you won't see.
Lucky bastard.
This is the voice of World Control. I bring you Peace.
A year ago, I was in the same boat as the poster, with about 5-10 spams a week. Now, I'm getting closer to that many a day. It's annoying, but not unmanageable. For my part, I'm grateful that my spam load is much lower than some people have reported. The key benefit (besides less spam, of course :) is that all the anti-spam tools that have been developed to handle more spam easily take care of the compartatively little spam amount I get. In any case, I don't doubt that the huge numbers given for spam loads are at least close to accurate (unless those numbers come from AOL, which classifies way too much non-spam mail as spam).
However, I do wish the anti-spam leaders would finally start encouraging people to PGP sign their emails. While perhaps not perfect, it has all the benefits of systems like hashcash and allows for much easier verification of senders.
But what do I know -- I'm not an anti-spam leader. And I run my own mail server, so in their eyes, I *am* a spammer (just ask the more radical of them).
"Save the whales, feed the hungry, free the mallocs" -- author unknown
You may have successfully protected your email address and have ordered from businesses with some degree of integrity. You may also have a spam filter in place somewhere.
Bleh!
Let's face it, if spam isn't a big problem for you, then why would you want to pay money to BrightMail or some other spam filtering service in the first place? I think it's a pretty reasonable assumption that a large percentage of spam filtering services' customers have a problem with spam that they feel unable to cope with themselves. By definition then, they will have an above average percentage of spam in their legitimate email.
UNIX? They're not even circumcised! Savages!
I must be lucky, or have good filters :)
I rarely get spam, whereas my workmates get an average of 100 spams a week
---- Put Sig here:
just an addendum: I should mention that these figures are based on the average day, in which I see about 120 spam messages and six real messages.
Standing at the very edge of my imagination, I peered into the inky void and realised -- I couldn't think up a new sig.
I get approx 3000-4000 spam per day.
Training spam filters are taking some time.
domain name registrations
online fora and blog comments
usenet
Yeah, I leave my real e-mail address in all of those places. I used to be more careful, but SpamBayes is so good, spam just isn't a problem for me.
I work in academia. My email ends up on things like conference abstracts and journal articles, not to mention the University's on-line directory.
I get, on average, 300 emails per day, Over 250 of which are spam. Spam-assassin catches maybe 90% of those.
I have several email accounts. The ones that I have through my ISP get a few dozen spams a day, on the order of about 5% of my email. Just for grins, I set up a completely unfiltered account using my own hosting service, and attached it to PopFile (a free Bayesian filtering program for email) to see the relative spam load. It is roughly 98% spam, or roughly 500 spams/day. I use that address for some mailing list subscriptions, and it is posted on my hosting service site, which probably where it got harvested. Unfortunately, PopFile is still getting some false positives, so I still check the spam bucket for improperly marked messages. (I'm definitely looking for something that won't give me false positives; I would rather get a small amount of spam than lose any legitimate messages.)
From your description, I would guess that your ISP is nuking most of the spam before you see it.
Concealed Handgun License Courses in Plano, Texas
One of the many duties I hold at my company is managing the email flow.
We receive between 60k-80k messages a day into our company and of that, about 90% is spam.
I have found the people who get most of the spam are those who have their addresses in other people's address books. I think that spammers get lists of emails gathered by viruses that collect address books.
Of course my boss is the worst because his email is set up as the billing email for all of our domains. The benefit of this is I have a great control subject for my home grown spam solution. I can tell when it is working well by how much spam gets through to him. He gets about 1000 spam messages a day.
The "expert" estimates on spam percentages do vary. But one thing seems pretty clear. CAN-SPAM hasn't perceptibly reduced the flow of junk email since it went into effect 1/1/2004. That's why I have suggested that Congress seize a simple way to put some teeth into the law. Give U.S. citizens a right to private action. Why save the privilege of suing spammers just for ISPs, attorneys general, and the FTC?
I'm the mail admin at a company with a little more than 500 active mail accounts. We get about 110,000 Internet messages per week, and about 80% of those are spam. We're using SpamAssassin to detect it, and running a script against syslog to get those numbers.
Our SpamAssassin server correctly detects over 99% of the spam, and rejects about 92% of it outright at our Internet gateway. The 8% least-spammy-looking-spam is tagged and allowed through to allow for false positives, though none have yet been reported.
include $sig;
1;
I am the editor of a mid-size magazine (hard copy, not web). By necessity, my email address and those of the various department editors are published in the magazine and on our website so readers can contact us. Obviously, this is one of the worst possible scenarios, but necessary to address the lowest common demoninator among readership.
Due to this, I and the department editors that work for me (as well as the advertising and circulation departments) receive hundreds of spam messages daily.
I eliminate most of mine at the server level by filtering all email from non-U.S. servers based on IP (APNIC, LACNIC, and RIPE registry). The remainder get diverted to a spam folder by SpamBully, and are then reported to the FTC and to the originating ISP via SpamCop (not because I think it does any good, but because it makes me feel better).
Bottom line: about 80-plus percent of email is spam (except on deadline day).
Ignorance is curable, stupid is forever.
... 2795 spams in my GMail, to which I redirect three or four other addresses. Last delete was on Dec 1st (logically).
So I get roughly 100 spams per day, of which gmail will let one, maybe two through every fifth day or so. pretty good. I now use my gmail account pretty much exclusively.
Thinking back, my spam volumes appear to have gone UP since CAN-SPAM went into effect. As for my work address, 3 a day or so, but we run a lot of spam filtering here, and I don't have access to the figures blocked. I've certainly not seen any marked effect of recent legislation on the amount of crap I get in my inbox.
Screw you all! I'm off to the pub
I'm the Network Administrator of a moderately-sized University, and we have a Barracuda spam appliance as our mail gateway. It tags about 75% +/- 3% of all incoming mail as spam, and has a very, very low false positive.
Yes, spam volume really is that bad.
My Department manages mail servers with ~400 mail accounts. We would say that the spam problem has increased (along with the virus-generated-email problem) because we see the reports generated by the mailserver and filter. Our users, however, seem to have forgotten that spam is a problem at all. They have forgotten that mere months ago they received dozens (or in some cases hundreds) of spam messages per day. Now they receive few or none, and when they do they send them to us as trouble tickets! At the same time, looking at the growing number of messages hitting our servers (we filter out ~90% as spam at this point) it's pretty clear that spam has gotten worse since CAN-SPAM rather than better.
So it really depends on who you ask. Users may not even realize that their ISP or employer is aggressively filtering. To them it just looks as if spam has evaporated.
I wonder if we're actually filtering TOO well. With bosses having only slightly pointier hair, it might be hard to justify the budget amount we plow into spam/virus filtering. I've been tempted to knock the filter down a few percent to admit more spam, just to keep people remembering it's a problem! (except then I'd get more too)
~
"A foolish consistency is the hobgoblin of little minds." -Emerson
Many ISPs (and the webmail providers) have taken to just blocking the most egregious spam before it even gets to users. So your mailbox could be getting some spam that you don't even see. It still gets sent and clogs up the network though.
A Multiplayer Strategy Game for Mac OS X, Windows, and Linux
To get a rough idea of trends, I've been plotting stats on a mailserver I manage. In general, we see spam and viruses are increasing, while ham is decreasing. Spam is about 67% of incoming mail.
I also plot my personal spam stats but obviously an individual account is hardly representative.
I recently activated a spam collector on my inbox. Since December 12th (time of activation), I have received about 1200 spam messages. Not counting mailing lists, this is at least %75 (if not more) of my total mail income. So yes, I would say those figures are accurate.
I've worked in the email industry for almost 7 years now providing mail server solutions and support to thousands of companies all over the world all needing their own configurations. In my experience, the amount of spam sent on the internet reported to be being processed by mailservers on the internet is probably accurate, however, the amount DELIVERED is much lower. So much of the spam goes to non-existant addresses. Throw in dictionary attacks, and you're probably seeing a delivery rate of 1 in 20,000 across smaller systems, and a rate of 1 in 5,000 on larger systems.
With the dramatic improvements in spam filtering software, getting rid of spam is no longer the technical problem it once was. In my experience as a consultant to email administrators and as a market research in the messaging industry, other, derivative problems are now taking over. And these problems are the result of filtering.
There are several problems that now plague email administrators: 1) satisfying the vast resource requirements of a modern email filtering system, 2) handling an increased flow of end-user complaints (yes, increased), and 3) dealing with false positives.
Everyone knows that spam is an enormous problem. The 75% number quoted in this article is conservative. Many organizations I work with receive in excess of 90% spam. Dealing with a problem of this magnitude is of course absolutely necessary -- and most large companies have by now installed a spam solution.
Unfortunately, implementing a large scale spam filtering solution requires rolling out sophisticated enterprise software and managing expensive, complicated, and high maintenance storage devices. This storage is mostly eaten up by the spam quarantine (or "junk mail folder") -- something that is necessary to deal with the possibility of false positives.
Even assuming that the system is correctly installed, maintaining it is an ongoing nightmare. And with a spam filter in place, end-users tend to assume that any spam that does get through is the result of a system failure that should be reported immediately as a trouble ticket -- adding to the email administrator's burden.
Finally, even though the latest spam filters are pretty good at what they do, if you're looking for a 95% spam rejection ratio, getting a false positive rate of less than 0.5% in the real world is a challenge. And while most false positives are things like newsletters that you don't normally care about, occasionally something critical is eliminated. When that happens, the email administrator can lose his job.
So what does he do? He tunes down the capture efficiency of the filter to drop the false positive rate. In a recent survey, Sophos PureMessage (one of the big iron enterprise anti spam solutions) had a capture rate of 90% and a false positive rate of 0.04% (Network World Spam Survey from December 2004). IMHO, 90% is a terrible capture rate that would result in an unacceptable flow of end-user complaints. Why did Sophos tune their product this way? Because false positives are the number one concern of email administrators.
Bar none. Number one.
False positives get you fired. Spam gets you a few more trouble tickets. You decide.
Spam filtering will always be necessary, but a complete rethink is required to take the problem resolution to the next level without the attendant drawbacks of filtering. The rethink involves end-user authentication (read: this is not the same thing as SenderID's domain authentication), something that can be implemented today using an aliasing system.
I use ASSP for any of my customers who've implimented spam filters. It keeps global stats for anyone who wants to report back to them. My spam hovers around 60%, more on holidays when there is less legitimate mail. Oddly, within the first few weeks after installing the filter, my spam dropped down from 80% to 70%. I guess the spammers realized they weren't getting through.
I've had email addresses rendered useless by the sheer volume of spam. 50-100 spams per day, with maybe 10 legitimate emails hidden among the noise.
/etc/aliases file. Whenever one of these aliases starts receiving spam, I delete the alias. Problem solved.
Thanks to MS-Outlook worms, even internal corporate email lists started receiving some really offensive porno-spam.
Today I get only a few spams per month, but to achieve this I ended up abandoning my old domain and setting up a system of aliases whereby I give a different email address to every person or organization that asks me for one. I now have several hundred entries in my
Yes, I even give aliases to my family members, since they'll inevitably divulge my address to e-card companies and so on.
This Canadian scumbag, lawrence.marshall@gmail.com, alone is sending me 10-20 porn spams a day with phony unsubscribe links at the end. He's the "Ashley Spears" spammer.
r shall@gmail.comr shall@gmail.com
IT WOULD BE WRONG TO SEND lawrence.marshall@gmail.com SPAM AND JUNK MAIL!!!
mailto://lawrence.marshall@gmail.com
lawrence.marshall@gmail.com
WHOIS information for teenwhorepussy.com:
[whois.namebay.com]
Domain Name : TEENWHOREPUSSY.COM
Created On : 2004-11-09
Expiration Date : 2005-11-09
Status : ACTIVE
Registrant Name : Lawrence Marshall
Registrant Street1 : 260-3495 Cambie Street
Registrant City : Vancouver
Registrant State/Province : BC
Registrant Postal Code : V5M4R3
Registrant Country : CA
Admin Handle : LM93819
Admin Name : Lawrence Marshall
Admin Street1 : 260-3495 Cambie Street
Admin City : Vancouver
Admin State/Province : BC
Admin Postal Code : V5Z 4R3
Admin Country : CA
Admin Phone : 604-871-0037
Admin Email : lawrence.marshall@gmail.com
mailto://lawrence.ma
Tech Handle : LM93819
Tech Name : Lawrence Marshall
Tech Street1 : 260-3495 Cambie Street
Tech City : Vancouver
Tech State/Province : BC
Tech Postal Code : V5Z 4R3
Tech Country : CA
Tech Phone : 604-871-0037
Tech Email : lawrence.marshall@gmail.com
Billing Handle : LM93819
Billing Name : Lawrence Marshall
Billing Street1 : 260-3495 Cambie Street
Billing City : Vancouver
Billing State/Province : BC
Billing Postal Code : V5Z 4R3
Billing Country : CA
Billing Phone : 604-871-0037
Billing Email : lawrence.marshall@gmail.com
mailto://lawrence.ma
Name Server : NS2.DFDHOSTING.COM
Name Server : NS5.TPTHOSTING.COM
Registrar Name : NAMEBAY
Registrar WebSite : http://www.namebay.com
Admittedly, this is only my particular case. However...
In January 2004, I received roughly 1,020 spams. Last month (December 2004), I received over 3300 spams. And the number has not decreased in any month since March 2004.
Effective law, my a**.
Culprits?
http://www.spamhaus.org/rokso/
We have unique WHOIS addresses and a lot of the spam comes from here but also from website scraping.
You can also see the source of SPAM migrate around the world, as new lists are produced and the old ones sold on. Our oldest unique addresses now receive almost all their SPAM from Asia in non English Languages.
At my previous ISP I had once changed my userid to run away from the spammers (maybe 25% of my mail back then--maybe three years ago). When I got that new account it was spam free for a few months before I went to JavaOne and gave my email address to a few vendors. When I got back from JavaOne the trickle had started. I get about 99% spam in my inbox now.
The only usable email account I have is gmail. It catches 100% of spam, which I get sent very little.
I work for a corporation, our email scanner recieved 120,000 emails within the past 12 days. It only sent 10,000 that it determined wasn't spam to our email server.
"When I want your opinion, I'll give it to you." --leonstryker
I do PGP sign my mail, at least whenever http://enigmail.mozdev.com/enigmail is updated for the latest version of http://mozilla.org/products/thunderbird/thunderbir d. I have found that it causes problems. AOL has classified some of my mails as spam because of the PGP signiture. Apparently they can't differentiate between a PGP sig and random characters to fool filters. It hasn't been reported that other mails have been lost for the same reason, but it seems like a valid assumption.
Anyone else noticed this?