Slashdot Mirror
One Third of Email Now Spam
Himanshu writes "The volume of spam received by business has doubled over the last two years and it's going to get worse.
Analysts IDC reckons that spam represented 32 per cent of all email sent on an average day in North America in 2003, doubling from 2001. That figure is less than the 50 per cent or more junk mail statistic commonly cited by email-filtering firms like MessageLabs and Brightmail but it still represents a serious problem,"
One-third of e-mail is spam? But nine out of ten of my e-mails are spam... Nobody loves me. :~(
I get a ton of spam, check out some of my recent spams and a frequency plot. starting from when I began saving and filtering them. Many thanks to Paul Graham for his plan for spam, or I would be buried by 350 spams per day by now. It is only going to get worse! Based upon how many I get, the probability is more like 95% percent of my email is spam.
Then who is getting the other 66.6% of my email?
-------
Support Indy Music. Buy
From the logs of our anti-spam appliance, over the last six weeks or so:
That's right, about 96% of our email is spam, viruses, or otherwise ungood.I'd be delighted if the spam dropped off so it were only 32% of our mail. Think of all the things I could do with that extra bandwidth...
In fairness, the study says they were looking at businesses, and this is at a small ISP, mostly residential customers. But it's a good number to chew on nonetheless.
spam really needs to catch up. I know that over half the snail-mail I get is junk mail...
... another 2/3 to go then our job is done.
Sanford Wallace
I've had the same domain name for around ten years with a catch all email acount. 1 in 3 is nothing, for me its closer to 99 out of 100.
"Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
... that 1/3 of email is *not* spam. Where do they get these figures from? Is there a computer that tallies all the spam up, and if so, why can't it just kill the spam along the way?
It's about time it started going down.
"After I received 80,730 different emails trying to sell viagra, I started to wonder: How many different ways are there to spell Viagra?"
http://cockeyed.com/lessons/viagra/viagra.html
I think that they goofed. 1/3 of it is virus infected, another 1/3 is spam, and the remaining 1/3 are jokes from people that you barely know that are not that funny.
I get about 2500 spams a week to my work address, and I can't change my work email. It's on my business cards, and as a DB geek they won't get me new ones :(
Because of the extreme amount of spam that I get, my Bayesian spam filters are pretty strict. I lose valid email all the time!!!
Why just this morning, I came in and was going through my spam folder, and found that my good friend Gooshot Moneyface has been trying to get in touch with me! I was wondering why I hadn't heard from her for so long.
Why should I argue rationally with someone being irrational? I'll just mock them instead.
According to this article the problem is worse
"A diplomat is a man who always remembers a woman's birthday but never remembers her age." -Robert Frost
I'd like to have a statistic on how much of that spam is do to worms relaying themselves from infected networks. 80% of the spam I now filter has a worm or trojan attached. I rarely get the marketing spam anymore.
As more spam gets sent, the rate of response to spam will decrease. Which means spammers have to send EVEN MORE spam emails to get the same return on investment that they did a few weeks before.
I'm surprised it took this long for the ratio of spam to real to reach the level it has.
So things are better than the last time slashdot ran this story?
I doubt it.
-Colin
OK, so some I can understand, like how to make millions of dollars by investing in some guy in Nigeria. Or increase the size of your sexual organs (though I'm disturbed by the ones that state "I went from 2" to 6"!" I mean, my 2 year old son is 2", you know? What of freaks are in these testimonials?)
But the ones I really don't understand are the "stop spam with this email!" It's like the phone company selling you caller-ID systems that block unlisted or telemarketers numbers - then sell the telemarketers systems to get through those.
That would never happen, right?
52 Weeks, 52 Religions with John Hummel
anyone know how these stats compare with standard mail?
Comment removed based on user account deletion
I heard about it here on /. and installed it the same day. At first it marked ALL my mail as spam because I'm on a few list servers, but the adaptive learning function of it is getting much better. After I "unlearned" my list mails as spam, it'd still let about 60% of spam through. Now it gets about 40 out of the 42 spams I get a day. I don't mind deleting two (or hitting "j" for junk), and recent searches through the junk folder show no false positives.
Check it out...
See
Happy Spamiversary!
Celebrating Spam's Ten-Year Anniversary
U.S. is World Leader in Spam
This is by no means a good list of all the spam stories that have hit slashdot, just a list of the ones that seem to have no point, are glaringly obvoius, or are redundant.
Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
Filtering doesn't mitigate the problem.
So what if I don't have to see the mail? That doesn't mean my mailserver isn't using cycles to talk to some originating server, transfer, store and eventually delete that spam. The only saving grace is I don't have to pay for bandwidth on a usage basis (cable modem is still, happily, "flat rate").
But what happens if that volume gets to be high enough that it starts to affect my ability to use the bandwidth for other things?
What we have available are basically work-arounds; we need a concrete solution that addresses the basic problem.
So what is the problem? People soliciting without you opting in? Deceitful mail designed to make you open it thinking it is from a friend? The sheer volume?
The real problem is we haven't found an effective way to trace this crap back to the people supposedly "making money" with these schemes.
Solve *that* issue... put a name, address, and bank account to that spam, and we'll clean this stuff up in a hurry!
Diplomacy is the art of saying, "Nice doggie!" until you can find a rock.
Funny, here 98% of spam comes FROM our CEO. :)
Deletes every third email. No mess, no fuss.
Internet providers need to configure their mailservers to accept e-mail from authenticated servers and hosts only.
Finally, digitally-signed messages should become the norm, not the exception, where it's easy for Joe Newbie to check the signature against known databases.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Stick a mail proxy between the internet and Exchange, that way he still gets to keep using Exchange, and you have a simple proxying machine that can do arbitary scanning and filtering.
You can scan all incoming mail with spamassissin and clamav before it reaches exchange, bounce or drop bad mail and forward "passed" mail into the Exchange server
You could also hookup a challenge response script there too.
I do the same thing for a company mail server running Lotus Notes.
to get that much spam???
I tried to get as much spam as possible in order to test spamassassin. I posted my email address on usenet and on all porn sites i've found. I have also tried installing spyware and toolbars. Internet explorer now crash on all sites but no spam so far.
Now, i resort to post my address on slashdot
sm@bigserver.hopto.org
It does eight (yes, eight) tests on the subjects of every message. I havent even added body checking yet, and it catches most spam. I even tried replacing these 8 tests with the SpamAssassin engine and found that it was less good at detecting spam mails. The tests are so simple:
The blacklist is checked after first collapsing spaced-out words like "V I A G R A" and removing the above-mentioned obvious obfuscation. It's regex-based and contains the typical stuff like "meds" "medication" etc, but also a test for a subject that ends in 3 or more spaces followed by a string of random consonants.
When it detects SPAM, it simply changes the subject line to indicate that the message is spam.
In addition to spam-checking, it also removes all HTML mark-up (removes the tags leaving plaintext behind), deciphers MIMEd messages and recompiles them into multipart/mixed format (so images etc. are attachments) and renames many-extensioned attachments, so girl.jpg.pif becomes girl.pif.
It's still in dev, but it'll be available on baxpace.com in the next week or so for Win32 (as an exe) and UNIX platforms. It's written in Perl.
And one third of Slashdot posts are First Post
Here's what I do: Bitty Browser & Andromeda
If every Linux and Windows machine ran Postfix with CRM114 by default (and with manpages and documentation), this would help. Maybe a new anti-spam Linux distribution is needed. MacOSX ships with Postfix, but not CRM114.
Do you have any idea how many open-relays still exist? Why does SMTP software allow '*' open-relays in the first place? Do you know how many proxy servers are out there on the Internet? How many SOCKS4&5 proxies that just allow any SMTP to be bounced? How many are seemingly closed but available with the CONNECT method? Let's close some of our holes, and prevent software from opening them in the first place.
Also - know your enemy. Why haven't people dissected the software these creeps are using. The majority of spam comes from a program called DarkMailer or DM. Let's reverse engineer this application and figure out how it works, so our defenses can be built around the enemy's weapons and not just generalizations about spam.
Finally, let's set some ethics and procedures about how to deal with spammers. Too many is the case that people just want to beat their heads in with baseball bats or delete all their files on all their computers. This activity is not productive. It's my firm belief that if you take away their tools and educate them, less spam will be out there. You make it a war -- and that's what you'll get. Passion drives creativity and efficiency.
IPv4 allocations for hobbyists? join the ipalloc-l mailing-list! www.operations.net/mailman/listinfo/ipalloc-l
One day I'll simply snap and actually contact a spammer with the following order:
From: my@email.com
To: spammer@email.com
Subject: Order req'uest for >X<@n4x and V1agro! fxfj aspll cps
Dea'r Si:rs,
I w.ould l1ke t0 pl@ce 4n 0rder for tw0 p.ortions of Xa:n:aX' and v_i_a_g_r_a. P13aS.e sh1p im:medi`ate1y, 1 h@ve an><1ety and ne:ed a -bo-ner-
Y0urs s.incerel'y
S@vvy 1nvest0r
akdf k- dfks. dfk v9iew casoji ropdfk hork
aso, ckdo ofgkf opwerk- mmos odkaok s
w eofk, eoro gksod bz o-
Ran into this same problem at my company... Tested two different things out:
Mailwasher - Not a challenge/response like you asked for, but allows you to send bounces back to spam, and delete them off of the server before you donwload them. Can tie into SpamHaus and such.
ChoiceMail - Challenge response, both single user and enterprise are available. Single user sits on local machine, enterprise ties into Exchange. Can quickly add anyone in your Outlook contact list to the whitelist, and anyone you send an e-mail to can be set to be whitelisted. The challenge message can be customized. Biggest problem with the bounce (at least in my testing) is that the challenge gets rated as spam by my filters. I'm sure if the challenge was tuned up it wouldn't be that big of a problem. And they have a free trial so you can test it for 14 days
Nephilium
Their stats don't line up with mine -- the only thing I do agree with is that it is getting worse. It continuously has since March of last year it seems. Back then my base was about 500 a day THEN. Today it's much different, but let's digest some numbers.
:).
:)
:) yesterday alone (a typical day). This includes the harvested messages -- which now puts the email traffic at almost 98% being generated by spam.
... you do know me, right? :). Yesterday's already blocked address' attempting to send even MORE spam in was 2,251 for a total of 2 email address' which may send/get 6 emails in the same time frame. Now we're at 99.7% of the potential email traffic was all generated by spam. .3% was real.
Forgetting work -- let's just look at my home domain. Hosting my wife and myself I'll look at my email alone. In the last week we've sent/received 42 legit emails. That's about 6 a day between the two of us. In the same week the average _daily_ traffic looks like this:
I'll start by saying that actual junk mail that may make it to the Inbox in front of me is maybe 1 a week. I find even that annoying. Yesterday, an average day -- there were 109 messages harvested by spam sucking address'. Our daily average [last typical week] at home was 6 emails (sometimes less, sometimes more
By my numbers that is almost 95% of my email traffic which is simply not wanted, nor allowed.
There were also a total of 291 subnets blocked (for various other noticeable offenses
Of course, once blocked there's a URL sent back (-0- lookups in the same time frame) which tells you what to do (email a unblocking address or pickup the phone and call me
They're numbers, well -- just don't jive with my real life experiences.
While we've surely all seen enough spam, this is about the most thorough bit of spam I've seen in a long time. And its short - way more crap per line than usual.
Not only is it spam, it claims to be consistent with the CAN-SPAM act. How wonderful is that?
It has the usual set of junk words intended to try to disguise itself from the normal anti-spam software. And it has the usual image to load that contains my email address so it will know I visited there. And it encourages me to send it to all my friends. And it has the usual "visit here to get off our list".
Even better, if you go to their web page you'll find a pointer to a page where they say "It has come to our attention that ..." spammers are advertising their product, and you can complain by filling in a form. And, of course, giving them your email address! For those who are amused by such things, look at the source - its obfuscated to the point of absurdity and does not seem to like running under mozilla.
See my journal for more info, including the source of the mail, the urls involved and a decoding of their web page.
... is if they count the volume of "intranet" mail.
Corporations deal piles of mail on the inside, that never gets out to the genpop: HR crap, memos, meeting notices, etc. etc.
Customer relationships also generate piles of e-mail, but that should be visible to your average slashdotter who buys stuff.
I wonder if they're counting automated, machine-read e-mails such as SEC filings and other things that humans never read?
Design for Use, not Construction!
This message contains no text. Surprisingly, all the contents of the message has fit into the subject line. Clicking at a subject line with (n/t) for 'no text' brought you to read this incoherent drivel. Thank you for participating.
If programs would be read like poetry, most programmers would be Vogons.
We are using a new product called GWGuardian that we spotted at Brainshare. On average I was recieving somewhere in the range of 1500+ SPAM messages a week. With the GWG I have had 1 Spam mail make it into my inbox. Have to love it.
I doubt it'll keep spammers at bay forever, so I really should start looking into some more spammer hostile things I can do to my mail server. Worst case, I can always shut the damned thing down. I was ready to do that anyway. If the service is useless to me (Because filtering spam takes so long that I don't have time for anything else) why should I bother running it?
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
While the economics of email favor spam, spam will flourish. It's as simple as that.
To get rid of spam, we need to change the economics of email.
However, most systems proposed are too simple in that they serve to make a lot of the legitimate purposes of email too expensive, Maillists being a primary one, as well as mail from new potential customers.
Essentially, we can arrange email into a grid of Expected or Unexpected vs Desired or Undesired. We need a way to freely receive all Desired mail whether it is Expected or not, while making it expensive for mail that is both Unexpected and Undesired.
To address this, I believe a system where the promise of payment is encoded into the delivery may solve the problem. Note that the promise of payment doesn't mean that payment will be necessarily be required. However, having the promise encoded into the email does require that it be possible to place a charge on that email by the recipient. This would require verification at intermediate servers that the mail came from a known system that allows payment to be made before relaying it on.
Legit users send out so few emails that they could easily send out mails with promise of payment encoded, companies would not require the payment be made (as what a great way to lose a potential customer) so the status quo is preserved, and friends who they send mail to similarly would not bother requiring payment. Of course, if payment is required (you get into a fight with your friend) it should be a small enough amount (sub-dollar range) that it is not an extreme hardship even then.. provided you're only getting charged for one or two.
Mail-lists could be sent without the promise of payment, but since they are typically subscribed to directly, it becomes very easy to implement a white-listing solution for all the lists you're on.
Spam could not be sent using promise of payment -- if it was, the costs would quickly dwarf the profits since it is only the very low cost of email that makes spamming possible. Anybody receiving the spam would simply click the "Require Payment" button or some such, and the spammers credit card would be automatically charged the amount. Assuming only 25% of the recipients are actually able and willing to require payment, since the typical spam run sends out hundreds of thousands of email, the charges mount significantly quickly. Yet if spam was forced to not promise payment, since all legitimate email is using promise of payment, it becomes very easy to whitelist the spam out of existance.
Essentially, the promise of payment system allows unexpected but desired mail to proceed as normal, while unexpected undesired mail incurs a fee. Expected mail can use the standard email system with whitelists, or still use the promise system with no difficulties.
That Jesus Christ guy is getting some terrible lag... it took him 3 days to respawn! -NJ CoolBreeze
I have to assume IDC based its studies on mail filtering reports and technologies using servers that at some point, started deferring SMTP traffic and didn't actually compile complete stats on spam. There's NO WAY the spam-to-legit ratio is 33%. It's more like 85%, especially for any boxes hosting e-mail addresses which may be on file with domain records.
That study is flat-out inaccurate. When they use those lame content-based filtering systems, their mail system slows down so much, they cannot handle all the inbound connections so they never really know how much SMTP traffic they actually get. Spammers hit their lame servers, get deferred, and don't come back. I guess this might be one reason why you might want to use MS Exchange: it's so slow it can't actually process all the spam sent to it, and then you get incomplete figures on mail traffic and spam.
IDC estimates that each worker would spend an average of 10 minutes a day dealing with spam.
That seems a bit low to me. Maybe with content-based filtering in effect. But they should also ask IT managers how much time is wasted per-employee looking for legitimate messages that have been held up by the inbound mail filtering/flagging systems that erroneously trap legitimate mail. I bet that figure is much higher.
RBLs work. Content-based filtering doesn't. This whole study is basically a shill for promoting more ineffective "strip-searching" of e-mail content as a "solution" [sic] to the spam problem.
Nonsense. Even setting aside the obvious frauds contraband offers, unauthorized use of trademarks, etc. found in 99+% of spam, it is a violation of property rights. The First Amendment does not protect spamming any more than it protects grafitti vandalism.
At most, the law might reasonably tolerate spam if it evidences no attempt to evade filtering -- no forged headers, no "v1agra" munges, no misleading subject lines, no nothing. The use of such techniques creates a "bright line" between spamming and legitimate bulk e-mail, because it constitutes prima facie evidence of intent to intrude without permission (and, indeed, against an express prohibition).
Bottom Line: The computer-cracking laws ought to be clarified so that the evasion or spoofing of a spam filter is treated just like the evasion or spoofing of a password prompt.
/. If the government wants us to respect the law, it should set a better example.
Can anyone suggest a decent, doesn't have to be perfect, server side anti-spam filter?
/etc/access file such as: "connect:218 REJECT" will knock off about 200-5000 spams per day utilizing minimal system resources).
2 ,210,211,213,217,218,219,220,221 and you'll stop a TON of spam from a lot of foreign countries you likely never communicate with.
Don't waste your time implementing a content-based filter. The best solution is to incorporate a real-time spam relay blacklist. I recommend bl.spamcop.net. It's very effective and accurate with an extremely low legit mail blocking rate.
RBLs are great because they refuse spammer connections before the mail even gets delivered, so you don't waste bandwidth and system resources downloading spam crap and trying to interpret the contents. RBLs respect the sanctity of the e-mail message as a private communication medium and penalize those ISPs which allow spammers to operate.
If you're using Sendmail, you can also hard-code some of the IP regions where tons of spam is originating (signal-to-noise ratio for most people on the Chinese IP blocks is 0% so why allow them to hit your server in the first place? A few lines in your
Personally, if you want to get aggressive, block the following Class As: 61,80,81,82,83,142,164,193,194,195,196,200,201,20
Set up a web-based e-mail form and put a link to it in your Sendmail access configuration so that if any legit mail gets bounced, they can redirect to a web page to contact you in the [unlikely] event they were inappropriately blocked.
So, I'm thinking - those people who actually respond to spam? We should host an awards show for them - called "Too dumb to Live". We give them a chance to give their speeches and thank their whatevers, and then, when they leave the stage to go to the "press interviews", we can just dispatch them in some nice, efficient manner.
We should ALL do something to make the world a better place to live, ya know...
I really do think they means One-Third of Mail NOT Spam. I've read a dozen reports in the past year that said that half of all the email was spam. I know it's not decreasing. 2.5 years ago half of the email coming into a provider I contract with was getting rejected as spam. Now that number is even higher. 1/3 my foot. 3/4 is more like it.
If you're lucky enough to get a valid email address, feed it in to your other spam (using their handy verify^H^H^H^H^H^Hunsubscribe link). Also useful for abuse/postmasters who do nothing.
Seriously though, nothing will happen as long as China (and a few other countries) don't care. A spammer recently picked up my cable address (which I don't use), and hits me 2-3 times a day. I've traced it back to china, contacted the appropriate admins, and received a "abuse mailbox full" bounce.
I use Macs to up my productivity, so up yours Microsoft!
Seriously- if you think about it, spam may be our last hope for privacy on the net. The more legal measures we put against spammers, the more freedom we lose ourselves. So why not just accept spam as a fact of life and find some useful purpose for it, like camoflage for stego. I know there's several stego programs out there that disguise their transmissions as spam- if we get rid of the spam, no more camoflage. Don't get me wrong, I don't like getting ads for pr0n at work any more than anyone else, but I think there are other ways of dealing with it- without legally screwing ourselves in the end. (pun intended)
DISCLAIMER: This post was not checked for speling and grammar- if you complain- you're a whiner