Slashdot Mirror
One Third of Email Now Spam
Himanshu writes "The volume of spam received by business has doubled over the last two years and it's going to get worse.
Analysts IDC reckons that spam represented 32 per cent of all email sent on an average day in North America in 2003, doubling from 2001. That figure is less than the 50 per cent or more junk mail statistic commonly cited by email-filtering firms like MessageLabs and Brightmail but it still represents a serious problem,"
One-third of e-mail is spam? But nine out of ten of my e-mails are spam... Nobody loves me. :~(
I get a ton of spam, check out some of my recent spams and a frequency plot. starting from when I began saving and filtering them. Many thanks to Paul Graham for his plan for spam, or I would be buried by 350 spams per day by now. It is only going to get worse! Based upon how many I get, the probability is more like 95% percent of my email is spam.
Then who is getting the other 66.6% of my email?
-------
Support Indy Music. Buy
From the logs of our anti-spam appliance, over the last six weeks or so:
That's right, about 96% of our email is spam, viruses, or otherwise ungood.I'd be delighted if the spam dropped off so it were only 32% of our mail. Think of all the things I could do with that extra bandwidth...
In fairness, the study says they were looking at businesses, and this is at a small ISP, mostly residential customers. But it's a good number to chew on nonetheless.
spam really needs to catch up. I know that over half the snail-mail I get is junk mail...
... another 2/3 to go then our job is done.
Sanford Wallace
I've had the same domain name for around ten years with a catch all email acount. 1 in 3 is nothing, for me its closer to 99 out of 100.
"Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
It's about time it started going down.
"After I received 80,730 different emails trying to sell viagra, I started to wonder: How many different ways are there to spell Viagra?"
http://cockeyed.com/lessons/viagra/viagra.html
I think that they goofed. 1/3 of it is virus infected, another 1/3 is spam, and the remaining 1/3 are jokes from people that you barely know that are not that funny.
I get about 2500 spams a week to my work address, and I can't change my work email. It's on my business cards, and as a DB geek they won't get me new ones :(
Because of the extreme amount of spam that I get, my Bayesian spam filters are pretty strict. I lose valid email all the time!!!
Why just this morning, I came in and was going through my spam folder, and found that my good friend Gooshot Moneyface has been trying to get in touch with me! I was wondering why I hadn't heard from her for so long.
Why should I argue rationally with someone being irrational? I'll just mock them instead.
I'd like to have a statistic on how much of that spam is do to worms relaying themselves from infected networks. 80% of the spam I now filter has a worm or trojan attached. I rarely get the marketing spam anymore.
As more spam gets sent, the rate of response to spam will decrease. Which means spammers have to send EVEN MORE spam emails to get the same return on investment that they did a few weeks before.
I'm surprised it took this long for the ratio of spam to real to reach the level it has.
So things are better than the last time slashdot ran this story?
I doubt it.
-Colin
anyone know how these stats compare with standard mail?
Comment removed based on user account deletion
I heard about it here on /. and installed it the same day. At first it marked ALL my mail as spam because I'm on a few list servers, but the adaptive learning function of it is getting much better. After I "unlearned" my list mails as spam, it'd still let about 60% of spam through. Now it gets about 40 out of the 42 spams I get a day. I don't mind deleting two (or hitting "j" for junk), and recent searches through the junk folder show no false positives.
Check it out...
Filtering doesn't mitigate the problem.
So what if I don't have to see the mail? That doesn't mean my mailserver isn't using cycles to talk to some originating server, transfer, store and eventually delete that spam. The only saving grace is I don't have to pay for bandwidth on a usage basis (cable modem is still, happily, "flat rate").
But what happens if that volume gets to be high enough that it starts to affect my ability to use the bandwidth for other things?
What we have available are basically work-arounds; we need a concrete solution that addresses the basic problem.
So what is the problem? People soliciting without you opting in? Deceitful mail designed to make you open it thinking it is from a friend? The sheer volume?
The real problem is we haven't found an effective way to trace this crap back to the people supposedly "making money" with these schemes.
Solve *that* issue... put a name, address, and bank account to that spam, and we'll clean this stuff up in a hurry!
Diplomacy is the art of saying, "Nice doggie!" until you can find a rock.
Funny, here 98% of spam comes FROM our CEO. :)
Dear Freak,
Please stop measuring your son's penis.
Regards,
Child Protective Services
Deletes every third email. No mess, no fuss.
Stick a mail proxy between the internet and Exchange, that way he still gets to keep using Exchange, and you have a simple proxying machine that can do arbitary scanning and filtering.
You can scan all incoming mail with spamassissin and clamav before it reaches exchange, bounce or drop bad mail and forward "passed" mail into the Exchange server
You could also hookup a challenge response script there too.
I do the same thing for a company mail server running Lotus Notes.
It does eight (yes, eight) tests on the subjects of every message. I havent even added body checking yet, and it catches most spam. I even tried replacing these 8 tests with the SpamAssassin engine and found that it was less good at detecting spam mails. The tests are so simple:
The blacklist is checked after first collapsing spaced-out words like "V I A G R A" and removing the above-mentioned obvious obfuscation. It's regex-based and contains the typical stuff like "meds" "medication" etc, but also a test for a subject that ends in 3 or more spaces followed by a string of random consonants.
When it detects SPAM, it simply changes the subject line to indicate that the message is spam.
In addition to spam-checking, it also removes all HTML mark-up (removes the tags leaving plaintext behind), deciphers MIMEd messages and recompiles them into multipart/mixed format (so images etc. are attachments) and renames many-extensioned attachments, so girl.jpg.pif becomes girl.pif.
It's still in dev, but it'll be available on baxpace.com in the next week or so for Win32 (as an exe) and UNIX platforms. It's written in Perl.
And one third of Slashdot posts are First Post
Here's what I do: Bitty Browser & Andromeda
If every Linux and Windows machine ran Postfix with CRM114 by default (and with manpages and documentation), this would help. Maybe a new anti-spam Linux distribution is needed. MacOSX ships with Postfix, but not CRM114.
Do you have any idea how many open-relays still exist? Why does SMTP software allow '*' open-relays in the first place? Do you know how many proxy servers are out there on the Internet? How many SOCKS4&5 proxies that just allow any SMTP to be bounced? How many are seemingly closed but available with the CONNECT method? Let's close some of our holes, and prevent software from opening them in the first place.
Also - know your enemy. Why haven't people dissected the software these creeps are using. The majority of spam comes from a program called DarkMailer or DM. Let's reverse engineer this application and figure out how it works, so our defenses can be built around the enemy's weapons and not just generalizations about spam.
Finally, let's set some ethics and procedures about how to deal with spammers. Too many is the case that people just want to beat their heads in with baseball bats or delete all their files on all their computers. This activity is not productive. It's my firm belief that if you take away their tools and educate them, less spam will be out there. You make it a war -- and that's what you'll get. Passion drives creativity and efficiency.
IPv4 allocations for hobbyists? join the ipalloc-l mailing-list! www.operations.net/mailman/listinfo/ipalloc-l
One day I'll simply snap and actually contact a spammer with the following order:
From: my@email.com
To: spammer@email.com
Subject: Order req'uest for >X<@n4x and V1agro! fxfj aspll cps
Dea'r Si:rs,
I w.ould l1ke t0 pl@ce 4n 0rder for tw0 p.ortions of Xa:n:aX' and v_i_a_g_r_a. P13aS.e sh1p im:medi`ate1y, 1 h@ve an><1ety and ne:ed a -bo-ner-
Y0urs s.incerel'y
S@vvy 1nvest0r
akdf k- dfks. dfk v9iew casoji ropdfk hork
aso, ckdo ofgkf opwerk- mmos odkaok s
w eofk, eoro gksod bz o-
Ran into this same problem at my company... Tested two different things out:
Mailwasher - Not a challenge/response like you asked for, but allows you to send bounces back to spam, and delete them off of the server before you donwload them. Can tie into SpamHaus and such.
ChoiceMail - Challenge response, both single user and enterprise are available. Single user sits on local machine, enterprise ties into Exchange. Can quickly add anyone in your Outlook contact list to the whitelist, and anyone you send an e-mail to can be set to be whitelisted. The challenge message can be customized. Biggest problem with the bounce (at least in my testing) is that the challenge gets rated as spam by my filters. I'm sure if the challenge was tuned up it wouldn't be that big of a problem. And they have a free trial so you can test it for 14 days
Nephilium
... is if they count the volume of "intranet" mail.
Corporations deal piles of mail on the inside, that never gets out to the genpop: HR crap, memos, meeting notices, etc. etc.
Customer relationships also generate piles of e-mail, but that should be visible to your average slashdotter who buys stuff.
I wonder if they're counting automated, machine-read e-mails such as SEC filings and other things that humans never read?
Design for Use, not Construction!
While the economics of email favor spam, spam will flourish. It's as simple as that.
To get rid of spam, we need to change the economics of email.
However, most systems proposed are too simple in that they serve to make a lot of the legitimate purposes of email too expensive, Maillists being a primary one, as well as mail from new potential customers.
Essentially, we can arrange email into a grid of Expected or Unexpected vs Desired or Undesired. We need a way to freely receive all Desired mail whether it is Expected or not, while making it expensive for mail that is both Unexpected and Undesired.
To address this, I believe a system where the promise of payment is encoded into the delivery may solve the problem. Note that the promise of payment doesn't mean that payment will be necessarily be required. However, having the promise encoded into the email does require that it be possible to place a charge on that email by the recipient. This would require verification at intermediate servers that the mail came from a known system that allows payment to be made before relaying it on.
Legit users send out so few emails that they could easily send out mails with promise of payment encoded, companies would not require the payment be made (as what a great way to lose a potential customer) so the status quo is preserved, and friends who they send mail to similarly would not bother requiring payment. Of course, if payment is required (you get into a fight with your friend) it should be a small enough amount (sub-dollar range) that it is not an extreme hardship even then.. provided you're only getting charged for one or two.
Mail-lists could be sent without the promise of payment, but since they are typically subscribed to directly, it becomes very easy to implement a white-listing solution for all the lists you're on.
Spam could not be sent using promise of payment -- if it was, the costs would quickly dwarf the profits since it is only the very low cost of email that makes spamming possible. Anybody receiving the spam would simply click the "Require Payment" button or some such, and the spammers credit card would be automatically charged the amount. Assuming only 25% of the recipients are actually able and willing to require payment, since the typical spam run sends out hundreds of thousands of email, the charges mount significantly quickly. Yet if spam was forced to not promise payment, since all legitimate email is using promise of payment, it becomes very easy to whitelist the spam out of existance.
Essentially, the promise of payment system allows unexpected but desired mail to proceed as normal, while unexpected undesired mail incurs a fee. Expected mail can use the standard email system with whitelists, or still use the promise system with no difficulties.
That Jesus Christ guy is getting some terrible lag... it took him 3 days to respawn! -NJ CoolBreeze
Nonsense. Even setting aside the obvious frauds contraband offers, unauthorized use of trademarks, etc. found in 99+% of spam, it is a violation of property rights. The First Amendment does not protect spamming any more than it protects grafitti vandalism.
At most, the law might reasonably tolerate spam if it evidences no attempt to evade filtering -- no forged headers, no "v1agra" munges, no misleading subject lines, no nothing. The use of such techniques creates a "bright line" between spamming and legitimate bulk e-mail, because it constitutes prima facie evidence of intent to intrude without permission (and, indeed, against an express prohibition).
Bottom Line: The computer-cracking laws ought to be clarified so that the evasion or spoofing of a spam filter is treated just like the evasion or spoofing of a password prompt.
/. If the government wants us to respect the law, it should set a better example.
If you were my parental unit and I ever found out that you had broadcast in a public forum the size of my sex organ, I would probably start my teen angst REALLY early.
Can anyone suggest a decent, doesn't have to be perfect, server side anti-spam filter?
/etc/access file such as: "connect:218 REJECT" will knock off about 200-5000 spams per day utilizing minimal system resources).
2 ,210,211,213,217,218,219,220,221 and you'll stop a TON of spam from a lot of foreign countries you likely never communicate with.
Don't waste your time implementing a content-based filter. The best solution is to incorporate a real-time spam relay blacklist. I recommend bl.spamcop.net. It's very effective and accurate with an extremely low legit mail blocking rate.
RBLs are great because they refuse spammer connections before the mail even gets delivered, so you don't waste bandwidth and system resources downloading spam crap and trying to interpret the contents. RBLs respect the sanctity of the e-mail message as a private communication medium and penalize those ISPs which allow spammers to operate.
If you're using Sendmail, you can also hard-code some of the IP regions where tons of spam is originating (signal-to-noise ratio for most people on the Chinese IP blocks is 0% so why allow them to hit your server in the first place? A few lines in your
Personally, if you want to get aggressive, block the following Class As: 61,80,81,82,83,142,164,193,194,195,196,200,201,20
Set up a web-based e-mail form and put a link to it in your Sendmail access configuration so that if any legit mail gets bounced, they can redirect to a web page to contact you in the [unlikely] event they were inappropriately blocked.
So, I'm thinking - those people who actually respond to spam? We should host an awards show for them - called "Too dumb to Live". We give them a chance to give their speeches and thank their whatevers, and then, when they leave the stage to go to the "press interviews", we can just dispatch them in some nice, efficient manner.
We should ALL do something to make the world a better place to live, ya know...
Seriously- if you think about it, spam may be our last hope for privacy on the net. The more legal measures we put against spammers, the more freedom we lose ourselves. So why not just accept spam as a fact of life and find some useful purpose for it, like camoflage for stego. I know there's several stego programs out there that disguise their transmissions as spam- if we get rid of the spam, no more camoflage. Don't get me wrong, I don't like getting ads for pr0n at work any more than anyone else, but I think there are other ways of dealing with it- without legally screwing ourselves in the end. (pun intended)
DISCLAIMER: This post was not checked for speling and grammar- if you complain- you're a whiner