Slashdot Mirror
Spam Volume Jumps 35% In November
gregleimbeck writes "Spam volume soared another 35% in November, an e-mail security vendor said Thursday, and the month saw spam tactics that reduced the efficiency of traditional anti-spam filters.
'There's been a huge increase in spam volume,' says David Mayer, a product manager at IronPort Systems, 'from 31 billion spams a day on average in October 2005 to 63 billion in October 2006. But in November, we saw two surges that averaged 85 billion messages a day, one from Nov. 13 to 22, the other from Nov. 26 to 28.'"
If Bush wants to regain some popularity he should consider nuking some of the spammers.
It's not going to stop. It's a multi-billion dollar industry.
If it wasnt bad enough get 10 to 15 stock "tips" via spam a day, in mid-December, i started getting the same stock spam via SMS! Yes, SMS! I got a burst of 6 one morning, then another 5 later in the day. Theres $1.10 of SMS fees courtesy of Cingular. I cancelled my SMS service (which they enable automatically) immediately. Wonder how many people are unknowing getting charged for these messages. Starting January 07, Cingular will start charging 0.15/sms -- perahps a response to record SMS revenues :-) ?
I'm using greylisting and a number of RBLs, including DUN and SpamHaus.
I see perhaps a dozen or so spams/day despite my email address being plastered all over the Intarweb for the last 6 years. (I've made no effort to hide it)
This combination stops a ridiculous percentage of all inbound email.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
I don't know of ANY reputable person or business that uses pictures to send email. For some reason email filters (either product or service) let this stuff through.
Why ?
#1. Aggressively whitelist - since I have the records of all the email received I can just send my users a list of all the email addresses that have sent mail to them and they can pick out the legitimate addresses.
#2. Block email during SMTP transmission - this is where the whitelists and blacklists come in. Everything else gets greylisted. I also use fake addresses to create my own blacklists.
If something is rejected, my phone number is included on the rejection notice. A person will see it and can call.
#3. Monitor the reject logs to see any names that may be useful (legit and fake). You'd be amazed at how many times the spammer's software trashes an address in a unique enough way that you can use it as a spam trap.
#4. Use anti-virus on anything that makes it this far.
#5. Use SpamAssassin on anything that makes it this far that is not on a whitelist.
These practices won't help so much with a personal account. But they've cut almost eliminated the spam where I work. But we don't sell over the Internet. 90%+ of our email is with the same people at the same mail servers and the same IP addresses every day.
If for example each spam message was around 1k of info, that's on average 63 tera bytes of info! Using the new Seimans 107gb speed record connection, that would take almost 10 minutes to transfer all that spam! I just wonder how much faster the internet would be without spam.
No, its not just you.
I've always preferred to run my own spam filters, I trust myself not to filter out a genuine email by mistake more than I trust my ISP, but last week the spam level got to the point where I'd go away for a couple of hours and there would be 200 new spams in my Junk folder, so I enabled the filter in my ISP's mail settings to try to get some bandwidth back. But as this article said, the latest batch seems to be evading conventional filters, so I'm still buried and thinking along whitelist lines myself (I had a whitelist system years ago, but one day found I had missed several important emails because of it).
I have noticed this as well and so have my friends and family. In fact, the number of daily spams caught and trashed by my Spam Bayes filter has nearly tripled in the last six months. The probable cause of this increase is a recent surge in the number of zombies now controlled by spam trojans in the bot networks. This was covered here on Slashdot last month in Bot Nets Behind Recent Spam Surge. As for the trusted email addresses, some of us are already doing this with whitelists, but as you say the good guys are losing right now. The one good thing, if you can call it that, that might come out of this whole scenario is that the spammers speed the coming of the day when classic e-mail is retired from general use and something better is put in its place. The greed of the spammers may ultimately prove to be their undoing as they collectively kill the goose that laid the golden eggs.
Spammers are scum. Introduce the death penalty for them - I'll gladly throw the switch, however I would argue a new extra painful method of execution should be devised just for them.
Although there are many very effective antispam techniques, some common methods are worse than the problem they are attempting to solve.
Content filters are code that effectively say "I know spam when I see it." Given that people can't say exactly what spam is, why would they trust code written by humans to do the same. Likewise, blacklists are dangerous. We have a mail list machine that hosts hundreds of thousands of subscribers. A lot of people classify any email they don't want as spam, so we occasionally get blacklisted, because a handful of people weren't expecting something (though many ISP's have whitelisted us).
We deal constantly with people who lose email because they set antispam measures as paranoid as possible (alternatively, their mail admins do this for them without their knowledge). This inevitably intercepts a certain amount of legitimate email. Then they get upset because they presume email is 100% reliable and mission critical communications are getting lost.
Only accepting mail from trusted senders is hopeless unless you already know everyone you need to communicate with. Frankly, anyone who knows everyone who needs to be in touch lives in a pretty closed world......
That's definitely one approach. Unfortunately, it means that my mail would then be at the mercy of a thousand servers' bandwidth, and that reading my mail would take a lot longer on the average as a result.
What we really need is E2EASMTP: End-to-end Authenticated SMTP. The design is basically just the existing SMTP. The only changes are as follows:
The key is that the entire abuse reporting process should be automated and that no email messages without an initial host signature should be delivered. This will make it impossible for continued operation of spam zombies in two ways:
In effect, by ensuring a trusted (albeit not necessarily encrypted) path for all email messages, you make spamming orders of magnitude harder with minimal performance impact. Best of all, I think that this could be implemented with relatively minor additions to the SMTP protocol and phased in over a period of time, ensuring a smooth transition from the spam nightmare we have now to a more modern, usable email infrastructure.
Check out my sci-fi/humor trilogy at PatriotsBooks.
someone enlighten me please!
i dont understand why there is so much spam! 90% of the spam i get, EVEN IF I WANTED TO READ IT, i dont understand it!! its just full of crappy stories, spelling mistakes and stupid stuff....
WHAT FOR??
is someone on the other side just getting pleasure in annoying people all over the world? (seems like a bofh story, or dilbert strip)
I wonder how much it would cost to outsource to India or (the irony, Nigeria) for a human spam filter. Nothing beats the human brain at pattern recognicion...
Cause you know if it only cost me 5 bucks a day to have someone else scan my spam folder for false positives it may just be worth it.
-nB
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
The image spam is the one thing that gets through my (and gmails) spam filtering. I know people are working on OCR solutions, but spammers are already actively avoiding this with all the random dots and lines you see over their stock spam images.
So what I'm wondering, and I'd be interested if anyone on Slashdot knows about or is working on this - surely it wouldn't be too hard to detect the presence of these anti-OCR techniques? The standard way seems to be putting extra lines and edges, and a spotty background to throw OCR recognition off - why not look for those signs in an image, and add to the "Spam" score if this is present?
Content-Type contains "multipart"
or Content-Type contains "text/html"
and not in address book.
What those don't catch, along with a couple filters for non-english, Thunderbirds filters do. Haven't had a false positive yet. It gets all that image spam, and before that, it caught all that HTML. That same logic working in Mail.app.
- Adam L. Beberg - The Cosm Project - http://www.mithral.com/
Something worth pointing out to people who don't want to use gmail, is that you can use gmail as an enterprise grade anti-spam filter for your personal inbox.
Simply forward all of your mail on to gmail, and then either collect it from gmail using POP3, or set gmail to forward it back to a "clean" account on your server that you can pick mail up on. You can set gmail to delete the mail after it forwards it, so you essentially get one of the best anti-spam filters out there, for free.
Of course, what is annoying me is all of the penny stock image spam that gets through most spam filters. It's getting to the point where I really am considering stripping image attachments from messages. See this post further down for a bit more on my thoughts on image spam.
We use Postgrey to filter the spams out.
It works wonderfully even without additional filtering (blacklists, for example.. Which we do still use, though).
Postgrey is a grey-list system por Postfix (for a description on how it works, click here), and there are probably other good greylist filters around.
We've had (like everyone else has) massive amounts of spam going through Spamassassin, our server was down its knees all the time.
Now the machine is typically 95-98 percent idle and the spams we receive (remember I've said we use blacklists aswell) is only the ones which come from our intranet (from hijacked machines we quickly disable when discovered).
That tool saved the day.
Eventually those bastards will have a way around it, but for now it works very well.
Combined with an idea like Hashcash (although not a direct copy), you could send a computationally-intensive hash of the message body combined with the recipient's e-mail address. When the receiver picks up the message, the client can verify the notification hash with the message hash. If they don't match, throw the message away (or notify the user, etc).
this is an excellent idea. but rather than having the registrar generate the SSL keys why not add them to the dns like in spf. this would allow the admins to generate the keys the way they want and if somehow a key is compromised (one of the mail servers gets stolen/hacked) they can quickly and easily generate a new key. also it would be valuable if you could have different keys for different servers.
Content-based spam filters can be much more accurate than humans. In particular, they can have lower false positive rates. That is, a good spam filter is less likely to discard good email than a human is to overlook good email in a sea of spam.
I'm not exactly sure how the article supports the title "It's not worth worrying about spam." Does this mean you freely distribute your email address, and you simply sort through all your messages by hand, and you've never overlooked a good email, and you have some way of knowing whether or not this is the case?
If you want to test your own ability to separate spam from good email, visit www.spamorham.org
Maybe the best solution is to stop filtering at all for a bit. Let everyone know just how bad the problem is. This was a technique used in the Usenet community every once in a while to let more people know just how much work is being done behind the scenes.
I propose that we turn off all RBLs and filters for 24 hrs the day before congress sits for the 1st time in the new year.
When the stock is worth a decent amount of money, the scammers sell and leave everyone else that bought into their so-called, "advice," with worthless stock.
:)
So what happens if I short the stock every time I get one of those damned emails?
Seven puppies were harmed during the making of this post.
The secret is that I reject all but a few hundred of those 11000 spams in SMTP envelope. Correspondents must have some form of id, currently one of:
- a valid rDNS
- a valid RFC 2822 HELO that resolves to connect IP
- an RFC 4408 sender policy (SPF) with a PASS
If you can't get one of the three right, you should fire your email admin.That gets 3/4 of the garbage. Next, SPF FAIL is rejected, including for HELO. You'd be surprised at how much spam has my own domain for the HELO! For SPF SOFTFAIL, since the sender is requesting debugging info, I send a DSN to the purported sender reporting the SOFTFAIL. For senders with no SPF, I match domains with HELO and rDNS, and look at MX to try to get a match - which is then treated like and SPF pass. For SPF neutral, I do a CBV, and blacklist the sender if it fails.
This reduces the spam from 11000 to several hundred. The content filter is auto trained. A honeypot mailbox provides spam training. Messages from (verified by SPF PASS) senders that users reply to provide ham training. Users have a web interface to the quarantine.
The false positive from content filtering is extrememly low. The biggest problem is VIP correspondents with clueless email admins who are unwilling to educate or fire them. (E.g. one admin insisted I didn't know what I was talking about and "JUPITER" was a valid HELO name...) In these cases, I have extensions to the sendmail access database to provide policy exceptions. I can also provide local SPF records for correspondents to get them a PASS.
One customer had to resort to spamsoap.com because they were getting 2 million spam connection attempts a day, and my python based filter could only process 80000 or so on his 400Mhz server.
The increase in November of 35% is pretty accurate - but where the real story is is when you look at the 6 month trend.
In July of 2006, my enterprise was blocking approximately 20 million spam messages per week. Last week, we blocked 86 million spam messages - over 400% increase in 6 months.
Most of the growth occured in September & October. We're projecting to hit 100 million per week by the end of January.
The only good news here is that the amount of valid email that we're letting into our enterprise is remaining flat, indicating that pretty much the entire increase is successfully blocked by our anti-spam. *whew*.
-Lokatana
I now scrub mail for friends and familly through my Postfix mail server using Fetchmail, Fetchyahoo and Gotmail. Amavisd-new, Clamav, Spamassassin, various DNS blacklists includung URIDNSBL and a sprinkle of bayesian filtering have pretty much solved the problem as far as I'm concerned. The only remaining annoyance was image spam, but that has even been solved thanks to FuzzyOCR that is now in Debian !
I you still have spam, it just means that you are not using the freely available tools to eradicate it. Just do it ! I found it is suprisingly easy and we have to thank Debian for that !