Slashdot Mirror
Spam Volume Jumps 35% In November
gregleimbeck writes "Spam volume soared another 35% in November, an e-mail security vendor said Thursday, and the month saw spam tactics that reduced the efficiency of traditional anti-spam filters.
'There's been a huge increase in spam volume,' says David Mayer, a product manager at IronPort Systems, 'from 31 billion spams a day on average in October 2005 to 63 billion in October 2006. But in November, we saw two surges that averaged 85 billion messages a day, one from Nov. 13 to 22, the other from Nov. 26 to 28.'"
Maybe it's just me, but my spam volume seems to have jumped at least 200% in recent months.
Are we finally going to reach a point where only trusted addresses can email us? Seems the arms race is being severely lost. I've got a pretty good spamassassin config and I can't keep up anymore, I find myself having to manually delete literally hundreds of messages a day now.
http://www.babysmasher.com
http://www.openingbands.com
It appears to me that the increase is almost all due to a small number of messages swamping us. One advertises the stock symbol PHYA and has no link. The scam is that if you Google for that symbol, there will be a full-width paid ad for a fake broker/analyst site. About 10% of my email for the last couple of weeks (i.e. over 100 of 1000 spams/day) advertises this stock symbol.
If it wasnt bad enough get 10 to 15 stock "tips" via spam a day, in mid-December, i started getting the same stock spam via SMS! Yes, SMS! I got a burst of 6 one morning, then another 5 later in the day. Theres $1.10 of SMS fees courtesy of Cingular. I cancelled my SMS service (which they enable automatically) immediately. Wonder how many people are unknowing getting charged for these messages. Starting January 07, Cingular will start charging 0.15/sms -- perahps a response to record SMS revenues :-) ?
I use (amongst other thing) spamhilator. It's free, and its pretty reliable. The trouble I have is that I *have* to allow everyone to mail me. When you run a business, you *do* occasionally get people guessing your email address from your domain and sending you a potentially vital email. I just can't afford to block emails by default. And anything (like captchas or auto-response systems) that makes it hard for my customers to contact me is just BAD.
I don't see why we are always fighting this problem at the reception end, rather than the source. Spam filters can work quite well, but why are they mostly applied right at the very endpoint of the chain?
I'd be very happy for some basic filtering to take place on my outgoing mail at the ISP level. If it meant the odd automatic email with a captcha saying "are you sure you intended to send this mail?" before a spammy-looking email went out, thats fine with me, and wouldn't that approach cut down on all those twits whose PC's are part of a botnet without them realising it?
Bah, why is firefox suddenly getting me to spell check in American?
DRM-free indie games for the PC and Mac: Positech Games
"Two years from now, spam will be solved" - Bill Gates
Did you ever notice that *nix doesn't even cover Linux?
I'm no MS fan, but I have to admit, a quick bit of maths show that Outlook gets over 95% of my spam. Gotta hand it to them.
Man wird am besten für seine Tugenden bestraft.
Apparently, image spam beats a number of spam filters. But not all. Try another filter. I haven't done an extensive test in the post-image-spam era but OSBF-Lua is the best available filter I know of, and I haven't noticed that it is compromised by image spam. It is free.
Well I'd just switch to a white list of e-mail addresses and everything else be damned! Captcha based filtering for application to join my white list if I wanted it too.
Shh.
The great irony of the spam arms race is that the better we get at filtering the spam, the more garbage the spammers send out just to get the same return. You can't stop filtering it, because the mail you want would be buried in a torrent of spam. But filtering more just raises the bar for the next round of spam.
Eventually it may get to the point where (a) email is unusable or (b) spammers have to send such a massive volume of cr@p that it no longer becomes a cheap business, and it ceases to be worth spamming. Until then, things will keep escalating.
#1. Aggressively whitelist - since I have the records of all the email received I can just send my users a list of all the email addresses that have sent mail to them and they can pick out the legitimate addresses.
#2. Block email during SMTP transmission - this is where the whitelists and blacklists come in. Everything else gets greylisted. I also use fake addresses to create my own blacklists.
If something is rejected, my phone number is included on the rejection notice. A person will see it and can call.
#3. Monitor the reject logs to see any names that may be useful (legit and fake). You'd be amazed at how many times the spammer's software trashes an address in a unique enough way that you can use it as a spam trap.
#4. Use anti-virus on anything that makes it this far.
#5. Use SpamAssassin on anything that makes it this far that is not on a whitelist.
These practices won't help so much with a personal account. But they've cut almost eliminated the spam where I work. But we don't sell over the Internet. 90%+ of our email is with the same people at the same mail servers and the same IP addresses every day.
Spam has become such a problem where I work that it has completely flooded the corporate Internet connection. I personally feel they should host an external mail server and spam filter off-site someplace. For my personal server I use various RBLs and country blacklists, like blocking all of China, Korea, Russia, Nigeria and a few other countries. Those seem to block most of the spam from even entering my mail server.
I know people talk about legal solutions not working, but I think if law enforcement made use of existing laws and went after these people it might make a difference. I'd love to see the FTC go after the pump and dump spammers and confiscate everything they own before locking them up, or the food and drug administration go after all the enhancement pill spammers. Also, perhaps a law to fine idiots who buy from these spammers.
Just change the federal law to let some of the state laws take effect, i.e. defeat the Can-spam act.
I think if law enforcement made a good effort to go after these spammers and lock them up then it might make a difference.
-Aaron
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
If for example each spam message was around 1k of info, that's on average 63 tera bytes of info! Using the new Seimans 107gb speed record connection, that would take almost 10 minutes to transfer all that spam! I just wonder how much faster the internet would be without spam.
It gets through for two reasons:
1. It's harder to extract useful data from an image than from text or a markup language like HTML. OCR is possible, but wasn't worth the effort until the volume jumped up recently.
2. Without that meaningful data, it looks a lot like messages that people forward each other. A picture sent from a cell phone, for instance, or the latest funny animation, or pictures from last week's party, or whatever. The filter is left with header info and not much else.
Filters aren't just acting on spam vs. business mail -- they're also acting on spam vs. personal mail.
And that's why the US Treasury announced a surplus, from all the fines collected from all that spam violating the CAN-SPAM Act. We're funding free WiFi for every American, while exterminating all the spammers!
--
make install -not war
He's got 9 days left!
"I am the king of the Romans, and am superior to rules of grammar!"
-Sigismund, Holy Roman Emperor (1368-1437)
If you don't do business outside the US, filtering by geographic registration for the subnet works wonders. A little hard to set up but once you use the geographically filtered email to train your Bayesian filter, you really get 99.9% or better. Currently getting approx 99.97% accuracy and very little false positives. Pleased as punch.
Spammers are scum. Introduce the death penalty for them - I'll gladly throw the switch, however I would argue a new extra painful method of execution should be devised just for them.
Although there are many very effective antispam techniques, some common methods are worse than the problem they are attempting to solve.
Content filters are code that effectively say "I know spam when I see it." Given that people can't say exactly what spam is, why would they trust code written by humans to do the same. Likewise, blacklists are dangerous. We have a mail list machine that hosts hundreds of thousands of subscribers. A lot of people classify any email they don't want as spam, so we occasionally get blacklisted, because a handful of people weren't expecting something (though many ISP's have whitelisted us).
We deal constantly with people who lose email because they set antispam measures as paranoid as possible (alternatively, their mail admins do this for them without their knowledge). This inevitably intercepts a certain amount of legitimate email. Then they get upset because they presume email is 100% reliable and mission critical communications are getting lost.
Only accepting mail from trusted senders is hopeless unless you already know everyone you need to communicate with. Frankly, anyone who knows everyone who needs to be in touch lives in a pretty closed world......
I'm sure that it'll go back down to normal levels real soon now. Why heck, it may even withdraw from the Internets.
--
My God! It's full of tubes!
"You'll get nothing, and you'll like it!"
Wow... Yeah, umm, wow.... What more can one say?
Anti-MS zealot: "The increase in spam is caused by Vista".
MS Fanboy: "Don't be silly - it was obviously the 2.6.18 kernel release that did it".
IT Professional: STFU, both of you.
Fuzzy OCR for Spam Assassin. It does a pretty great job on it.
No need. As I've been saying for several years, only servers really need to have a cert. If every server had a cert and no messages from machines without a cert were accepted, spammers would have to have a cert or would have to send through normal channels through people's ISPs.If they get a cert, you know who and where they are and you can arrest them.
If they don't get a cert and their spam bots go through people's ISPs, you can set up an automated "this is spam" reply mechanism that would stop the spam bots at the source much faster than existing measures, thus making the amount of effort needed to maintain zombie botnets orders of magnitude greater because they'd be going offline after sending a much smaller number of messages and would be affected by email message rate throttling at the ISPs.
Either way, spam becomes much, much harder....
Check out my sci-fi/humor trilogy at PatriotsBooks.
The image spam is the one thing that gets through my (and gmails) spam filtering. I know people are working on OCR solutions, but spammers are already actively avoiding this with all the random dots and lines you see over their stock spam images.
So what I'm wondering, and I'd be interested if anyone on Slashdot knows about or is working on this - surely it wouldn't be too hard to detect the presence of these anti-OCR techniques? The standard way seems to be putting extra lines and edges, and a spotty background to throw OCR recognition off - why not look for those signs in an image, and add to the "Spam" score if this is present?
Content-Type contains "multipart"
or Content-Type contains "text/html"
and not in address book.
What those don't catch, along with a couple filters for non-english, Thunderbirds filters do. Haven't had a false positive yet. It gets all that image spam, and before that, it caught all that HTML. That same logic working in Mail.app.
- Adam L. Beberg - The Cosm Project - http://www.mithral.com/
Something worth pointing out to people who don't want to use gmail, is that you can use gmail as an enterprise grade anti-spam filter for your personal inbox.
Simply forward all of your mail on to gmail, and then either collect it from gmail using POP3, or set gmail to forward it back to a "clean" account on your server that you can pick mail up on. You can set gmail to delete the mail after it forwards it, so you essentially get one of the best anti-spam filters out there, for free.
Of course, what is annoying me is all of the penny stock image spam that gets through most spam filters. It's getting to the point where I really am considering stripping image attachments from messages. See this post further down for a bit more on my thoughts on image spam.
We use Postgrey to filter the spams out.
It works wonderfully even without additional filtering (blacklists, for example.. Which we do still use, though).
Postgrey is a grey-list system por Postfix (for a description on how it works, click here), and there are probably other good greylist filters around.
We've had (like everyone else has) massive amounts of spam going through Spamassassin, our server was down its knees all the time.
Now the machine is typically 95-98 percent idle and the spams we receive (remember I've said we use blacklists aswell) is only the ones which come from our intranet (from hijacked machines we quickly disable when discovered).
That tool saved the day.
Eventually those bastards will have a way around it, but for now it works very well.
The thought of the idiots who receive the junk and buy the crap advertised in it.
~Philly
He's got 9 days left!
Nine days ought to be enough for anybody.
Just pulling numbers out of my ass... but let's say that one in a million people is dumb enough to fall for the crap they're trying to sell, and actually falls for what they're doing. Let's say it's your typical buy/dump scheme where they buy up, say, 50,000 shares of some penny stock. Net cost to them, $500 for the stock, and, let's be really generous and say $100 to send a million e-mails. Realistically, it doesn't cost them nearly that much to do it, but that's beside the point....
The idea is that they'll create a run on the penny stock. Create some demand on a stock that's worth $0.01 a share, even a little, and it might go up to $0.02/share. Not a significant jump, except when you consider that they could have $50,000 invested in the company already. That run would turn into $50,000 profit overnight. And that's assuming a relatively small one in a million people being dumb enough to fall for it. People in general are a hell of a lot stupider than that.
And here's the rub... it's not illegal to create a run on your stock like that. It's not fraud, it's not stock manipulating, it's not deceptive marketing. The company whose stock is being traded usually has absolutely nothing to do with the scheme. And thanks to overly relaxed laws in countries like China and the USA when it comes to bulk e-mailing, it's not illegal to send the spam. They word it in such a way that it looks, to an idiot, like they've received an e-mail they aren't supposed to have received, talking about some sure-fire hot stock, and enough people will fall for it that you're able to turn a profit.
Spam in general is like that. They don't care that 99.999% of the messages they send out get ignored. They care that 0.001% arrive in the inboxes of the criminally stupid.
If you believe everything you read, you'd better not read. - Japanese proverb
Content-based spam filters can be much more accurate than humans. In particular, they can have lower false positive rates. That is, a good spam filter is less likely to discard good email than a human is to overlook good email in a sea of spam.
I'm not exactly sure how the article supports the title "It's not worth worrying about spam." Does this mean you freely distribute your email address, and you simply sort through all your messages by hand, and you've never overlooked a good email, and you have some way of knowing whether or not this is the case?
If you want to test your own ability to separate spam from good email, visit www.spamorham.org
Parent does not understand grandparent. The Google ad points to a stock market manipulator, not PHYA.
Maybe the best solution is to stop filtering at all for a bit. Let everyone know just how bad the problem is. This was a technique used in the Usenet community every once in a while to let more people know just how much work is being done behind the scenes.
I propose that we turn off all RBLs and filters for 24 hrs the day before congress sits for the 1st time in the new year.
The real disease is: those vast botnets. Really, it's a scary thought. We are lucky that they only being used for spam and the usual phishing scams and the like - as far as we know! Imagine if the terrorists buy themselves some botnets for some nefarious purpose, or the Chinese or North Korea government corner the market on them to run millions of bots to steal corporate secrets or IDs or who knows what? What I'm saying here is that the large increase in spam should be triggering off alarm bells everywhere. The spam is not the problem - it's the botnets. Why in the world don't responsible world governments unite to put a swift end to this problem? Really - it could be dealt with swiftly and effectively in a hundred different ways that I will up to the imagination of the reader. I am just astonished this hasn't happened. I mean - couldn't our friend and champion of democracy George W. include this in his initiative against terrorism? He would probably have more luck tackling this problem then he is having in Iraq. What if he put that on his agenda - and set loose all his military might along with the help of some coalition of the willing? Perhaps he could salvage what's left of his image? Are you listening Mister Bush?
http://www.magma.ca/~gtaylor/AudioTestFileGen.htmI'm writing this from my chateau in France. I flew here earlier today from my horse farm in Virginia in my new Gulfstream. Can't believe my good luck: couple of months ago, I discovered this unsolicited stock tip in my email. The stock was cheap and the tip seemed pretty solid, so I invested my life savings in it. And my grandmother's life savings, too; I have her power-of-attorney. The next day, I got nervous. Remembered the old line about if it seems to good to be true, it probably is. So I decided to unload the stock. Damned if the price hadn't gone up 6000 percent! In one day! Incredible! Anyway, I sold it all ... and here I am. Grandma's taking a round-the-world tour in her Gulfstream -- we bought a matched pair.
An important feature that is used by the spammers to verify that the email has been sent and read is external
images, if you completely block those they cannot use the servers statistics/unique session id to figure out
which mails worked or didnt.
2. Use those remote image location to flood their session stats and pollute their databases and tell their ISPs to drop them too.
Liberty freedom are no1, not dicks in suits.
The secret is that I reject all but a few hundred of those 11000 spams in SMTP envelope. Correspondents must have some form of id, currently one of:
- a valid rDNS
- a valid RFC 2822 HELO that resolves to connect IP
- an RFC 4408 sender policy (SPF) with a PASS
If you can't get one of the three right, you should fire your email admin.That gets 3/4 of the garbage. Next, SPF FAIL is rejected, including for HELO. You'd be surprised at how much spam has my own domain for the HELO! For SPF SOFTFAIL, since the sender is requesting debugging info, I send a DSN to the purported sender reporting the SOFTFAIL. For senders with no SPF, I match domains with HELO and rDNS, and look at MX to try to get a match - which is then treated like and SPF pass. For SPF neutral, I do a CBV, and blacklist the sender if it fails.
This reduces the spam from 11000 to several hundred. The content filter is auto trained. A honeypot mailbox provides spam training. Messages from (verified by SPF PASS) senders that users reply to provide ham training. Users have a web interface to the quarantine.
The false positive from content filtering is extrememly low. The biggest problem is VIP correspondents with clueless email admins who are unwilling to educate or fire them. (E.g. one admin insisted I didn't know what I was talking about and "JUPITER" was a valid HELO name...) In these cases, I have extensions to the sendmail access database to provide policy exceptions. I can also provide local SPF records for correspondents to get them a PASS.
One customer had to resort to spamsoap.com because they were getting 2 million spam connection attempts a day, and my python based filter could only process 80000 or so on his 400Mhz server.
The increase in November of 35% is pretty accurate - but where the real story is is when you look at the 6 month trend.
In July of 2006, my enterprise was blocking approximately 20 million spam messages per week. Last week, we blocked 86 million spam messages - over 400% increase in 6 months.
Most of the growth occured in September & October. We're projecting to hit 100 million per week by the end of January.
The only good news here is that the amount of valid email that we're letting into our enterprise is remaining flat, indicating that pretty much the entire increase is successfully blocked by our anti-spam. *whew*.
-Lokatana
I now scrub mail for friends and familly through my Postfix mail server using Fetchmail, Fetchyahoo and Gotmail. Amavisd-new, Clamav, Spamassassin, various DNS blacklists includung URIDNSBL and a sprinkle of bayesian filtering have pretty much solved the problem as far as I'm concerned. The only remaining annoyance was image spam, but that has even been solved thanks to FuzzyOCR that is now in Debian !
I you still have spam, it just means that you are not using the freely available tools to eradicate it. Just do it ! I found it is suprisingly easy and we have to thank Debian for that !