Slashdot Mirror
AOL Blocks 2 Billion Spam/Day
T_moz writes "According to this article
AOL has blocked over two billion (2000000000) SPAM emails in one day!"
This figure is 70-80% of all mail incoming to AOL users. Utterly insane. Unfortunately, all this blocking means spammers will just send more mail to make
up for it until a real solution is found.
No wonder I can't get any help with my nigerian bank account problems!
Wow, the filters are sooo good they blocked all the comments to this story.
Execute a spammer. It's clean, it's quick, and it's efficient. Desperate times call for desperate measures.
See if people will keep sending unsolicited email then. Matt Groening had it right with Futurama.
Computer: "You've got mail!"
Leela: (Groans)
Computer: "It's not spam!"
Leela: Ohhh
"It takes many nails to build a crib, but one screw to fill it."
Now, if I could only stop these assholes who send me unwanted CD-Rom's to my home 3 times a month...
I'll wager that a fairly significant portion of that blocked mail is wanted by the recipients. I know that we get many calls when our AOL recipients don't recieve their expected daily/weekly newsletters.
The only ``intuitive'' interface is the nipple. After that, it's all learned.
Funny how nobody ever mentions the false positive and false negative rates in these stories.
.sig
If AOL has a false positive rate of 0.01%,
That means over 200,000 incorrectly blocked emails per day.
If they have a false negative rate of 1%,
That means over 20,000,000 spams got through.
2 billion sounds like a big number, but it's still only 10-30 spams for the typical AOL user.
-- this is not a
Simple rule of thumb:
1 spam = 1 bps.
11 million spams = 11Mbps or less than a 1/3 of a T3.
Even if they weren't using relays to multiply the bandwidth, it's doable.
-- this is not a
2 Billion emails divided by 180 spammers equals approx 11 millions emails per spammer per day *just to AOL alone*.
The word I hear from a reliable source is that to do spamming as a viable business, you must be sending at least 10 million spams per day.
So if the low end of the bell curve is circa 10m, it's easy to believe that AOL's share of that can peak at an average of 11m per major spammer. It would make sense for spammers to focus on AOL users, both because there's a lot of 'em in one place and because they are, uh, less sophisticated than your average internet user.
I can't compete with 2bil., but here's my spam blockage for a measly 80 users on Sunday the 27th:
Postfix log summaries for Apr 27
Grand Totals
------------
messages
2454 received
185 delivered
183 forwarded
1 deferred (17 deferrals)
0 bounced
2359 rejected (92%)
0 reject warnings
0 held
0 discarded
3102k bytes received
3162k bytes delivered
152 senders
98 sending hosts/domains
39 recipients
2 recipient hosts/domains
:wq!
do
iptables -A FORWARD -j DENY -s ${i} -p tcp --destination-port 25
done
Click here or here.
There is the graph they have on the wall in one of their Dulles offices that shows how the filters are working. It's scary, when a new type of spam filter is put out, AOL mail traffic decreases about 60%. The graph line plummets. Then, you watch it creep and spike until barely a month, maybe even a couple of weeks later, it's back up again. The spammers have found another way around it. People joke and laugh about AOL and spam, but AOL is really serious about getting rid of it. It costs them uncountless piles of money just to keep spam from breaking down their walls.
I have also attended some pretty heavy security conferences about spamming for ISP folks. It's not just a mail flood technique anymore. Spammers are not just some freak in China with an ISP who looks the other way, some spammers are actually crackers. Crackers who break through an ISP's security, just to get around mail filters, or relay it from within. Some of the spam you get is not just because the ISP didn't filter it, it's sometimes because some cracker found a new way to bypass the filter, a back door to the ISP's internal services, so they send it in, even relaying spam from personal accounts. These are not script kiddies doing this, there are bonafide hacking geniuses working as spammers.
Spam can shut down an ISP, and AOL knows that all too well.
Most email that appears to come from AOL in fact comes from somewhere else. Same for all the big ISPs like yahoo, msn, hotmail, and so on. Not only do spammers forge the From: headers, they are also forging the SMTP envelope MAIL FROM as well.
Actually we were inadvertently relaying undeliverable spam back to AOL customers and found ourselves blacklisted by AOL until we cleared it up. No, this is not an "open relay" problem; this was an "undeliverable bouncing" problem. But the effect was similar. You really need to be careful because spammers are getting very smart.
What was happening was that mail which got through our SMTP gateway (running sendmail) and into our back end internal email server (running Exchange) was being bounced as being undeliverable because of the made up recipient addresses that spammers use. The problem was Exchange was creating these "bounces" as NEW email messages rather than as an SMTP DSN rejection, mearly prepending "Undeliverable:" to the subject and sending the message to the supposed sender. But those forged senders turned out to be real AOL user accounts, and being AOL users they flagged our bounces as being spam, and poof, after about 15,000 in one day we got blacklisted....actually I can't blame AOL at all.
The AOL postmasters were surprisingly helpful and courteous in helping us resolve this. What I now do is to take the connecting IP address and do a reverse DNS lookup. If it is not from within the aol.com or aol.net domains, it is rejected as being forged (regardless of what the headers or even the envelope say). Likewise I also check the responce on the HELO/EHLO greeting to make sure it is also from aol.com. And just as an extra check, I finally configured our sendmail milter interface to use LDAP to the exchange backend server to reject mail for invalid mailboxes before it is ever passed through to our backend server.
Now if there were reliable was to detect forged mail from the other big ISP players. I can only perform those forgery catching tricks with them because AOL has a policy that ALL outbound mail from AOL will ALWAYS be sent from an SMTP server registered within the aol.com DNS domain. I don't know if that is necessarily true for the other big ISPs.
There seems to be a solution to the spam problem - but one that is not backwards compatible.
:), the hash function could be controlled by the server which would require the sender to sign using a function of higher complexity when loads are higher.
I have seen this solution posted as a comment to some story in the past - so the credit is not mine, but of some comment writer I do not recall.
The idea is to create a complicated and expensive hashing algorithm that costs quite a few cycles - and use it as a "signature" for each mail's content, including the from and to addresses.
This would mean that sending mail could require a few seconds and be cpu-bound instead of network bound, but this is almost nothing for the average mail user. The spammer, however, would be required to calculate the hashes of the hundreds of thousands of mails he is sending - which could be a costly calculation.
Perhaps, (and this is my idea
Perhaps (another idea of mine), users could signify as part of their email addresses - the complexity of the hash function required to send them mail, or at least know what complexity of a hash function was used when sending them mail.
This could allow users to reject mails that weren't at least a bit costly for the sender to send, thereby making spam too costly to practically send.
White lists can also be used by users to save their friends from the trouble of calculating a hash of their mails - but this is probably unnecessary as it should only take a few seconds at most.
Ofcourse verifying the mail's hash should be trivial, no matter the complexity of the hash function - and mails with unmatching hashes would simply be thrown away immediately.
Tagged Message Delivery Agent (http://www.tmda.net/).
For mail coming in, the user maintains a "whitelist" of accepted sender addresses. Unknown senders get a confirmation request that says, "Thanks for your mail, please reply or click here to verify you're a legitimate sender".
For mail sent out, the user's mail gets tagged automatically so the recipient can reply and the reply will be accepted automatically.
TMDA is GPL licenced, and it works with all the popular MTAs (Postfix, Exim, Sendmail, etc).
My friend Luis is a mail admin for AOL in Dulles (it's funny, I gave him his first Linux CD a couple years ago...). He runs a server off his Comcast cable modem, and has had to remove himself and me from the block list a few times, due to entire IP ranges being blocked (he does this by adding exception rules). He says AOL spends 20-30 million dollars a year paying for servers, storage, bandwidth, technicians, etc. related to spam. He himself works on the block lists. When you think about the distributed nature of the internet, all that spam is eating EVERYONE's bandwidth. Tends to piss me off, and I'm glad these big guys might be getting slapped around a bit soon.
"The Tree of Liberty must be refreshed from time to time with the blood of Patriots and Tyrants." --Thomas Jefferson
What's your point? For a while now it has been pretty standard fare that the only way to have reliable outbound SMTP traffic is to smarthost it to your ISP's official mail server. There are just too many cable and DSL connections out there that can be hijacked. Also, many ISPs block outbound port 25 traffic, and lots of ISPs require that inbound SMTP traffic come from hosts that have forward and reverse DNS mapping.
What is the possible advantage of not smarthosting to your ISP SMTP server? Seems to me that you will encounter problems with many other ISPs besides AOL, and it can only bring headache...
-Steve
Hey, Windows users, there is no such thing as "forward" slash, there is only slash and backslash.
Same thing here. I know legitimate email from my server is part of their 2bn figure. AOL may block 2 billion emails a day, but that includes a larger number of false positives than ever in light of their cable/dsl blockage months ago.
I can't even receive from AOL now as they've landed on a RBL I reference. Not because they're blocking cablemodems (which is their choice), but because their implimentation violates the SMTP RFC. The RBL blocks non-compliant servers, confirmed open relays and smtp agents confirmed vulnerable to exploit (via correlation between version # and security advisory).
AOL's mail server sends a 550 and disconnects you the instant you connect. 220 and 554 are the only allowed responses at that point, and immediate disconnection is not permitted; The server must wait for the client to send a QUIT before closing the connection.
Since you're disconnected immediately, this behaviour also indirectly violates the requirement that the server always accept e-mail for postmaster.
According to this post on March 5, AOL canned a billion spams. Today, two months later, they canned two billion. In four more months, they will have canned more than one spam for every single human being on earth. Is that fascinating or just a little fucked up?
While I'm sure this has been done long before I came up with it, i'm going to still tell my story.. When I first got to college a little under 3 years ago I started getting hit with spam. So, I wrote a script that sent a challenge/response email to the user who sent me mail. All of my friends/family/ebay bidders responded, never a spammer. My name is dknj, I've been spam free for 3 years.
-dk
Post your email address here.
People will assist you making that experience. Then let us know what solution you came up with.
i know the feeling
/etc/postfix/transport to route aol.com mail thru my isp, but it's stupid i have to do that (and i understand aol's view on it, it's the spammers abuse that caused this). /And/ i didn't even get an alert about this, my girlfriend just said she didn't get that email when i mentioned it and i had to check my logs. go.com is even worse, they just close connections without giving a little reason why.
550-The IP address you are using to connect to AOL is either open to
550-the free relaying of e-mail, is serving as an open proxy, or is a
550-dynamic (residential) IP address. AOL cannot accept further e-mail
550-transactions from your server until either your server is closed to
550-free relaying/proxy, or your ISP removes your IP address from their
550-list of dynamic IP addresses. For additional information,
550-please visit http://postmaster.info.aol.com.
550 Goodbye
And yes I just added the line to my
Because I'm on a "dynamic IP" I'm blocked as spam. My IP hasn't changed in over a year, and my server does *NOT* allow open relaying. Thanks AOL, you're really helpful.
~Anztac