Slashdot Mirror
Some Anti-Spam Vendors Blocking and Slowing Gmail
fiorenza writes "Google's Gmail (and corporate mail) are being throttled and sometimes blocked by some anti-spam services, including MessageLabs and Antigen. Ars Technica reports that the blocking is a result of the Google CAPTCHA crack, which has allowed a deluge of spam from Gmail's clusters. Most users won't get blocked mail, but Ars confirmed with MessageLabs that Gmail delivery delays are to be expected."
I definitely agree with you, if a mail server accepts my mail with a 200 code, then the mail *should* be delivered. Even if its put in someone's spam folder, the message should get there. That's one of my pet peeves. That being said, from my experiences when setting up my mail server, Gmail was probably one of the best about not blocking legit mail (I've had an SPF record since the beginning though). I had lots of problems with Hotmail, and I think my mail was usually marked as spam by Yahoo until I enabled DKIM signing. With SPF records and DKIM, I don't think I have any major problems (though my mail server handles a pitifully small amount of mail, so its not like we're going to get marked as a bulk sender).
Every time you post an article on Slashdot, I kill a server. Think of the servers!
Just to add something, the problem with 5xx replies is filter is *before* queue so some mail may be delayed and servers need to be contacted a few times before they get a delivery slot. For example, say gmail can filter 1 million messages at a time. That means 1 million open connections. So, if you are connection 1,000,040 you get 4xx response - temporary failure due to no available resources. So try again later.
/dev/nulling spam while accepting it with 2xx code is like burning unopened envelope at post office because it was typed instead of handwritten indicating possible spam.
This is not a problem, really. You can wait a few days until you can deliver the message as long as it is *delivered* eventually.
Pre-queue filter with only 1 unique IP connection at a time to mail server. Problem solved.
Huge email servers get reasonably constant and predictable amount of mail per day and per hour and even per minute. They can plan pre-queue filtering with some margin for any spikes. And if there is a huge bomb and your mail doesn't get there for 7 days and your server gives up, hey, at least you get a "Could not deliver the message because destination was not available". Much better than "err, never got any mail from you" from the destination party.
and you certainly haven't used Citadel, which trumps Exchange by a wide margin.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
What they need to do is have a process for detecting when an account is spamming.
Now, you and I would just say "when an account is sending 10,000 messages a day" and that would be correct for about 99.9% of the cases.
I'd also recommend Google "seeding" the spammers databases with "spamtraps" (not tied to Gmail or Google in any way). If an account sends email to a spamtrap, that account is frozen.
And so forth.
I'm forced to use Notes every day.
Exchange is -years- ahead of notes.
Website Hosting
I think the safest thing they can do right now is return to their invitation only registration in an effort to close the breach. Then they have to start deleting spam accounts quickly before the spammers adapt to inviting themselves. If they are lucky they will be able to delete spam accounts faster than they multiply.
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
Our company uses Messagelabs. Just tried a quick message from my Gmail account. Almost immediately received the message. No delay for my account, at any rate.
I remember reading a post about one poor hapless admin, who had come across an exchange server that was eating 10 GB of HD space a day, couldn't figure out what was causing the massive use of disk space, his company was in the middle of their most critical time of year, and he had 3 days left before the server crashed again and he'd be out of a job if he lost 12 of so days (since the last backup of that servers files) of e-mail.. it was an old post, and the people who had ideas were ideas the admin had already tried.
I'm fairly sure that the shit hit the fan and he took the blame, and i can't imagine a single reason why anything other than poorly designed malware, or a really rare hard to reproduce bug could be eating 10 GB of disc space a day...
If it had been recent i would have suggested he find a tool to let him add an external raid array for the OS to keep eating the 10GB a day until he had the problem locked down... but, it was too late for my advice...
https://www.gnu.org/philosophy/free-sw.html
Wait? Exchange 2003/2007 Recovery Storage Group? Maybe it's time your Exchange got an upgrade.
Since Exchange 2003 SP2 I haven't seen Exchange Database corrupt itself and I deal with servers running 100-200 users on single servers. These servers have had RAID drives fail, Power pulled from them and users do some really idiotic stuff. Databases always came back ok.
You could have really nice LInux server for Exchange money, but you would also have something with a bunch of half baked software that didn't have nice Desktop Client, didn't support your blackberry like it should and is asking to break when you update one piece of software.
The IPs doing this shit are the end user addresses for home and office computers that are no different than all the other end users that use Gmail. They could block an IP, but eventually that IP will be used by someone else who is a legitimate and secure Gmail user. They are better off closing accounts that send spam. But Google isn't doing that (based on having seen spam from the very same user I reported to them as a spammer 2 weeks prior). If they do decide to pursue the user of the IP, once they get past the legal roadblocks of getting the identity out of the ISP (while doing this for 100,000 such IPs at the same time), all they get is some stupid loser who has an infected Windows box being used as part of the botnet. They can get this machine cleaned up, but they aren't anywhere near the real culprits.
What Google needs to do is segregate all users that are new since the crack (they know when it was, because they can see a spike in new user signups from random end user IP addresses). In the mean time, close down direct signups and fallback to the invite system only allowing the old users to send invitations. Re-engineer the CAPTCHA system to at least temporarily thwart the signups before bringing that back online. For all new users, run all their outgoing mail through the same filters that are used for incoming mail. Mail that can't be sent, put it in a new folder type for "blocked outgoing". The user has to pass a new CAPTCHA per each message to do a re-deliver around the filtering (or just rephrase and send a new one). And limit the number of these to 3 per day (although this may not do much good since the botnet may only be doing this much or less over a million accounts).
now we need to go OSS in diesel cars
Expect to see a technological solution, this isn't a company full of middle managers or people who are used to losing technical battles.
If I were a betting man I'd say Google will either A) release a new authentication/authorization scheme for creating new accounts, or B) they'll evolve their current system to be resistant to delivering false negatives on bot provided responses.
Because honestly, isn't this just graphical/visual acuity based Turing test that needs to be treated as "passed" by the industry? The reasoning being: the equivalent of Alicebot now exists for the graphical world, so the test needs to be re-engineered to test another (currently) unpassed Turing style evaluation.
Based on that realization: the whole reason capcha's are stupid is that if you keep the existing design but try and make it "harder" to break, the designer of the Bot need only account for that change and not an entire redesign.
All this sounds like a great technical challenge: think up a new Turing test... When in reality those posting go back to invite only are absolutely right but it's likely we won't see that come out of Google.
My 2 Cents... Show the user 5 images. Your job is to 1) Select the one image that is out of place with a radio button 2)Solve the captcha that is one of the 5 images 3) Choose which word best describes the remaining 3 images from a drop down/combo.
..1....2......3.......4......5 (Captcha image)
How this would work
Cat Cat Money Cat "Peaches"
Drop down choices (Housewive, Gutter, Salsa, Fruit, Cat)
Answer: 1-(Image3-Money) 2-(Peaches) 3-(Cat)
Of course this would only be reasonable for something one time only like signing up for gmail.