Slashdot Mirror
Earthlink Deploying Challenge-Response Anti-Spam System
deliasee writes "The Washington Post reports that Earthlink is preparing to offer new spam filter technology that requires sender authentication. AOL is still concerned that such technologies will put too much burden on consumers." The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers...
every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers
Not exactly right. It happens only for the first time to detect whether the sender is legitimate or not. Quote the article:
The system automatically recognizes future e-mails from the same sender, so the verification needs only to be performed once.
The problem with this system is that the spammer can still spam using legitimate e-mail accounts as a camouflage (or expired e-mail accounts). Once the legitimate e-mail address is procured, the spam still goes on. It is futile, IMHO.
--
Error 500: Internal sig error
Ha! I can just see it... Alice@me.com send and e-mail to Bob@you.com. Bob@ send a challenge to Alice. Alice, never having heard from Bob, send a challenge back to Bob. Either Bob ignores the second e-mail, or sends another challence. Of course, if the e-mail software allows any outgoing e-mail address to reply without challenge, this wouldn't be a problem.
Moderation: Put your hand inside the puppet head!
the article implies that an image would be part of the response, such as ticketmaster's please type the word in the picture into the box.
Squirrel Mail
SpamAssassin Config for Squirrel Mail <- Register Globals must be turned on in php.ini to use this.
Now, that being said, I run an ISP in St. Louis, and spam is a problem, but for the precise reason mentioned on the submission, I can't use a challenge-response system. The reason is that our support staff equals myself plus 1. If I want to answer phone calls all day from people complaining about not being able to get mail from their daily spamming of mailing lists, I best allow all. The problem is that these same people complain about all the spam they get...ugh. The above solution is elegant and leaves the ability to control the filter to the end user via webmail. If they don't like it, set the threshold high and it's 'off'. Been using this for months without a complaint.
Now if you don't use lists, and it's for your own mail server...go for it. That has to be the most effective method available, but not appropriate for wide scale use.
Karma: Chameleon (mostly due to the fact that you come and go).
From the article:
So if earthlink people are on your mailing list, you'll get a challenge next time you send it out. It should only happen once, and from then on, you're email addy is "legit".
It's not like you get 9000000 challenges from everyone on the list. But if every ISP did it, you'd get a challenge from every ISP on the list.
This is the first step towards email being such a pain in the ass, that people just no longer bother using it.
Kiss SMTP and POP3 goodbye.
I don't need no instructions to know how to rock!!!!
The article clearly states that the user turns this on or off. So it seems unlikely that a large number of challenges will start going out. As far as Grandma is concerned, you can add her email address to the OK list yourself so that she never sees a challenge. The only minor problem I see is receiving email from text only people, (Pine, etc..), or portable devices that might not render the bitmap correctly. But it seems a minor complaint, really.
Dave Williams
Jeez people, read the whole article, it's not that long:
The challenge-response system will be optional and free for EarthLink subscribers, Anderson said. It will allow users to automatically clear the e-mail addresses of friends, family members and other associates in their electronic address books, so those people would not receive the challenge e-mail.
That's called a "white list"-- a list of addresses you know are legitimate.
When someone responds to a challenge and you accept their response, they go on your whitelist.
When you turn on this gadget, add your mailing list addresses to your white list. If you suddenly stop getting a list, go find out if they changed their sending address and add it to your white list.
If that's too much of a burden, feel free not to use the service, and go back to complaining about spam.
How do two people with challenge and response communicate?
My C/R setup (TMDA) automatically put anyone I send email to on my whitelist; therefore I'd get their challenge message.
NO CARRIER
But spammers have found ways to defeat them and spam accounts for 40 percent of all e-mail
Is this true?
Of all my email accounts, the only one I ever get spam on is my yahoo account, which I set up pretty much to get spam on, since any websites I visit that require registration, I always give them the "spam" address I got for free. I don't even check that email for anything. Human beings are the only recipients of my paid email addresses. I am for measures like this though, because even though I'm not affected directly by spam, increased traffic on the net is bad for everyone.
We need to punish the sensless posting of one's own email address to anonymous sources. These are the same people that give out their address and phone numbers when they buy batteries from radio shack. Use your head, they don't want to know where you live so they can send you a case of scotch. They want to drink your beer, crash on your couch, sleep with your daughter, and have you pay them for the privelege.
Waiting for ad.doubleclick.net...
Take a look at this
War is necrophilia.
-Esme
Alice@me.com sends an email to Bob@you.com
Mailing program adds "Bob@you.com" to Alice's list of valid emails (after all, you're not often going to send email to somebody that you don't want responding, right?).
Bob@you.com sends a challenge to Alice@me.com
Alice@me.com accepts the challenge, since she already sent the original email to "Bob" and had him added as an authorized user
Alice authenticates to Bob's system, and all is good
Another way would be to make all "challenge" type emails follow a specific pattern - with little to no allowance for anything other than the challenge. Then, challenges will be accepted as legit without bouncing back-and-forth, and spammers cannot simply send a message as a challenge with extra spamcrap attached - and still cannot send non-challenging email.
Now, an ignorant spammer could send a flood of challenges just to be annoying, but this isn't very profitable as they wouldn't be able to contain penis/viagara/etc ads.
In theory, someone could send me a spamlike message and would have to reply to the autoresponder. In theory, a spammer could validate himself. In practice, those two things almost never happen. The system catches about 150 spams a day and over 90% of its autoreplies immediately bounce. Last time I analyzed it, only about 2% of my legitimate correspondents had hit the autoresponder (note, that's a fraction of a percent of my total legitimate email, since a given correspondent only has to validate once.)
I have yet to see a notification from Amazon, my bank, or other similar email trip the filter. Haven't had any of my correspondents complain yet, but I have had a couple of them ask how they can set up the same thing for themselves.
So if it's implemented carefully, I think this could be a big win for Earthlink subscribers and more or less invisible to everyone who communicates with them.
"What if I'm registering at eBay or PayPal or some other site which sends an automatically-generated email when I complete the first step?"
That's a good point, but the solution is simple: throw-away addresses.
If you are an earthlink subscriber, you get an email address like nanogator@earthlink.net. (Hey, that useta be my address!) Then, Earthlink could provide a service where you create a unique address that expires after x amount of time. so nanogator.dkaf3fj39@earthlink.net becomes active, and that's the one you use. From there, you can add them to your whitelist.
It's a bit round-about, but that's the beauty of Earthlink. They're a major ISP. Surely places like Ebay will have to stand up to comply with the upcoming standard. It'll never happen if some people don't have issues like this.
"Derp de derp."
First it is important to note that the challenge system at Mailblocks is not something that can be automatically replied to. Much like the signup verifications for many forum systems out there the Mailblocks challenge email is simply a link to a web site. On that web site is a dynamically generated .gif of a number. The image is formatted in such a way so as to make it difficult for screen scrapers to write an algorithm which can decipher the numbers in the image (multiple fonts, different colors, background noise). If ever a spammer figured out how to programatically decipher the image then Mailblocks simply has to rework their image generation system and stay one step ahead of the spammers.
Next you have throw away addresses. Maiblocks calls these trackers. When you create a tracker a number and short ID are appended to the end of your username. This email address is then immune to the challenge response and can either be delivered to a purpose built folder or directly to your inbox. So if you wanted to have an address to get receipts from you simply make a tracker named say [username]+receipts4325@mailblocks.com. Then any email to this address can be delivered to the +receipts folder in your inbox. If you start getting spam at that address you just delete the address and create [username]+receipts5563@mailblocks.com and start giving this out. It can be a little bit of work to maintain your trackers but compared to deleting 20-30+ spam mails from my accounts each day it's well worth it.
When an email is successfully delivered to your main address the originating address is entered into your address book including the reason why this address was validated (completed puzzle, user added). Mailblocks also adds the address of any outgoing mail you write to your address book so that responses can be properly delivered without challenge. Finally, if you are expecting something to appear in your email that doesn't the 'pending' folder holds all email that hasn't been validated for a certain amount of time before deleting. If you really want to you can go back and dig through the email there to find the one you want, validate it, and it will be delivered to your inbox. If something gets validated you don't want simply go to your address book and either delete it or check 'do not deliver mail from this address'. Viola. Also of interest is the fact that Mailblocks can provide the same security to any other mail account you have. It can check POP3, IMAP, accept forwards, and even screen scrape web mail to bring all of your mail to a central location. When it does it provides the same callenge-response capability through these other accounts.
Who moderates the meta-moderators?
ROFLMAO.
"teergrube" - German word for "tarpit".
Teergrubing FAQ
Teergrubing is a good idea, but it dates back from the days when open relays, not open proxies, were sending the emails. One spammer (with dialup) would hit you from one relay (with broadband) from the spammer's own (dialup) connection, and the goal was to slow down the open relay so that the open relay wouldn't be able to spew as many emails. Eventually, the admin of the open relay would wonder why his outbound queue was so huge, or why Sendmail fell over and died because /var/spool got full, and secure his server. In the old environment (spammer has narrowband, must hunt down broadband by finding open relays to steal from), one teergrube could "fix" one open relay at best, and at worst, would at least prevent delivery of several hundred thousand spams.
Doesn't really work as well in a world with millions of open broadband proxies. The spammer no longer cares if any individual open proxy hits a teergrube, because there's plenty more bandwidth where that came from. (And because open proxy luzers tend to be clueless twits, they're less likely to notice even if their machine crashes.) In today's environment (plenty of bandwidth on both the spammer's end, and plenty of proxies to steal bandwidth from), teergrubing in its original form is somewhat less effective.
I've recently implemented my own Bayesian system on my server. While my first-cut was very CPU intensive, very straight-forward techniques can be made to make it extremely CPU-friendly. In fact, I'll bet my current Bayesian system is less CPU-intensive than a simple keyword-filter that has 5000 "keywords" in its database.
I don't use SpamAssassin and can't comment on its toll on the CPU, but there is no inherent reason why a Bayesian system can't be deployed by ISPs. About the only drawback I see is that you have to store a corpus for each user and that ends up being between 1MB and 2MB per user. But disk space is cheap...
Here's the internal description of the service, which, by the way, is always going to be optional -- users have to turn it on manually. So fears of mass confusion from users when Earthlink turns this system on are a bit unfounded.
This is what the automated reply looks like:
And finally a more detailed description they supply:
Then, if you added a dozen more equally clever features, and a nifty web interface availible, you would have TMDA
:)
He either comes off as a real interesting guy with encyclopedic knowledge,or a pathological liar with an ax to grind
He either comes off as a real interesting guy with encyclopedic knowledge,or a pathological liar with an ax to grind
Email addresses are forgable. The from / reply-to fields are NOT TRUSTWORTHY - they are effectively USELESS for ANTISPAM purposes. Once an effective whitelist system is in place that relies on from, we'll see spam that works like Klez.
The only way to effectively defend against SPAM is at the IP level - via MX from DNS.
Hotmail, yahoo, free mail clients etc. are all doing a good job of policing themselves. If they can't police themselves, then punt the server. The spamboxen which increase the scale of spam that can be sent are the real problem.
The other important thing to do is to TAG the messages that aren't on the whitelist rather than deleting them, so the user can still find them.
Is this harder to use than current mail? I say NO because the amount of spam that people have to deal with is now so bad that the costs of dealing with managing the list is less than the cost of managing the spam.
But half the poseurs/posters here don't even understand how whitelisting or SMTP work before they go blathering off about 'throw out SMTP' or 'I won't get my f*cking mailing list'
I've been using TMDA (http://www.tmda.net) for well over a year now, had maybe five or six spam emails sneak through the system in that entire time. Twice a day it sends me a list of "pending" emails so I can manually release and/or whitelist a message.
Challenge/response systems DO work, and they work extremely well. I think those who have not used one should give it a try before throwing rocks.
Eagles may soar, but weasels don't get sucked into jet engines.