Slashdot Mirror
Earthlink Deploying Challenge-Response Anti-Spam System
deliasee writes "The Washington Post reports that Earthlink is preparing to offer new spam filter technology that requires sender authentication. AOL is still concerned that such technologies will put too much burden on consumers." The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers...
I was hoping more ISPs would adopt the challenge-response system, like MailBlocks, previously featured on Slashdot. Way to go Earthlink! If I was interested in dialup, this would be a big selling point for me. I'm still waiting for a service that offers the challenge-response feature of MailBlocks but allows me to forward to my existing provider. I mean, a 12MB inbox is pretty lame. There are free providers that can give me that much space...
As a network admin, many of the remote users I support (sales reps, on-the-road types) use Earthlink dial-up while travelling. At times, some of the program's that Earthlink has used to stop people from using their services to spam have make my job harder. However, I do not begrudge Eartlink for these inconviences, at least they, as a major ISP, are doing *something* about this problem.
My two cents,
-- RLJ
Of course it is no good if the spammers can set up automated systems to respond to the challenge. There are only two ways around this:
- Make the challenge 'AI-complete', that is, to give a correct answer you must be a thinking human being and not a computer. But then how can the other end check that the answer is correct? Having humans generate a fixed number of questions and provide sample answers also isn't going to work, since spammers will learn the correct answers. You need a way to generate an unlimited number of questions and to mark the answers automatically, and clearly this can't be done if the questions are intended to be too hard for a computer.
- Make the response computationally burdensome, so a computer can do it but only at the cost of some CPU power (so large bulk mailings would be impractical). This is what Hash Cash and similar systems suggest.
It looks like Earthlink's system will rely on sending pictures you have to look at. Apart from the practical problems of clogging the wires with image files, I worry about OCR potential. The examples of this stuff I've seen on Yahoo, where you have to type in a number shown in a partially 'obscured' image, wouldn't have been difficult to develop OCR software for if you were so minded.
There's also the question of the spammer taking the challenge and sending it out to some other user. That user, by now used to replying to challenges from Earthlink and other addresses, will respond to the question and send the correct answer back to the spammer. D'oh!
-- Ed Avis ed@membled.com
> How do two people with challenge and response communicate?
:)
> If the challenge always gets thrugh, then the spammer will just issue cahllenges as spam.
> If they don't get through, then you would have a nasty mail loop.
In TMDA (a challenge response system in python) at least, when you send a email to somebody, they don't get a challenge when they answer. It's logical because if you send him an email, you know he will not spam you
So i assume earthlink system will act the same.
Spammers seem to be sending a whole bunch of crap from my address (ed@membled.com) even now. At least, I keep seeing what appear to be genuine delivery failure notifications of Russian spam sent from my address. Any system which trusts individual email addresses, without relying on some real authentication such as PGP signatures, is broken.
A simple rule is: Headers can be forged. Don't trust anything in the headers for antispam purposes. This includes the sender and recipient.
-- Ed Avis ed@membled.com
One idea: Any emails you send out, the recpt is automaticly added to the "ok, let through" list.
Slashdot is like Playboy: I read it for the articles
I don't know about earthlink but ticketmaster's sys uses random different patterns obscuring the text. As for the text, the fonts they use vary, size varies, lines are not straight, and most of the fonts look like they are hand written (with even a single letter appearing differently in the same image)
I'd guess there system is pretty effective.
I assume that the challenge-response is intended for messages already tagged as potential spam. In other words, low-scoring messages (spam-wise) wouldn't get the challenge. I certainly wouldn't expect a perfectly not-spam message to require the CR. Earthlink's (and other) spam-rating systems are pretty good, I think using it for the 'grey-area' emails would work well. And block the obvious spam without hesitation.
One question: shouldn't it be REALLY OBVIOUS to ISPs what is spam and what isn't? It seems that if a nearly-identical message gets sent to a large enough percentage of their users, it's clearly spam. Is this hard to do? Are spammers clever enough to distribute emails to avoid this?
Earthlink offers DSL and cable. I'm using it right now.
I am definitely in favor of a little pain up front in increased traffic from challenge-response to get the spam boys off the net.
I suspect that when the spammers stop sucking up so much bandwidth, net speeds will increase for everyone--including dial up users.
Remember when 14.4K was fast? So do I. And I think with a correction in the system, it can be a decent speed.
Laws are for people with no friends.
A residential broadband customer mailing through his ISP's mail server is whitelisted (most stuff from that server is nonspam). An rr.com luzer with an open proxy is tarpitted into oblivion (everything else in 24.0.0.0/8 is spam). Yes, Joe Linux running (non-relaying) Sendmail on his Linux box is also tarpitted, but he's not trying to send a million mails a day. So he's not hurtin'.
I can see a scaling problem in that you'd have to run some sort of adaptive filtering process on the receiving end, which might be prohibitive CPU-wise. OTOH, if you only scanned 1% of all inbound mails for "spamminess", you'd still rapidly figure out that for a US ISP, 24.0.0.0/8 is an ocean of spam with a few islands of real email, and 200.0.0.0/7 is a shitstorm of spam. You don't need to analyze every inbound mail - you only need a statistically-valid sampling of the inbound mail queue to figure out which netblocks are teh sux0r.
Having it be adaptive would be cool - because a South American ISP (which probably has less of a problem with 200.0.0.0/7 than, say, Earthlink does, because they have legitimate users emailing each other from within those netblocks). So an ISP in .mx would end up with a different set of teergrubing weights. They might end up letting most of 200.0.0.0/7 in, only tarpitting the worst /24s, and teergrubing all 24.0.0.0/8 because so few of their users get anything but spam from rr.com netblocks.
Think of it as combining the best part of SPEWS (naughty netblocks are noticed semi-automatically), without as much collateral damage (if you're an ISP, a 10 second delay to anyone emailing one of your customers from a naughty netblock will never be noticed, but it'll *kill* some dirtball trying to spam to 10000 of your users through an open proxy.)
That's not true. There is an approach where you show a "proof of computational effort"; that is, your computer spends 10 or so seconds computing the response to a challenge. Here's a paper on the subject.
So does this mean that if you're blind, you don't get to send mail to C/R users? Another hurdle for blind users is just what the net needs.
This sig is not the Zahir. Lucky for you.
Perl gurus, start your editors!
How many lines will it take to write a script to automatically reply to challanges? As long as the messages have predictable structure, you should be able to write a parser to pick out the word or picture they want, then throw it back.
College kids: Are you bored, broke, and of weak moral fiber? You too can make money while sitting on your ass by replying to email challanges for the princely sum of 3 cents per message! Combine the first suggestion with the second, and you've got yourself a money machine.
It's great to see an ISP take some decisive steps, but this scheme has weaknesses. Interesting to see how it goes. Despite the concerns, I'm cautiously optimistic.
As a twist, it would be interesting to see how that anti-spam vs. spam lawsuit with the copyrighted haiku goes (don't recall the parties names, but it's gotten coverage here). Maybe something similar could be combined with the challange-response system to make it illegal to respond to the challange under false pretenses. Raises a few slippery-slope legal issues that if you're going to touch, you might as well criminalize spam outright (which would be fine, of course).
"These people look deep within my soul and assign me a number based on the order in which I joined" --Homer re:
I agree. It's so simple yet so effective. It really makes me wonder why people invest time and money in silly, less-friendly and potentially less-effective solutions such as C/R.
it seems to rate the spam based on its content, which no spammer can get around.
They're starting to try. When they start breaking up words so that "cock" is "c.o.c.k" they're making an effort to avoid filters, but also are addressing the Bayesian filters since that will normally get broken up into 4 tokens, one for each letter. Of course, if they do it enough then a single token "c" might actually become a commomn characteristic of spam for that user.
Anyway, Bayesian works great now. I think spammers will evolve to deal with it, but all that is necessary is to implement new token-identifying logic in the Bayesian filter... the Bayesian approach itself is very solid.
It's a hell of a lot faster to do than actually placing calls to people and talking to them, and people
I agree. I suspect you will see spammers actually analyzing the C/R responses. If it's something the software has seen before and is capable of responding automatically, it will. Those that it can't will be forwarded to someone to quickly deal with it. If some of the megaspammers make as much as they supposedly do, hiring a teenage kid at $6/hr to spend the day answering C/R responses is not a huge investment.
Once the spammers are obliged to label their stuff "bulk", half the battle is won. Then they start collecting a "white list" of legitimate mailing list sources, and label every bulk message not on it as "suspected spam" and dump it in a separate folder.
If the challenge is based on an image ("please respond with the fuzzy word in the subject line" or somesuch), where does that leave vision impaired email users? How do they respond to a challenge to get their email delivered?