Slashdot Mirror
Earthlink Deploying Challenge-Response Anti-Spam System
deliasee writes "The Washington Post reports that Earthlink is preparing to offer new spam filter technology that requires sender authentication. AOL is still concerned that such technologies will put too much burden on consumers." The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers...
I was hoping more ISPs would adopt the challenge-response system, like MailBlocks, previously featured on Slashdot. Way to go Earthlink! If I was interested in dialup, this would be a big selling point for me. I'm still waiting for a service that offers the challenge-response feature of MailBlocks but allows me to forward to my existing provider. I mean, a 12MB inbox is pretty lame. There are free providers that can give me that much space...
On one hand it (Earthlink's new "technology") seems reasonable enough to the every-day-joe. I'm sure that the majority of Earthlink subscribers don't utilize news or mailing lists, and don't bother paying their bills online. For these people, it's fine. On the other hand, many others use online banking and other such automated tools (even account control mechanisms for online games will be affected). How quickly will all of these vendors conform to Earthlink's new technology and make the needed changes in their automated systems? Will Earthlink simply render many of these domains exempt?
The answer to solving SPAM resides in the current mechanisms used for the actual transmission and delivery, the mechanisms that all participants must use, not just Earthlink. This is of course the mail servers themselves.
How do two people with challenge and response communicate?
If the challenge always gets thrugh, then the spammer will just issue cahllenges as spam.
If they don't get through, then you would have a nasty mail loop.
If I have nothing to hide, don't search me
I think forged headers are the calamity of the inprocess SMTP transfer mechanism. If we can liberate the dynamic IPs saturated on the IPlanet web matrix, then we could perform 3-way LDAP POP3 authentication with a digital certificate.
The other way this could be accomplished is to triangulate a 801.11b WAP source into an array of POSIX message headers that would reflect the consistency of the mail protocol.
What do you think?
I think this will create way too much hassle. There are some people who wouldn't mind, but others (such as grandma) who have to be told three times where the power switch is won't really know what is going on. At least now when I don't reply I'll have a decent excuse... "but grandma, you forget to send it twice, so i didn't get it"
Seriously, what are they thinking? TMDA might seem like a nice idea in theory, in practice, it's a pain to use and not exactly safe either. Once this gets widescale usage, the spammers will simply start responding to the challenges (after all, it's not like that couldn't be easily automated).
It will be interesting to see how well this method works now that it is going to be out there for mainstream non-geeks to use. I am a little curious about how the address will work for order confirmation, the article seems to hint at throw-away type address but doesn't give much detail.
This seems like it might be a good step, but it's missing the point. The only thing that will truly curb spam is to rework the SMTP protocol to not implicitly trust every host, as was mentioned in an earlier /. article.
If earthlink looked for mailing list headers or signs that the message is a mailing list they could allow it through... at least for awhile to avoid the challenge responses to mailing lists...
ugh.
As a network admin, many of the remote users I support (sales reps, on-the-road types) use Earthlink dial-up while travelling. At times, some of the program's that Earthlink has used to stop people from using their services to spam have make my job harder. However, I do not begrudge Eartlink for these inconviences, at least they, as a major ISP, are doing *something* about this problem.
My two cents,
-- RLJ
every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers
Not exactly right. It happens only for the first time to detect whether the sender is legitimate or not. Quote the article:
The system automatically recognizes future e-mails from the same sender, so the verification needs only to be performed once.
The problem with this system is that the spammer can still spam using legitimate e-mail accounts as a camouflage (or expired e-mail accounts). Once the legitimate e-mail address is procured, the spam still goes on. It is futile, IMHO.
--
Error 500: Internal sig error
Does anyone know of any open source challenge-response anti-spam projects similar to what Earthlink is developing? I've wanted something like this for a long time. While I don't have time to start a project myself, I'd like to contribute to someone else's.
Does this automatically allow messages from people you've sent email to?
I'd hate to think that there are two messaging systems sending challenges out to each other before they let the other one's challenge through.
Ha! I can just see it... Alice@me.com send and e-mail to Bob@you.com. Bob@ send a challenge to Alice. Alice, never having heard from Bob, send a challenge back to Bob. Either Bob ignores the second e-mail, or sends another challence. Of course, if the e-mail software allows any outgoing e-mail address to reply without challenge, this wouldn't be a problem.
Moderation: Put your hand inside the puppet head!
I'm not convinced whether it'll actually work, but I'm willing to give it a chance. The SPAM problem is obviously getting way out of hand. It's sort of like evolution -- if the system works, then it'll become more widespread. If it doesn't work, well that's the nature of evolution isn't it?
Some experts see problems with the technology and doubt that consumers will warm to a process that adds another step to e-mail delivery
I don't really agree with the article's assumption here. It's true that it's another step, but it's one-time-only, which makes it much more palatable in my opinion.
the article implies that an image would be part of the response, such as ticketmaster's please type the word in the picture into the box.
than just blindly blocking mail comming from small sites using dynamic DNS.
I use my filters this way:
upon recieving move all messages to folder spam
unless message is from "email@address.com"
if message in folder spam is older than 10 days move to folder trash
Each time someone I know sends me an email I add their address. Very rarely do I get new addresses once all of mine are set up. When they do, I add another address.
It takes a while to set up, but I don't have to depend on my ISP, and I can switch with no problem.
Squirrel Mail
SpamAssassin Config for Squirrel Mail <- Register Globals must be turned on in php.ini to use this.
Now, that being said, I run an ISP in St. Louis, and spam is a problem, but for the precise reason mentioned on the submission, I can't use a challenge-response system. The reason is that our support staff equals myself plus 1. If I want to answer phone calls all day from people complaining about not being able to get mail from their daily spamming of mailing lists, I best allow all. The problem is that these same people complain about all the spam they get...ugh. The above solution is elegant and leaves the ability to control the filter to the end user via webmail. If they don't like it, set the threshold high and it's 'off'. Been using this for months without a complaint.
Now if you don't use lists, and it's for your own mail server...go for it. That has to be the most effective method available, but not appropriate for wide scale use.
Karma: Chameleon (mostly due to the fact that you come and go).
me@challenge.earthlink.com
something like that. So that it allows users to gradually changeover to the system. That would allow them to be more extreme in their refusal to accept emails and much less compromising.
I like it.
And in day 2, spammers automate the responses.
Results:
1. Spammers get free AUTOMATED account verification.
2. The load on the email system doubles.
Conclusion:
Nice "solution" dumbass.
- Adam L. Beberg - The Cosm Project - http://www.mithral.com/
It's been about a year since I was an Earthlink customer, but they had Brightmail implemented and it was blocking 95+% with no false positives. I had gotten so confident in it that I never even bothered to log in to the web site to check the caught spam. Has that system gotten worse? It seems like a challenge response system will put even more of a burden on their network with incoming spam being the same, but now you add all the authentification requests, replies etc.
Of course it is no good if the spammers can set up automated systems to respond to the challenge. There are only two ways around this:
- Make the challenge 'AI-complete', that is, to give a correct answer you must be a thinking human being and not a computer. But then how can the other end check that the answer is correct? Having humans generate a fixed number of questions and provide sample answers also isn't going to work, since spammers will learn the correct answers. You need a way to generate an unlimited number of questions and to mark the answers automatically, and clearly this can't be done if the questions are intended to be too hard for a computer.
- Make the response computationally burdensome, so a computer can do it but only at the cost of some CPU power (so large bulk mailings would be impractical). This is what Hash Cash and similar systems suggest.
It looks like Earthlink's system will rely on sending pictures you have to look at. Apart from the practical problems of clogging the wires with image files, I worry about OCR potential. The examples of this stuff I've seen on Yahoo, where you have to type in a number shown in a partially 'obscured' image, wouldn't have been difficult to develop OCR software for if you were so minded.
There's also the question of the spammer taking the challenge and sending it out to some other user. That user, by now used to replying to challenges from Earthlink and other addresses, will respond to the question and send the correct answer back to the spammer. D'oh!
-- Ed Avis ed@membled.com
What will it do with mailing lists?
They won't accept return emails, so they will never get the challenge?
I won't know what email address they are coming from until I get one, so how could I manually add an address to accept?
Like vacation messages?
Maybe spammers will just submit "verfication" messages instead of actual messages.
I can't wait to see the piles of accumulated cruft on earthlinks servers.
"...the spam client MUST provide a Accept-Topics: header, where the value is one of 'penis-enlargment', 'make-money-fast', 'repair-credit', or 'any'. The server MUST reply with a Spam-Type: header, specifying the type of spam transferred. In addition, the server MUST respond with a Spam-Encoding: header, where the value is one of the options 'all-caps', 'many-exclamation-points', or 'broken-english'..."
While it seems obvious that something needs to be done to slow down the spammers, I dont think this would be the best way.
One of the great things about email is that it is fast, I send a message and it arrives almost instantly. However, this system would remove alot of this advantage.
Now i might be wrong here, but as far as I can see, this attempts to solve the problem by requiring users to send two messages instead of one. Not only will this greatly slow down the speed with which one can send a message, which is probably part of the point, but it will also increase bandwidth traffic. Also, you can bet that the spammers will find some way to get around these turing tests.
This is a good start but I am concerned that it will only increase bandwidth unecessarily.
Let's make a difference
What if I'm registering at eBay or PayPal or some other site which sends an automatically-generated email when I complete the first step? What if I subscribe to a mailing list where I can't get a response from a human to a challenge? What if I'm applying for a job online and the company sends me an email saying they've received my resume, which I will not be able to see?
I think this kind of scheme is only useful when the message sender is human and you know who they are, in which case the system is pointless anyway. What I think we need is to phase in a new, secure version of SMTP where emails aren't relayed unless the sender's ID can be verified.
the coolest club on
So when a spammer fires a few hundred or thousand emails to an ISP, they will sit on the mailserver waiting for him to respond.
Since the from address is faked, that same ISP will launch an acknowledgement flood against a third user.
Excellent.
I just see so many tricky things that someone somewhere will screw up.
I think that might even work out very nicely perhaps with a little notice at the top of the message with instructions to add the address to the allowed list (perhaps a link) or deny further messages from the address
AC comments get piped to
Every spam-subject
What would be so painful if all email content was simply a web link to the sender's server, their "outbox". When the receiver went to read it, they could store a copy then if they wanted mobility. Or, their email client could follow these links automatically when given the notice.
The differentiation between a content link and a malicious one would be a delicate but solveable problem.
However, since no transmission is until demand, we're not shipping terebytes of crap around the wires for naught. Thats the real issue here. Spammer's email content must be served to the receivers as they open the email. Since spoofing would be akin to removing the content, nobody could get a message across without it.
I know I've read about a formalized version of this idea here. Somebody post it again.
mug
What happens when the customer orders something from Amazon - the purchase confirmation email comes from a non-human address.
Just the other day I got an email from a company that I ordered software from describing a free upgrade that I could download. It came from donotreply@[host].com, meaning, if I was using Earthlink's system I probably wouldn't have received it.
The problem with Challenge - Response is that it makes the assumption that if there's not a human behind the email that it's spam. In practice, there are many legit emails that are not individually sent by a human.
There is no longer anything that can be done with computers that is nontrivial and clearly legal. -- Paul Phillips
Sending often 50 mails a day (business conversations with cooperants, mailing lists, friend communications,...) I really hate the idea. I must say it will be easier for spammers to employ character recognizing software than for me to reply to all those confirmations.
The problem is somewhere else and there is solution. The real problem with spam is to force senders to identify themselves correctly (if they identify, they can be easily filtered, maybe including databases of the spam senders being just the lists). And the solution is to require the email to be digitally signed so one can verify it against the sender public key.
The challenge-response thing is a great idea for yet-unknown senders. However, users should be able to have a white list that doesn't require a challenge. Using that, they could sent out an email/insert into paper bill statement that would give users information on where to grab a quick self-installing tool for their platform/email client that would allow 1 click additions to white lists... (or just add it to their Web Mail interface) Then, earthlink would give users 2 more months of spam (while they build their white list and such) before turning on the challenge-response system. Another idea, is take email that isn't obvious spam yet fails the challenge-response system and put it in a Junk folder of some sort where users can 1 click white list the sender... So Timmy goes, 'Hey, where's my starwars newsletter?' and Timmy checks his junk, finds the starwars newsletter and in 1 click sends it to his inbox and white lists newsletter@starwars.com or whatever. Of course, if a month or two goes by and you haven't pulled an item from the junk folder, it's assumed you don't care and it gets deleted. And yet another solution is to have earthlink build it's own white list of responsible, trusted senders (such as rhn-admin@rhn.redhat.com and such) so that users will only have to check that junk folder if it's either A) a sender that misbehaves or B) a sender that earthlink hasn't heard of yet. And to that matter... could always add a sender rating so that if enough people put a certain email address (rhn-admin@rhn.redhat.com) on their whitelist, earthlink would then either add it automatically or give some admin the task of checking out that the sender is really cool and then adding them to the earthlink wide white list. Anywho, that's just my 0.02USD
DONT PANIC
Just do the preemptive thing and remove all earthlink subscribers from any mailing list you admin.
Protocols like this are bad, especially when people like earthlink are the masterminds.
How small a thought it takes to fill a whole life
The answer is not attaching more bad ideas to an already bad protocol. The ultimate answer is in the protocol designers. A government/state can pass as many laws governing the interaction of people/things with the bad protocols, but the IETF/IEEE will still create them, and certify them. People should just wake up and realize that SMTP is to blame for this big mess. ISP's should stop offering SMTP outright, and think of ways to replace it. Chat programs are probably a better way to pass messages anyways. SMTP has become a massive bazaar that is full over everyone on earth, and since it is completely open, its also completely ok to send bulk mail. Forging headers is another issue, but simply spewing email is intrinsically allowed by the protocol, and thus taken advantage of. If everyone one on earth had a computer, and everyone on earth sent email to everyone else on earth every day, would that be spam? No, because it would cross the line into accepted practice, and that is what we are starting to see due to the sheer bulk of spam sent to everyone on a daily basis. The point is that as long as SMTP exists, so will spam. The answer is to replace SMTP with something that doesn't allow spam to exist by removing the ability to anonymously send people messages.
It isn't a lie if you belive it.
Because no OCR routines have ever been written, this is absolutely foolproof.
Even so, you only have to respond once, and you then have the full run of earthlink. So you spend a day responding to challenges from all the ISPs, then go back to business as usual.
I don't need no instructions to know how to rock!!!!
Ironically, AOL is delaying email from Earthlink members...seems a little funny that they might complain about positive efforts to control spam...
Members may see delays in mail being received by AOL members
If this were easy, they wouldn't need us to do it!
I had an Earthlink (Mindspring) dial-up account for quite a while.
I never gave out the address that was earthlink's (jester2@mindspring.com). However, I got tons of SPAM to that address. Seems earthlink is trying to play both sides of the fence. They want to lure customers with anti-spam feature, but they are still going to sell your address.
The article clearly states that the user turns this on or off. So it seems unlikely that a large number of challenges will start going out. As far as Grandma is concerned, you can add her email address to the OK list yourself so that she never sees a challenge. The only minor problem I see is receiving email from text only people, (Pine, etc..), or portable devices that might not render the bitmap correctly. But it seems a minor complaint, really.
Dave Williams
the article implies that an image would be part of the response, such as ticketmaster's please type the word in the picture into the box.
I give it about a month before someone figures out a way to use something similar to OCR technology to bypass this sort of thing. If this sort of challenge/response idea becomes very wide spread, the spammers will suddenly have a huge need to find a way around it, and they have the money to throw at it. It will eventually fail, just like every other filter out there. SPAM is here to stay, the best we can do is fight it constantly, and never respond to it, but even still we will never win.
Necessity is the mother of invention.
Laziness is the father.
Jeez people, read the whole article, it's not that long:
The challenge-response system will be optional and free for EarthLink subscribers, Anderson said. It will allow users to automatically clear the e-mail addresses of friends, family members and other associates in their electronic address books, so those people would not receive the challenge e-mail.
That's called a "white list"-- a list of addresses you know are legitimate.
When someone responds to a challenge and you accept their response, they go on your whitelist.
When you turn on this gadget, add your mailing list addresses to your white list. If you suddenly stop getting a list, go find out if they changed their sending address and add it to your white list.
If that's too much of a burden, feel free not to use the service, and go back to complaining about spam.
I see a slew of people saying "blah blah blah, they'll automate the response blah blah blah". And apparently, to alot of you, this is all new.
This is something that's been around for a few years and gee, spammers haven't gotten around it yet. C/R antispam systems work because spammers don't use valid Reply-to: or To: addresses.
If they did and the spam gets through the system, then great! There's one more point where we can nail them on when/if we go to hunt them down. Oh, you used your dialup with an SMTP server to auto-respond to the challenge (which is probably alot of work for the average evil spammer), great, email abuse@isp and have his account shutdown.
Since I have started using ASK to C/R my email. -zero- spams have gotten in my Inbox (which is what annoyed me the most about spam, the false positive I got when the little sound would ring telling me I had new mail.)
Intrusive? PLEASE! How lazy are you? Hit reply -once- and you'll never have to see it again when sending email to me. I'd say getting pelted with 200 spams a day is slightly more intrusive to me than what you're going to have to do to send an email to me.
I just wasted your mod points! HA!
But spammers have found ways to defeat them and spam accounts for 40 percent of all e-mail
Is this true?
Of all my email accounts, the only one I ever get spam on is my yahoo account, which I set up pretty much to get spam on, since any websites I visit that require registration, I always give them the "spam" address I got for free. I don't even check that email for anything. Human beings are the only recipients of my paid email addresses. I am for measures like this though, because even though I'm not affected directly by spam, increased traffic on the net is bad for everyone.
We need to punish the sensless posting of one's own email address to anonymous sources. These are the same people that give out their address and phone numbers when they buy batteries from radio shack. Use your head, they don't want to know where you live so they can send you a case of scotch. They want to drink your beer, crash on your couch, sleep with your daughter, and have you pay them for the privelege.
Waiting for ad.doubleclick.net...
Take a look at this
War is necrophilia.
I've just implemented a POP3 email checker that makes sure the FROM address is valid. It removes about 25 spams per day (out of 100) and MailWasher takes care of the rest.
If anyone is interested in trying out my program, drop me a line.
No sharp objects, I'm a programmer!
I use Earthlink, and they already have a decent spam-filtering system. I still use both SpamProbe and SpamAssassin, and the combination of all three works well enough that I'm not afraid to give my real address just about anywhere.
Well, except maybe Slashdot.
But perhaps with the new system, I can post it even here!
I don't know about earthlink but ticketmaster's sys uses random different patterns obscuring the text. As for the text, the fonts they use vary, size varies, lines are not straight, and most of the fonts look like they are hand written (with even a single letter appearing differently in the same image)
I'd guess there system is pretty effective.
Alice@me.com sends an email to Bob@you.com
Mailing program adds "Bob@you.com" to Alice's list of valid emails (after all, you're not often going to send email to somebody that you don't want responding, right?).
Bob@you.com sends a challenge to Alice@me.com
Alice@me.com accepts the challenge, since she already sent the original email to "Bob" and had him added as an authorized user
Alice authenticates to Bob's system, and all is good
Another way would be to make all "challenge" type emails follow a specific pattern - with little to no allowance for anything other than the challenge. Then, challenges will be accepted as legit without bouncing back-and-forth, and spammers cannot simply send a message as a challenge with extra spamcrap attached - and still cannot send non-challenging email.
Now, an ignorant spammer could send a flood of challenges just to be annoying, but this isn't very profitable as they wouldn't be able to contain penis/viagara/etc ads.
The people not hired to service homeland security webcams will be hired to service challenge/response programs for all the major spammers.
668: Neighbour of the Beast
One problem people are complaining about is that spammers will deploy OCR or other technology to answer the challenged. I believe that this is much harder than it sounds, OCR is hard even in the best cases. With 10,000 fonts in 100 sizes with lots of noise, it would be extremely difficult to do OCR correctly. People that bright aren't spamming.
What would also help is a pledge in the email, that by sending this mail you agree that this is not unsolicited commercial email. This would be used to sue the spammer if he is indeed spamming.
Of course this would only work for spammers from the civilized world, but that is still the majority of the spam.
thad
I love Mondays. On a Monday, anything is possible.
No, then the spammer would have to provide a valid and static reply-to in the email, and we'd filter based on that. Even if they had a large number of domains/addresses, distributed spam-cataloging tools would make that ineffective.
I assume that the challenge-response is intended for messages already tagged as potential spam. In other words, low-scoring messages (spam-wise) wouldn't get the challenge. I certainly wouldn't expect a perfectly not-spam message to require the CR. Earthlink's (and other) spam-rating systems are pretty good, I think using it for the 'grey-area' emails would work well. And block the obvious spam without hesitation.
One question: shouldn't it be REALLY OBVIOUS to ISPs what is spam and what isn't? It seems that if a nearly-identical message gets sent to a large enough percentage of their users, it's clearly spam. Is this hard to do? Are spammers clever enough to distribute emails to avoid this?
Other than using a cow prod or a red hot poker, how on earth do you "educate" a spammer? Send them to Spammer School? Enroll them in self esteem classes? D00d, this is just about the stupidest thing I have heard in in a loooooonnnnnnngggg time.
Perhaps education is the way to go for Slashdot posters...
Sue them if you're richt (read: AOL), complain about them if you're poor (read: everyone else)
Sue them if your rich? Perhaps you can enlighten the techno-elite here how exactly you find a spammer who is sending e-mails with forged headers, connecting through open HTTP proxies? If you're going to sue them, you gotta find 'em first, right?
and be happy if they loose your DSL connection because of you as one guy dig who pissed me of days ago.
Ohhhh great job, kiddie! Sounds like you did a denial of service on some average home user who didn't happen to know that he had an open web proxy server. Whoo hoo! You da man!
Hey, Windows users, there is no such thing as "forward" slash, there is only slash and backslash.
After all my arguments about whether copying music on Kazaa is theft (which, until the light of the millions of mighty /.'ers reigned down upon me), I realized no one should impede free speech. How dare these bastards try and stop spam. They have a right, like everyone, to step on somebody's else's right to freedom and property just like the MP3 traders.
We need to contact the EFF for support on this.
Earthlink offers DSL and cable. I'm using it right now.
I am definitely in favor of a little pain up front in increased traffic from challenge-response to get the spam boys off the net.
I suspect that when the spammers stop sucking up so much bandwidth, net speeds will increase for everyone--including dial up users.
Remember when 14.4K was fast? So do I. And I think with a correction in the system, it can be a decent speed.
Laws are for people with no friends.
As someone with an earthlink email account that gets something like 50 spams a day (and I don't even use the account for much), I will turn this feature on as soon as possible. I'll see how it works, and I'll let you all know.
I do agree with other posters. Earthlink accounts do seem to get tons of spam by default.
i don't like my old sig.
Is it clearly impossible for a computer to generate an AI complete problem? A priori the computer could start with the solution and then work out the question, which may be computationally feasible, whilst working out the answer is not (without intelligence).
I've been using ASK (http://www.paganini.net/ask) which is an Open Source PHP based Challenge-Response system. It has a "Whitelist" which allows you to add approved senders and listserves as you can have either a From or a To address. It works so well because virtually all spammers use phony email addresses. Until spammers use valid email addresses, this type of system will continue to work. If they start using valid email addresses, then they can be dealt with in other ways.
Earthlink customers won't be able to receive any email from me in the future then.
If it takes more than one message to send them a email, it's too much effort on my behalf.
-- Even if a god did exist, why the fsck should I worship it?
What happens if the spammer just uses the same address in the To: field and in the Reply To/From: field?
A challenge will then be sendt to you, and will be accepted (since it comes from yourself....)
main(i){putchar(177663314>>6*(i-1)&63|!!(i<5)<<6)&&main(++i);}
It seems like the current techniques they use to obscure words/etc works well enough. I remember reading on here a few months ago about a technique that used easily identifyable images as a means of verification. For example a picture of a clown which you'd respond with clown. Granted this would only work for english speaking users, but it seems like a good start. I'm sure it wouldn't eliminate spam entirely, but I don't doubt it'd reduce it at least.
It would be useful if the system could be used to filter instead of block, at least for the first few months. Perhaps, if there is not response to a challenge after 72 hours, and email could be redirected to a 'Spam' or 'Bulk' filder.
This way, If I get monthly newsletters from donotreply@... and I want to keep getting it, i can approve that email. After about 3 months of this type of filtering and I would probably have approved everything I want to receive. Then, I could turn it back to blocking instead of filtering.
-the Hun
I'm a Tasty-vore. If it's Tasty, I'll eat it.
Here's an approach that would reduce the bad effects. For the first month after someone signs up for the feature, there are challenges sent. All messages are assumed to be legitimate, delivered, and the sender recorded as if authenticated. After the month, authentications actually start, and the user can go in and remove addresses that shouldn't have been added to his acceptance list during that month. The month gives enough time for for most users to communicate with most of their regular mailers, so they won't be affected - just the few that never sent messages during that month will be affected. Adding a few more features, like autmatically recording the addresses to whom the user send messages, and allowing the user to add an address before any messages were received from that address, would eliminate most of the remaining unwanted challenges.
Spamcop.net used to provide a service much like the one earthlink is proposing. I used the original system, but they have since replaced it with a blacklist filtering and SMTP chain verification solution only.
Speaking from experience, the challenge-response solution worked like a charm. Sure the occasional contact made fun of the whole thing, but it was generally intuitive and easy to interact with. There was no image transcription or the like, just a link that the sender had to visit (The assumption there was that spammers never used a real address as the reply-to) so no need to thwart auto-responders.
One other big feature was that the mail recipient always had the ability to release emails from the quarantine, as well as the ability to white list particular senders (very important for mailing lists and other bulk commercial email you actually do want to receive).
In general I loved the challenge-response system, and I was a little peaved when they did away with it. But as it turns out the SMTP chain verification, combined with the filters does a very good job too (Only one piece of spam has passed their filters in the last 9 months or so)
A residential broadband customer mailing through his ISP's mail server is whitelisted (most stuff from that server is nonspam). An rr.com luzer with an open proxy is tarpitted into oblivion (everything else in 24.0.0.0/8 is spam). Yes, Joe Linux running (non-relaying) Sendmail on his Linux box is also tarpitted, but he's not trying to send a million mails a day. So he's not hurtin'.
I can see a scaling problem in that you'd have to run some sort of adaptive filtering process on the receiving end, which might be prohibitive CPU-wise. OTOH, if you only scanned 1% of all inbound mails for "spamminess", you'd still rapidly figure out that for a US ISP, 24.0.0.0/8 is an ocean of spam with a few islands of real email, and 200.0.0.0/7 is a shitstorm of spam. You don't need to analyze every inbound mail - you only need a statistically-valid sampling of the inbound mail queue to figure out which netblocks are teh sux0r.
Having it be adaptive would be cool - because a South American ISP (which probably has less of a problem with 200.0.0.0/7 than, say, Earthlink does, because they have legitimate users emailing each other from within those netblocks). So an ISP in .mx would end up with a different set of teergrubing weights. They might end up letting most of 200.0.0.0/7 in, only tarpitting the worst /24s, and teergrubing all 24.0.0.0/8 because so few of their users get anything but spam from rr.com netblocks.
Think of it as combining the best part of SPEWS (naughty netblocks are noticed semi-automatically), without as much collateral damage (if you're an ISP, a 10 second delay to anyone emailing one of your customers from a naughty netblock will never be noticed, but it'll *kill* some dirtball trying to spam to 10000 of your users through an open proxy.)
... seems like they thought it through really well. It's going to go down in flames. Wait till they get flooded with customer calls like : "But I signed up to receive emails from {insert company}, now I'm not getting my coupons. What do you mean I need to go into an ACL and add them, how do I know what to add, what's an ACL" and so on. It won't work.
I'm the big fish in the big pond bitch.
The only solution is to make ISPs hosting spammers accountable for the spammers' abuse.
If there are spammers, there will be spam.
They *will* find a way around this Earthlink system.
Proletariat of the world, unite to kill spammers. Remember to shoot knees first so that they can not run away while you slowly torture them to death
In Soviet Russia, I ruled you
In theory, someone could send me a spamlike message and would have to reply to the autoresponder. In theory, a spammer could validate himself. In practice, those two things almost never happen. The system catches about 150 spams a day and over 90% of its autoreplies immediately bounce. Last time I analyzed it, only about 2% of my legitimate correspondents had hit the autoresponder (note, that's a fraction of a percent of my total legitimate email, since a given correspondent only has to validate once.)
I have yet to see a notification from Amazon, my bank, or other similar email trip the filter. Haven't had any of my correspondents complain yet, but I have had a couple of them ask how they can set up the same thing for themselves.
So if it's implemented carefully, I think this could be a big win for Earthlink subscribers and more or less invisible to everyone who communicates with them.
So, they'll be a way around the system? And people will be giving this email address to forms without reading the privacy policy, same as usual? Sounds like this won't work at all, at least when it comes to accidentally subscribing to new "opt-in" lists.
First it is important to note that the challenge system at Mailblocks is not something that can be automatically replied to. Much like the signup verifications for many forum systems out there the Mailblocks challenge email is simply a link to a web site. On that web site is a dynamically generated .gif of a number. The image is formatted in such a way so as to make it difficult for screen scrapers to write an algorithm which can decipher the numbers in the image (multiple fonts, different colors, background noise). If ever a spammer figured out how to programatically decipher the image then Mailblocks simply has to rework their image generation system and stay one step ahead of the spammers.
Next you have throw away addresses. Maiblocks calls these trackers. When you create a tracker a number and short ID are appended to the end of your username. This email address is then immune to the challenge response and can either be delivered to a purpose built folder or directly to your inbox. So if you wanted to have an address to get receipts from you simply make a tracker named say [username]+receipts4325@mailblocks.com. Then any email to this address can be delivered to the +receipts folder in your inbox. If you start getting spam at that address you just delete the address and create [username]+receipts5563@mailblocks.com and start giving this out. It can be a little bit of work to maintain your trackers but compared to deleting 20-30+ spam mails from my accounts each day it's well worth it.
When an email is successfully delivered to your main address the originating address is entered into your address book including the reason why this address was validated (completed puzzle, user added). Mailblocks also adds the address of any outgoing mail you write to your address book so that responses can be properly delivered without challenge. Finally, if you are expecting something to appear in your email that doesn't the 'pending' folder holds all email that hasn't been validated for a certain amount of time before deleting. If you really want to you can go back and dig through the email there to find the one you want, validate it, and it will be delivered to your inbox. If something gets validated you don't want simply go to your address book and either delete it or check 'do not deliver mail from this address'. Viola. Also of interest is the fact that Mailblocks can provide the same security to any other mail account you have. It can check POP3, IMAP, accept forwards, and even screen scrape web mail to bring all of your mail to a central location. When it does it provides the same callenge-response capability through these other accounts.
Who moderates the meta-moderators?
An image? So now to stop spam you'll have Earthlink "spamming" senders with image-laden emails? Or perhaps they will display an image that is loaded from their server? The latter won't work because I (and many) people don't allow our email clients to load anything off of remote servers. And it really pisses me off when I get images embedded in emails.
I know someone once sent me an email as I run a niche technical website and someone was asking me for some advice. I don't always have time, but in this case I did make the effort to reply and actually wrote up a pretty decent answer. Sent it off and a few minutes later I got a challenge-response mail saying that if I wanted to email the user that I'd have to verify that I was human. Screw that, I just deleted the challenge message. Who knows if the guy ever got my response.
Challenge-response would be ok if email was used only by people sending and receiving emails from their friends and family. Everyone would just do it once for each of their contacts and bam, you're done. But that's not how email is. Many people contact many (unknown) people regularly. We receive shipping receipts when we order something from a website. We have mailing lists.
A C/R system is the right solution for a certain type of email usage, but I don't think that particular type of email usage is representative of what most people use their email for.
Not to mention one of the biggest problems: Every spam message sent will consume the bandwidth it always has consumed, but will now trigger the C/R system to send a message back. So you have twice the email traffic. And have you ever been the victim of a spammer that used your email address as the From/Return-Path and you received all the bounces? Now imagine a spammer doing this and not only receiving all the bounces but also all the C/R requests.
No, C/R is just wrong in so many ways.
-----
Free P2P Backup, Windows & Linux
I would think it's safe to say earthlink email addresses will be blocked from mailinglists in the future.
It's going to be like using Mosaic to browse the web. Or telnet port 80.
I guess blind people will just have to give up on using email then? Sounds like an ADA lawsuit in the making.
I was using this until I realized I was spending more time enabling/disabling the C/R system or screwing with the whitelist that I was dealing with SPAM. Everytime I wanted to sign up for some mailing list (it it coming from company.com or parentcompany.com or ???) or a user would sign up for some service that sent an email automatically, which, of course, would never appear, causing complaints and yet another trip to vi to modify the whitelist.
Don't even get me started on all those damn email card companies - lots of missing Easter cards because dumbassonlinecards.com wasn't in the whitelist and again, noone is going to send confirmation mails from an automated system.
The whole thing got dumped. Back to SpamAssassin, which causes far fewer headaches. Fortunately, this Earthlink deal is an opt-in system. I couldn't stand to use it myself and I bet few customers will live with this long-term.
It would also be a problem for people with text based email clients
So does this mean that if you're blind, you don't get to send mail to C/R users? Another hurdle for blind users is just what the net needs.
This sig is not the Zahir. Lucky for you.
I'd like to suggest a way this could all be done automatically, so transparently your an AOL grandma could do it, and almost non-intrusively. Like the lessig-style stamp, all users would be charged say 0.01 cents to send ME an e-mail. but I would automatically refund this payment if either 1) the sender was in my addressbook/whitelist or 2) I did not file the e-mail in my junk mailbox.
what is needed is some sort of distributed postal service to handle the actual micropayments. And this is the main problem--how to collect these. I think the least intrusive method is that when you get an e-mail account you put down a pre-payment, lets say $10 on account at the postal service. when you send messages that are welcome your account is not depleted. when you send messages that aren't it slowly drains.
the cost of the postal service ditributed servers could probably be paid for by
1) the charges for unwanted e-mail
2) interest on the deposits on account.
thus people would be willing to set up these servers.
the final missing ingredient is a centralized server that coordinated the actual postal servers. all this would be would be like a DNS that told all of the remote servers the names of the other ones so they could communicate account info.
the transactions themselves would be in number about twice as the number of e-mails handled (one to the post office from the first ISP to receive the mail to validate the payment code in the header, one from to the postal service me to authorize refund/no refund), and the accounting message size very small.
Perhaps this is a rotten idea. its main benefits are 1) its not intrusive and is nearly transparent 2) it pays for itself 3) requires changes only at the browser level.
I does not stop spam from showing up in my inbox, but makes it very expensive to mass mail.
flame on! or suggests problems and their solution.
Some drink at the fountain of knowledge. Others just gargle.
Perl gurus, start your editors!
How many lines will it take to write a script to automatically reply to challanges? As long as the messages have predictable structure, you should be able to write a parser to pick out the word or picture they want, then throw it back.
College kids: Are you bored, broke, and of weak moral fiber? You too can make money while sitting on your ass by replying to email challanges for the princely sum of 3 cents per message! Combine the first suggestion with the second, and you've got yourself a money machine.
It's great to see an ISP take some decisive steps, but this scheme has weaknesses. Interesting to see how it goes. Despite the concerns, I'm cautiously optimistic.
As a twist, it would be interesting to see how that anti-spam vs. spam lawsuit with the copyrighted haiku goes (don't recall the parties names, but it's gotten coverage here). Maybe something similar could be combined with the challange-response system to make it illegal to respond to the challange under false pretenses. Raises a few slippery-slope legal issues that if you're going to touch, you might as well criminalize spam outright (which would be fine, of course).
"These people look deep within my soul and assign me a number based on the order in which I joined" --Homer re:
Then again, when do spammers use real replyto: addresses. Maybe responder bots aren't such a big dea.
"These people look deep within my soul and assign me a number based on the order in which I joined" --Homer re:
What they need is a new mail protocol. One that would probably be much the same, but carry with it some basic, enforcable restrictions. Like all advertisements, solicited or not, are labeled as such. And all unrequested ads have to be labeled as such. Thus, the E-Mail programs can identify them and place them in different Inbox folders appropriately.
Of course, people are going to want to mark messages as non-ads, which is why it needs to be enforcable. It would be a standard that a country would have to agree to uphold in order for access to that system to be available, so that when spammers break the rules, regardless of where they are, they get in trouble. Also a great help would be if all E-Mails in this system had more definite information about their origins, meaning headers can't be forged. How this would happen, I don't know. Maybe it's a myth like "unbreakable encryption".
Can anyone else think of ideas along these lines? Or has E-Mail simply outlived it's usefulness? Should we all just resort to Instant Messaging and forums?
Blatant self-promotion: Jerek.net
I like the idea of C/R, but one problem I can see is that if a spammer sends some mail to a C/R user, his mail relay send the challenge mail to the reply-to address. This almost never exists, so some relay or other sends the C/R system an "undeliverable". If C/R catches on, thats a shed load of "undeliverable" messages being fired back. Sometimes the undeliverable messages are not aimed at the reply-to address but "postmaster" or something similar. I stopped sending 550's to known spammers as well - you just get back more crap! In one case, it set up a mail loop that took down the relay.
I just let it arrive at my ISP these days, and then zap it all off with Spam Assassin. I know it's a bit of a blunt instrument, and I have it set to be more aggressive than normal too, but then if someone I know really is trying to reach me, they can phone, and I'll add them to the whitelist. The only problem with this, is that the spam is still using Internet bandwidth. Perhaps it's time to build email filters into the core routers of the Internet?
So apparently Earthlink is saying they will no longer accept e-mail from people who use text-only mail systems. Now a graphical, html aware mail reader will be required to successfully authenticate in response to the challenge.
Once this gets widescale usage, the spammers will simply start responding to the challenges (after all, it's not like that couldn't be easily automated).
In order to send responses to the challenges, it means the spammer has to provide at least a valid return address, and dedicate resources to responding to those requests (even if it is automated). It raises the cost of sending spam, and increases accountability due to the valid return address requirement, which is the best we can hope for with a SMTP-based solution for the time being. It's not perfect, but nothing is.
NO CARRIER
mail from:<>
rcpt to:<clueless@earthlink.com>
data
From: MAILER-DEAMON
To: <clueless@earthlink.com>
Subject: Mail delivery failure
Your mail could not be delivered because...
YOU NEED TO BUY WIDGETS NOW! WIDGETS ARE GREAT!
Now if you`re bald it`ll give you hair
If you got straight trousers it`ll give you flares
Feeling up you`ll get depressed
Out of style here`s a brand new dress
The stuff we sell is just the best
Passing all consumer test
Days of heaven nights of sin
Voodoo stick and sharks fin
When all around you seems like hell
Just one sip will make you well
Multipurpose in a jar
If you ain`t ill it`ll fix your car
In days of yore for all bad feelings
Washing socks and stripping ceilings
Nowadays its used medicinally
For all known human malady
- For the complete works of Shakespeare: cat
Broad use of a challenge/response type system actually massivly increases the mail volume - a legitimate email (one that's not yet been whitelisted) will usualy generate traffic = 2x the origional message.
1. Initial message is sent.
2. Challenge system responds with request for verification, often attacing origional message.
3. If the end user is real, they then respond to authenticate.
Traffic volume is actually less then for the illegal spammers. Of course in theory, no one sees is.
We experimented for a while here using Marco Paganini's Active Spam Killer project - it did do an admirable job of preventing users from having to see unsolicited emails, however there were a couple of issues.
1. The challange/response model added substantial additional traffic to our primary MTA
2. The challange itself REPLIED to a UCE, thus verifying the address and making it a saleable commodity.
I finally settled on a combination that utilized Spamassassin as an initial test, then checked used ASK as a challenge response system for those users who wanted additional protection.
\Drew National Data Director, John Edwards for President
The Spam Sleuth program from Blue Squirrel has added Challenge-Response. They call it the Turing Test. The same program also has other methods built-in like Bayesian, EMail Stamps, Simulated NDR (Bounce), Whitelists (Friends), RBLs, executable attachment detection and removal, regular expressions, etc.
It appears their Enterprise version works with any e-mail server, but the POP3 version is Windows only :(
They are easily bypassed using a smart enough auto-responder. If all you do is fire back the original message then you're on their list.
Did you read the article? A picture of a word is sent to the sender. The sender then has to TYPE the word in a response email.
The autoresponder would have to be able to analyze a picture and interpret what 'word' was being shown. There are ways to make this more difficult for an AI to do.
They sometimes fail to pick up the human response. I have several cases where people will simply respond to the email, removing enough of the critical content, to render the reply useless. This comes in two flavors. Email clients will strip out the Header information needed, or people will strip out the Body information needed.
Maybe the system YOU designed words that way, but there should be NO reason why a response email should be rejected if the respondee followed directions.
One of the biggest problems that these systems have is that they are totally incapable of handling Solicited email from a Bot
You have a point here.
The fix would be for the enduser to be able to manually enter approved addresses. I.e.: I manually add in the rule that says mail from amazon.com is allowed.
ac
What happens if a spammer start putting cowboy neal (pater@slashdot.org) in as the return address? The amount a spam just doubled...
- Spammer sends tons of email to earthlink with the Reply-To: set to be a random known good non-earthlink address.
- Earthlink starts mail bombing Yahoo, AOL and Hotmail
addresses.
- AOL, Yahoo and Hotmail gang up and RBL Earthlink.
- Earthlink rethinks it's approach
- Profit!
There are many reasons why most commercial email vendors don't have this feature on their mail serversIt needs to be done.
But... "The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers... " is a good point. It should be done slowly. Say, the first week only affects people whose address begins with 0. Next week, 1. Then 2, all the way through Z. Sure it will take almost 40 weeks. But it will be better that way. And we've gone longer than 40 weeks without this. What will another 40 hurt? I am SO there... This *IS* a service for which I'll gladly pay.
...as far as hogging bandwith if one has their mail client to include the text of the email being responded to.
Every time you have a back and forth email exchange the conversational thread gets longer... and longer... and longer... every single time one sends.
While I find this feature useful - 99% of the time it is wasteful. Especially if you include full header information in the text!
I am very small, utmostly microscopic.
A number of folks have pointed out how this really doesn't work so well in a real world situation. This is pretty much true, there are myriad problems. What can work fantastically is a two tiered approach though: 1) Use a Bayesian filter to sort your mail however you want (for simplicity lets just say spam/not spam). 2) Forward all filtered mail marked as spam to your CR prog of choice - this chunk of mail should already be confirmed in the high 90%'s to be spam - the few false positives should get caught. The reason this works so well is that the Bayesian filter approach is pretty solid, but there's always a worry of a few important false positives sifting through. This gets rid of those. If you really want to go balls-out you could make use of a service such as spamgourmet.com for ordering processes. Whenever you order something where you are expecting some automoted return mail that might hit the Bayesian filter AND also not respond to the CR use one of the self destruct e-mails. You should never get more than 5 or so e-mails from an order anyway. You can then just filter everything from your bogus self destruct e-mails into a generic "orders" folder.
Enter that in the subject line, email yourself, and it will show you a list of undelivered mail.
Click the link for your bank (with the "add this user to whitelist" option) and the email will be delivered to you, and the bank added to your whitelist, without them having to respond.
It takes just seconds, and it even works in pine. W00T.
Of course, you only do this when you are _expecting_ a non-whitelisted email, so the spam still isn't a problem.
As a sidenote, one spammer did make it through, once (ever). It was a company I did business with once (but never have again, due to them spamming me). One *PLONK* later, and I was spam free again. No big deal, really.
If you could be told what you can see or read, then it follows that you could be told what to say or think - BoC
Excellent idea, NOT!
Now one spam message creates a reply which has 100 fold the size of the average spam message, and, since the mail is forged anyway, goes nowhere.
Worse, if spammers forge valid adresses, one poor sob get's 5 Gigs of useless pictures of "validation emails" in his inbox instead of 1000 hatemails from lusers accusing him of spamming.
At least they should send pictures of naked supermodels with the confirmation secrets tatood on their butts.
Of course, Ticketmaster are spammers themselves. I booked tickets through them last year. I did it on a Thursday for a concert on Saturday. I create a unique email address (alias) for them (mf_ticketmaster_ca@my-domain.ca), on the following Monday I received mail from a third party on that address. I'm careful to ensure I'm opted-out from these things if the option is provided. Almost a year later, I still see occasional attempts in my logs to deliver to that address, even though I commented it out immediately after the first spam.
Instead, I'd rather keep this burden to myself. I've been using the Bayesian junk mail filter in Mozilla Mail for a few weeks now and it's made a significant reduction in the amount of spam I see in my mailbox. It's not perfect: some messages still get through, but no spam elimination system is. At my office, we've spent thousands of dollars on mail servers that are designed to reduce spam, yet many of our users complain that they still see the same amount of spam or more than they did before we installed the servers. We're back to giving people the same old response about spam in their mailbox: delete it and move on with your life.
Lousy minor setbacks! This world sucks! -- Homer Simpson
Excellent idea, instant DOS attack:
The funny thing is, that a system like this might _drive_ spammers to use From: adresses which the deem more likely to be whitelisted (esp. since the possibility of whitelisting complete domains seems to be a nice feature at first).
To the user spam is [not..extremely] annoying.
The the telcoms and ISP's, spam is [very] expensive... which drives up the price if internet services for us all.
If it were simply a matter of otherwise harmless irritation as you seem to contend, this would be a nonissue.
I am very small, utmostly microscopic.
Once the spammers are obliged to label their stuff "bulk", half the battle is won. Then they start collecting a "white list" of legitimate mailing list sources, and label every bulk message not on it as "suspected spam" and dump it in a separate folder.
Not me. I put earthlink.net in my blacklist ages ago. Too much Spam from this domain.
I had such a challenge/response system for a while, based on procmail and some handy Perl scripts around it. It basically worked - incoming mail was quarantined until the response came, then delivered. Was pretty smart - the challenge included an MD5 checksum of the original message, making bypassing the system next to impossible. Fake responds with no corresponding pending messages were dumped. But - it pissed off many people who wrote me legitimate e-mail for the first time, and I got all the bounces from the poor open mail relays. No big win. I dumped it and moved on to Spamassassin. I'm now down from 40-60 visible spams per day to one or two which Spamassassin doesn't yet know about. I report them and don't see them any more.
open (SIG, "</dev/zero"); $sig = <SIG>; close SIG;
The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers...
Geesh, Michael. You'd think that it would become the default choice or something. When something is "Offered," it usually means that you have to turn it on. If someone is geeky enough to be a member of a real mailing list, he/she is probably not going to use this. Granted, there will be a few loonies that do, but when they realize that they got 0 messages in 5 minutes from Gentoo-users, they're going to suspect something.
Otherwise, I think it's fantastic. I just sifted through 2,500 spam messages yesterday, from a period of time starting April 20th.
You need to restart your computer. Hold down the Power button for several seconds or press the Restart button.
First of all, the system is completely optional for earthlink users. For the users that are stupid enough to opt-in, they deserve the extra hassles they'll receive.
...
... all their spam will arrive via bypass addresses. Awesome!
But here's what it means to me, a publisher of a popular website...
When a new user signs up for an account, they get a confirmation email. Since I'm not about to check the server's return-path for C-R messages, C-R users will be out of luck. This means that at the very least I'll have to update my site with a special notice during the sign-up process that will notify earthlink users to expect problems.
The crux of the matter, there are automated emails that will fall victim to this C-R paradigm that AREN'T spam!
So, what is earthlink's "fix" for this problem? Well, it appears as though they will assign special addresses that users can use for sign-ups, sales receipts, etc. that will bypass the regular C-R system. Ok, great. Two problems with that
1. If the special bypass addresses are only temporary, then my users' accounts will become invalid because their email address is no longer valid and I don't allow ghost accounts.
2. If the special bypass addresses are permanent, and they're used for sign-ups and sales receipts, well fsck! Thats where SPAM comes from. duh. Great
Skiers and Riders -- http://www.snowjournal.com
There are currently three defenses to this:
Admittedly it's not foolproof. There is no 100% effective way to combat spam (short of abandoning SMTP). There's always going to be a risk that some spam will leak through or that some legit email will bounce.
"The day after it's deployed, every legitimate mailing list on the planet will get challenges from all the Earthlink subscribers..."
The challenge response system is opt-in. Earthlink customers who use mailing lists don't need to use it.
As I understand it, you can have a whitelist from online services that send out mail from robots. But spammers will just forge mail from these whitelisted email addresses.
What we need is similiar to this solution. Multiple send-to addresses generated either on the fly by a secure interface that the owner of the email account can use. Or you could have a Challenge/Response system to generate a send-to address.
Unfortunately, because there will be lots of send-to addresses and they will have to be kept track of, it will be necessary to incorporate this information into the mail reader/address manager. Not my idea of fun, but SPAM sucks more.
This way if some online retailer sells your address, you will know who did it and you can cancel that email address.
This could be a separate header in the mail too, and in that case this could be entirely done in the mail reader and code generator and wouldn't require any modification of the current internet mail system. But the senders of mail would have to add the headers to their mail.
Just think. It could be awesome!
Imagine a law against unsolicited comercial email with stiff penalties for those who break it. Yes, you can track down spammers easier than you can file swappers. Nah, way too drastic. Let's just make it impossible to email each other instead with "white lists". S-T-U-P-I-D.
Friends don't help friends install M$ junk.
"Once this gets widescale usage, the spammers will simply start responding to the challenges..."
No, they won't, unless there are some breakthroughs in machine vision. You see, the challenge "key" is more than just plain text that needs to be repeated, parrot-fashion, back to the server. In its best form, it would be encoded as an unusual font with a curving baseline on an image with lots of "noise" in the background.
People are extremely good at picking out text like this but it's a very difficult problem for machines.
There should be very little burden on people in your email address book. Part of article reads.... "It will allow users to automatically clear the e-mail addresses of friends, family members and other associates in their electronic address books, so those people would not receive the challenge e-mail." So, grandma will never see any of the C/R email msgs. kgregg
Why not make it so that every email header has a auth key. When a user gives out his email address he generates a unique key for that person that can be placed in the address book along w/the email account. Now when a person recieves email, he can "trust" the keyed messages and knows where they came from. It would also make it easier to find who is being a bit "loose" w/a persons email address. So if you start getting spam, and its key was assigned to Amazon, well now you know what they do w/your account data. Effectively, it makes a unique email address for everyone you want to email you.
so how hard is it really to find them??
What do you do if they're foreign? What do you do if they host their site through temporary web pages that use IP numbers instead of URL's for links? What if they use PayPal to collect money?
There are a lot of ways that spammers can be anonymous. So suing them isn't always an option, as gratifying as it might be.
-Looking for a job as a materials chemist or multivariat
Guarded email completely deals with some of the problems noted in these comments:
- David A. Wheeler (see my Secure Programming HOWTO)
The system described in the article is doomed from the beginning. Some people, like my mom, do not dislike spam to a high degree; they simply delete it when/if they get it. For these users it is much easier to erase irrelevant messages rather than use a method that will slow them down. The spam vs. anti-spam issue is just another variant of the famous cop vs. criminal deal: both sides get more and more advanced with time without completely winning or losing. In order to make spam less efficient it will be wise to educate users. For example, I found that as I started to replace my email address blah@blah.com with 'blah at blah dot com' I almost eliminated all my spam that was a result of web crawlers which went through message boards and all the other places where people would normally put their email address. Finally, if ISPs are worried about bandwidth, won't this new method generate more load?
I have seen a challenge/response system that defeated the OCR problem. It's in an online game, Planetarion, that had a problem with cheats using programs to manage their accounts while they were away and to run large numbers of accounts (both against the game's rules.) To log in, you first have to give your username and password, and then answer a question that's in an image. The questions are always obvious to a human, but a computer would need to be able to understand English before it could answer them. Since they implemented the login question, the bots have (as far as I know) disapeared.
SMTP is just a message passing protocol. What features are missing from SMTP which would solve the spam problem? The idea that AIM is a suitable replacement for SMTP is laughable.
What are the protocols and environments which are already being spammed? E-mail. Faxes. Telephones. Chat rooms. Web guestbooks. Weblog comments. IM. Religious nuts knocking on the front door of your house. What all these interfaces have in common is that you can't offer them to your friends without it becoming available to strangers.
The solution is to either add authentication, try to decrease instances of spam through legislation, or ignore the problem. Examining how we reduced problems such as fax, telephone, and front door spam may provide uselfull lessons in how to fight this.
I've used Earthlink as an ISP for going on 6 years now, and I must say, I've never dealt with better. For one thing, in the years that I've had my earthlink address, I'd say I never get more than 3 or 4 spams per week. What is my secret? For starters, if I need to provide an e-mail address for something that may result in unsolicited messages, I use one of the free webmail providers (Hotmail, Yahoo!, etc.) I can check those to confirm what I wanted, then never check it again, and my Outlook (with my primary e-mail) doesn't fill up with useless crap.
Another way to stop the spam before it starts is to keep your e-mail address from getting on those lists in the first place. When posting to Usenet, BBSes, forums, even Slashdot, use some sort of clever cloaking (Slashcode does this already), or even a fake email. Encryption for e-mail such as using a free personal certificate from Thawte or a GPL encryption such as GNU Privacy Guard is always a good idea.
In addition, Earthlink's Spaminator is a Godsend. With that baby enabled, I'm lucky if I get one spam a month. Case in point: my mother has an Earthlink address that she uses for her business contact. She complained that she's getting hundreds of porn spam and "enlarge your penis"-type e-mails (no idea how these got here.) Setting up a few Outlook Express filters and enabling Spaminator cut the dirty messages by about 90%, and she is grateful she no longer has to wade through such filth to get to her real mesages.
The bottom line is, the fewer spammers that have your address, the fewer spams you're gonna get. I have a Hotmail that gets 1000+ spams a day. My real e-mails get next to none. It's just like telemarketers, they get your number from companies who need a contact info for whatever reason. However, Hotmail address are free, whereas extra phone numbers to give the telemarketers, and then never answer, are not. Well, we do have Caller-ID for that, but that's another post...
An image? So now to stop spam you'll have Earthlink "spamming" senders with image-laden emails? Or perhaps they will display an image that is loaded from their server? The latter won't work because I (and many) people don't allow our email clients to load anything off of remote servers. And it really pisses me off when I get images embedded in emails.
It does neither. I'm using the beta-test of the Earthlink C&R system right now. The response sent to someone who e-mails me doesn't contain any images at all, just an URL that must be visited. It's there, on Earthlink's site, that the challenge is presented.
Amazon and other could move to a system where the order is submitted via an (encrypted) email. That would put the receiving address on the white list, and automatically allow the confirmation email to get through.
Or you'd still send the order through the web, but generate an additional email just for this purpose. But then it's not one-click shopping anymore.
. . .as long as people aren't getting them from their buddies. Even so, if emails are scanned for viruses/worms in attachments before they get to the user, there can be more wins than just stopping spam.
Here's the internal description of the service, which, by the way, is always going to be optional -- users have to turn it on manually. So fears of mass confusion from users when Earthlink turns this system on are a bit unfounded.
This is what the automated reply looks like:
And finally a more detailed description they supply:
SMTP does exactly what it's designers wanted it to do: provide universal delivery. Any message from any source, verifiable or not, will be reliably delivered to any valid recipient address. It's a very simple concept (and "simple" is what it is called), but it is very important that we have a protocol which meets this need.
Should we be using a limited delivery protocol for personal email rather than a universal delivery protocol? Maybe. But there will always be certain needs for universal delivery, and, if we don't completely destroy the system by implementing knee-jerk spam solutions, SMTP will always be there to meet those needs.
The problem isn't with SMTP; it is a problem inherent in any universal delivery system.
How are you supposed to be able to decode a challenge response on a text only terminal? What about the blind or (insert other person with special needs here) how are they supposed to respond to the challenge?
How long until spammers sue Earthink to stop them from deploying this?
HTML is obsolete. It's time for a new, simpler and richer markup language.
If the challenge is based on an image ("please respond with the fuzzy word in the subject line" or somesuch), where does that leave vision impaired email users? How do they respond to a challenge to get their email delivered?
My first hour as an Earthlink customer saw 20 spam messages to my account. My last name is hardly common (though it is short) and I've never used a major ISP besides Time Warner Cable. CR would have kept it empty.
/.ers tend to be so smart that we forget that most people loath change.
The only other alternative is PGP but that requires widespread deployment of decent processors and computers that aren't bogged down with spyware and other crap.
To get acceptance of PGP, email needs to become a little more inconvenient. First, people need to accept the idea of "ok, it's frustrating but it stops spam." Then they need to get the idea that, "ok, spam is over. Is there anything that can eliminate this irritating CR stuff?"
Then, and only then, PGP can be deployed.
Laws are for people with no friends.
Did anyone notice that in order to workaround automated systems that need to send legitimate email, such as Amazon when you buy something, or mailing lists you subscribe to, they give you a second email address that will not be protected by Challenge/Response?
I can see this being a big problem. In my experience, people only get spam if they have done one of several things:
1. Published their email address on a web page to be picked up by harvesters.
2. Given their email address to an online retailer that sells it.
3. Signed up for some spyware scam where they again give their email address to someone that will add it to a spam list.
4. Opened a Hotmail account, which, it seems is automatically sold to all the various spam providers.
In almost all of these cases, the act that caused spam to be received was the user giving out their email address to a non-trustworthy source.
How is having a second email address that people will just type into any webpage that promises free porn and bypasses Challenge/Response going to curb the spam problem? I give this system only 1-2 months before spam is back at it's initial volume, just using the new email address instead of the old.
You need to also educate users about the problems of giving their email address out to unreputable places on the net. A lot of users don't correlate their spam problem with the fact that they typed their email address into some website to get a free porno password the night before.
"When the president does it, that means it's not illegal." - Richard M. Nixon
Maybe I'm missing something, but what about "unsubscribe" messages? By definition, it's to an address or domain from which I no longer want to receive anything... I guess you just have to remember to go and delete it from your "sent to" whitelist before the spammer picks up on your address being legit?
(Most of us probably gave up back in the early days on trying to unsubscribe to anything, due to the prevalence of bogus headers, but people who try to unsubscribe may do themselves more harm than good - kind of like the current situation)!
You shall see a cow on the roof of a cotton house.
"The system automatically recognizes future e-mails from the same sender, so the verification needs only to be performed once."
and how long do you thik it will take for spammers to then obtain "pre verified" email addresses? they may even use yours... so not only will it make life difficult for the regular joe. my idea has always been completely server side.. you send an email, my pop server will challenge your smtp server to be sure that you have an acount on it, that the acount is valid, and that the ip you're using is on their network.. the idea goes a bit deeper than that, but im at work right now and dont have the time to go too deep into it right now.
The number of procmail recipes increasing greatly
As an Earthlink customer with a couple of small kids it would be nice to at least block the porn (and BTW - my penis size is just fine thank you very much). I see this challenge-response approach as a minor inconvenience that clears up a major one. The only problem I can see is with spammers that put someone elses e-mail address in the "from" column. Even so, this should mitigate a lot of it.
A goal is a dream with a deadline
Hire poor college kids to wade thru the validation requests and manually get your spam thru. I'm sure it will happen if spammers really want to get their message thru.
Morphing Software
If someone from earthlink emails someone else from earthlink, how would challenge response handled then? Do they make all mail that is sent returnable without challenge responses, and if so is this a temporary rule or are the addresses of all mail you send permanently whitelisted?
If the challenge response triggers a mail daemon reply, is it filtered or do you get flooded with those replies caused by all the spammers with forged addresses? If they are filtered, how do you know when mail you send doesn't go through without the use of message reciepts since mailer daemon replies are all different.
If I mass email tons of earthlink addresses with a forge from address, would it mailbomb the fake address, or do they have flood protection to prevent this?
Please send me SpamAssassin Config for Squirrel Mail to cucnews at yahoo doot com ;), I cant get anything from the website you sent
thanks
Sorry, that's the whole point behind C/R. I show you an image of a bicycle (or a teepee or a mountain or a list of numbers, etc etc) and ask you 'what is this image'. If you can show me an image processing program that responds correctly in all cases, let me know, I've got $1 million for you. And don't tell anyone else I asked.
This post expresses my opinion, not that of my employer. And yes, IAAL.
Because of this..I had thought that posting the answer first, where you could read it quickly would be best...
You find this really annoying when reading groups through Google, where the long messages are continued through another link..so, you have to read 2+ pages to get to the bottom of LONG threads to read the 3 line answer at the bottom....
Anyway, nice to learn something new every day...
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
Some of these tests can be beaten by computers (with much CPU time), some of them cannot yet. All of them are nearly "AI complete" and all of them are backwards- but not forwards-solvable. The important thing is that the cost of solving the problem by a computer is far greater than the benefit derived by solving it, to keep spammers away.
It's rare that you're presented with a knob whose only two positions are Make History and Flee Your Glorious Destiny.
Sorry, I just don't see this system of authentication working. This system seems like it would filter out far too many useful emails that are automatically generated such as on-line sales receipts, shipping status information, newsletters and such. As bad as SPAM is, this alone would make it a no-go for me.
I always wondered why legitimate email servers can't obtain a signed certificate similar to the ones for SSL. There is a fairly lengthy, well-established process for getting a properly signed certificate with a definite lifetime that firmly identifies who is at the other end of a TCP/IP pipe. These certificates would be exchanged at the start of a TCP/IP session between email servers similar to SSL certificates. If they so choose, an organization can then configure their email server to only accept email from other email servers that have properly signed certificates. Mail could also be accepted from servers with unsigned certificates, but these would have to be manually installed at the receiving end similar to how you would install an unsigned SSL certificate in your browser. Also, email from originating servers without certificates or with unsigned certificates could be so marked in the headers for disposal by the end user if they so desire.
Such a system would seem to have many advantages. The vast majority of legitimate email servers could easily obtain and renew certificates for sending email using a well-established process they are already use for obtaining SSL certificates. These servers would form a trusted network of identified servers where SPAM could be detected and offending servers cut out from the trusted network in a variety of ways. Ultimately, organizations that flaunt the system would be unable to renew their certificates and they would be permanently tossed out of the trusted network.
As an end user sending email, to be sure that you are able to send email within the trusted network, your organization (school, business, charity, whatever...) needs to have an email server with a signed certificate or you need to belong to an ISP with a signed certificate that you use for sending email. If you didn't have this, you would still be able to use the existing email infrastructure, but you would probably find that an increasing number of servers would reject your email as coming from a non-trusted source.
I'm certainly not an email protocol expert so I wouldn't be surprised if someone could poke 100 holes in the system I described above, but I am pretty sure that the ultimate solution will require a combination of technology (signed certificates) and bureaucracy (Verisign, et al.) to form a trusted network for email that SPAMmers can be quickly and efficiently ejected from.
Check out
www.maverixsystems.com,
it is appliance which sits between MX and mail server and
does all work. In production on couple sites, works great.
ALL NEW DATABACE. OVER 50,000 CHALLENGES AND RESPONSES. GAURANTEED TO WORK. JUST SEND $29.95 TO IVAN RIPITOV, PO BOX 456, MOSCOW, RUSSIA.
Send e-mail to ivan@mafia.ru for more info on daplamas, diploomas, penice and virginia enlargement.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Almost all of the questions I have seen here about challenge/response systems have already been answered in the TMDA FAQ. If you have a question about how these systems work, try looking there first, you may find your answer.
Most spammers fake their domains. I've seen spam coming from big companies like apple.com (probably using a spoofed address).
So what do you do? Other than "white listing" each and every email you get, this will still allow spam to come through...
Beside, even if a challenge is sent, wouldn't you want to make sure those emails are all spam? Maybe (most likely) automated emails (like noreply@store.com or customer@company.com) won't reply to the challenge, thus you won't get those emails. So basically, you'll still have to take a look at the spam just to make sure you get all those emails.
What I see is a similar solution but at the sendmail level. Make it an automatic challenge/response issue. If a sender sends an email, there should be a flag set on the server. When you get an email, the software should check to make sure the flag exists on the remote server. If not, this is a spam (the email was basically spoofed).
-- Leeeter than leet
I'll be interested to see how Earthlink promotes this new system to its subscribers, and how many of the non-technical ones decide to implement it. I'm concerned that it's going to be too awkward for everyday users to use.
Lousy minor setbacks! This world sucks! -- Homer Simpson
1. Spam would greatly be reduced if people wouldn't allow sendmail to use unresolvable domain names (dfkljdsf.com). There is a flag and it's up to (lazy) admins to fix this problem.
:)
:)
2. Spam would be reduced if the same admin would turn off open relaying on their own machines. It's "ok" to use it internally, but PLEASE not on a mail gateway!
3. Upgrade sendmail and read http://www.sendmail.org/antispam.html - if you're using windows, well hmm I don't know
Quit slashdotting and fix your sendmail.cf
PS: Also fix your broken proxy and vulnerable mailto scripts - if you need help, hire me
-- Leeeter than leet
I wonder if they're using Active Spam Killer:
http://a-s-k.sf.net/
He either comes off as a real interesting guy with encyclopedic knowledge,or a pathological liar with an ax to grind
I can't believe how behind-the-times you all are. The spam problem has been solved for over a month. And it was covered on slashdot FOUR times, for crying out loud:
one
two
three
four
All you have to do is filter e-mail packets with that bit set. Get with the program, people.
Email addresses are forgable. The from / reply-to fields are NOT TRUSTWORTHY - they are effectively USELESS for ANTISPAM purposes. Once an effective whitelist system is in place that relies on from, we'll see spam that works like Klez.
The only way to effectively defend against SPAM is at the IP level - via MX from DNS.
Hotmail, yahoo, free mail clients etc. are all doing a good job of policing themselves. If they can't police themselves, then punt the server. The spamboxen which increase the scale of spam that can be sent are the real problem.
The other important thing to do is to TAG the messages that aren't on the whitelist rather than deleting them, so the user can still find them.
Is this harder to use than current mail? I say NO because the amount of spam that people have to deal with is now so bad that the costs of dealing with managing the list is less than the cost of managing the spam.
But half the poseurs/posters here don't even understand how whitelisting or SMTP work before they go blathering off about 'throw out SMTP' or 'I won't get my f*cking mailing list'
that Earthlink/Mindspring will give up on blocking "residential" IP addresses? AOL seems to have already given up on that scheme.
I will not send mail through my local ISP's SMTP server as I'm not so hot on the retry settings. Consequently, I've told Earthlink/Mindspring customers that they just won't get mail from me anymore unless they change ISPs. Some of them are hopping mad at Earthlink about the whole thing. I'm sure they'd be happy to hear that Earthlink is finally going to stop blocking their incoming legitimate mail.
I've been using TMDA (http://www.tmda.net) for well over a year now, had maybe five or six spam emails sneak through the system in that entire time. Twice a day it sends me a list of "pending" emails so I can manually release and/or whitelist a message.
Challenge/response systems DO work, and they work extremely well. I think those who have not used one should give it a try before throwing rocks.
Eagles may soar, but weasels don't get sucked into jet engines.
The Washington Post reports that Earthlink is preparing to offer new spam filter technology that requires sender authentication.
I guess earthlink customers can't sign up for a slashdot account.
This solution won't work until webmasters realize that providing an email address is no more evidence that you are not a troll than providing a driver's licence is that you are not a terrorist.
Automated responses to whitelist queries will not guarantee that the Reply-To address is valid. Somebody can create an address dedicated to responding to e.g. TMDA queries, and then everyone and his brother can use that as the sender. Bounces, flames, and whitelist queries would go to that address and be promptly ignored.
What liberties would you have to give up? The right to send mass unsolicited emails is all. If you do a lot of mass emailing for legitamite purposes, then you can simply include a checkbox (on paper or online) that gives the recipient's express consent to be emailed. Enforcement is easy, anytime someone recieves a spam, they can forward it to the FTC (as is already implemented). The FTC can then launch a case against the spammer. Even if the spammer disguises their identity, this can be revealed by bringing the advertised company into court and serving them a subpeona to reveal their financial records to see which spammer they hired. It would probably also be appropriate to serve the company some sort of punishment for hiring the spammer in the first place. If spammers spamvertise companies without their express consent, then the company could be free to pursue a civil suit against them for damages.
How this entire process infinges on you civil liberties is completely beyond me. This is NOT the patriot act or something. The ONLY problem that presents itself is the international nature of spam. However, if enough countries cooperate on this issue then other countries (e.g. Korea) will be forced to comply or else many people will simply block ALL email from that country.
A fundamental problem of Spam is that the sender of an email cannot be identified and verified with 100% accuracy, so it is difficulty to filter 100% effectively. However, there is one and only one part of an incoming message that must of necessity be accurate- the To: address. So use the To: address to identify the sender! Publish your public address: "foo@bar.com". Any email to foo generates a reply "Thanks for the note. Mr. Foo loves you so much that he's generated a special personal email address just for you to use: 'foo_RANDOMSTRING@bar.com'. Please use this address in the future- sorry but you'll need to resend the message just sent to this new address. Don't ever give out this secial address to any else, because if Mr. Foo begins to receive spam on this To: address, he will automatically filter all future messages to foo_RANDOMSTRING straight to the trash." Every sender gets a unique RANDOMSTRING, so you can filter on the To: address. It's similar to throw-away email addresses, but coupled to a public address that triggers auto-generation of new RANDOMSTRING addresses. The sender has the inconvenience of adding foo_RANDOMSTRING@bar.com to their address book. Also, spammers can read the auto-reply and then add foo_RANDOMSTRING to their spam list, but this could be made difficult by putting it in a distorted gif image. The email client would also need to be configured to set Reply-To: correctly on folowups. One nice thing is that for user-requested bot-generated emails, one can simply give them a new RANDOMSTRING-based email address right off in the registration form or whatever. The ever-expanding number of foo_RANDOMSTRING@bar.com addresses adds to the overall load on the servers, but is that handle-able (nasty things could happen if your inbox got Dos'd)? In such a world, people would get used to pinging new people with just a short message to obtain their personalized RANDOMSTRING address. Kind of a weird system but maybe it's interesting to think about?
Curtains for windows?
Challenge/Response is a DDoS tool hidden in an anti-spam system. Consider this scenario: mallory@spamcompany.com sends out a million spams in which he puts alice@wonderland.com in the "From" field. Those running a challenge/Response tool automatically send out a challenge to alice@wonderland.com on receipt of this spam. If there were 10,000 people running a challenge/response tool, Alice will receive 10,000 challenges! If all of these had 10k+ graphics in them (as they usually do), Alice would receive 100Mb of mail in a matter of few minutes. This might disrupt Alice's mail servers, cause her to lose legitimate mail, waste several hours of her time, and quite likely force Alice (or her mail administrator) to drop all future challenges generated by Challenge/Response softwares involved in the incident; even those sent on receipt of emails that were written by Alice. (See my complete response to PC Magazine reviewers on whitelisting and Challenge/Response here)
Maybe I'm missing something but what happens when I send an email to you and your system sends me a challenge using a different email address? Then my system sends yours a challenge and it could go on forever...
Is there some simple way to prevent this?
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
As a happy Earthlink customer I've noticed the shockingly large amounts of spam that get through their normal filter, so I'm happy that they're moving to a better system.
However,
What happens to normal mailing lists? I'm on a few lists at Yahoo.
Before you say that, try living someplace where you actions are illegal? My church has several ministers in China. They store all their mail, both snail and real outside the country. They don't write letters back home (and some write excellent letters) while they are in the country. They must travel outside the country to do those activites that most of us consider everyday.
Sure there are many bad uses to forged headers. However if even once it can be used to get legitimate communication out of a repressive country then I'd prefer all the Spam I get (60+/day, most of it offensive) to losing that one communication.
Right now spam-catchers spend a lot of time designing a system to spot spam, and the spammers spend a tiny fraction of that time defeating the new system. A challenge system, ANY challenge, reverses that equation: the challenger spends a small amount of time creating a new challenge, and the spammer has to spend a lot of time figuring out how to make an automated response. Thus the arms race shifts from the bad guys to the good guys. Don't get hung up on what the challenge is to start with, it can be made harder with little effort.
Build a man a fire and he will be warm for a night; set him on fire and he will be warm for the rest of his life.
Earthlink users will stop getting spam.
Challange response is the way to go to prevent spam.
autopr0n is like, down and stuff.
I suggest that no two Earthlink customers will be able to communicate with each other unless they work something out beforehand to add each other to their approved senders list.
The Challenge-Response Authentication Protocol! ;-)
Please mod me gently. :)
--
Runnin' around, robbin' banks all whacked on the Scooby Snacks...
Top posting isn't the answer; trimming the quoted text is.
Good, inexpensive web hosting
You're honestly believing that I'll go type text from an image just to send you a mail? I might clink on a link like with TMDA or hit reply but that's about as far as I go out of my way to help you getting a spam free inbox.