Slashdot Mirror
Yahoo and Unilateral Anti-Spam Technology?
EatenByAGrue asks: "According to this Business Week article, Yahoo is planning on distributing a toolkit for Sendmail and other mail daemons that adds an encrypted source domain key to email headers to verify where they came from. However, critics are concerned that the scheme will be easily bypassed and that it ignores standards bodies. What does the Slashdot community (representing countless email admins, I'm sure) think of this proposal? On one hand, its a commercial enterprise dictating standard technology, on the other hand, the standards bodies have proven themselves helpless and hopeless when it comes to providing solutions."
easy email tracking system will be gladly welcomed by police and other agencies...
This Is Not a Sig
I try to be as standards compliant with my mail servers as is humanly possible. Even with numerous spam filters, I get about 10 legitimate email messages a day and 100 spams. Something has got to change.
Whether it is this technology, or another, something has got to be done. I'll implement this and hope that other admins do the same.
-sirket
I think this is a good move on Yahoo!'s part. As a developer I think a solution that is available and 50% effective is better than a solution that no one has implemented yet.
Lets get the implementations out there in the wild and use the feedback to create real solutions!
These days I can't even open by inbox, it is so overflowing with spam. I'm exaggerating, but at some point email is going to become completely useless because of spam. I do a lot of business over telephone (the way I used to do it before email) and have an ftp site to which customers can copy shared files.
It's slower, but not as slow as deleted emails that I never see and can't respond to.
I have been pwned because my
Spam is a SOCIAL problem, not a TECHNOLOGICAL problem. Spam must be solved by economics and/or behavior.
It's important for standards organizations to be taken seriously if people want to actually see careful and appropriate change made. We could, I suppose, say that the W3C is completely useless because Microsoft essentially dictates what will and will not be a standard on the majority of platforms but that doesn't make the W3C any more useless. Actually, it makes it much more important to look for a body that can develop RFC's and such so that we can all look at the proposed solutions and say yes or no. When a corporation decides on something it just happens and all we have to fall upon to stop the adoption of a (potentially) damaging standard is the free market system. However, in this situation that wouldn't have much of a bearing on a system that doesn't technically bring Yahoo! any more revenue.
while the effort is noble, it seems it will fall prey to the same beast that kills alot of the good ideas that rely on wide adoption. the greatest force in the universe. inertia.
Web folk always moan about MSIE's poor standards complience, for instance, but forget that CSS/Text came from them -- Netscape was pushing CSS/JavaScript at the time. Now, one of those is a standard, and the other is dead.
Ultimitely, either people will like Yahoo's idea and adopt it and it will eventually become a new standard, or it will be ignored by everyone else and forgotten. Only time will tell.
The extra key could be used by anybody who wants to, and ignored by the rest. And their implementation is open-source, so it doesn't look like a way of making an end-run past other ISPs. And since many spam messages come from fake Yahoo email id's, this would be a great way to immediately filter out those ones: if it says Yahoo but doesn't carry a key-->SPAM bin
I like the idea of a major player getting on with it and DOING something.
Would we rather have MS dictating an anti-spam standard? You can be sure such a beast would be a lot less benign than Yahoo's proposal
"From" address from what your SMTP server is, in which case I don't see how it could work for you.
This may put a lot of travellers out in the cold.
A solution is badly needed, but it has to work for everybody.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
...de facto standards emerge. One need look no further than POSIX/SUS and GNU/Linux for an example.
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
Doesn't sound like this will be too effective in stopping spam for
Yahoo users, and Yahoo is already a pain
to work with.
I setup a proxy and was a spam relay (unknowingly of course) for just
under a week. I got blacklisted on a couple of email sites, my ISP
bitched and I fixed it. So sorry.
So I'm now off every blacklist I know of, and everyone loves me again.
That is except Yahoo, the evil nazi bastards. I've filled out their
stupid, "fill this out to get
un-blacklisted" form at least 30 times (twice a day normally).
It must go into a black hole because they still are rejecting my mail.
Everyone else lets me through but stupid Yahoo, who seem to have NO
admins, no technical people, and a violate once banned for life reject
policy. Grrr. So I guess, if this new system lets them drop their damn
overbearing blacklists, I'm all for it.
Now that RIAA has gotten rid of Napster and trying to crack down, what did most people do? Other programers created other way to share music. Now all of this was just so we could get free music. These spammers are making money at what they do. How hard are they going to try and find a way to mail in our inbox? What we need to do is find a way to keep spammers from making money. That would stop them.
Since there doesn't seem to be any other way to deal with SPAM, I don't object to this. Especially if this is just a temporary measure.
It could be argued that if people go all out with these measures, in a while SPAM will no longer be sent, and then they can all be relaxed. But what will probably happen is this will just be another measure that will get circumvented.
Hopefully I didn't put any [] around my words.
It would be much simpler to add a record type to DNS servers to identify **outgoing** mail servers. Email proxies, where 60% of all spam comes from, would be immediately eliminated. Spammers with fixed servers and addresses are easily taken care of by the RBLs. Why introduce something that is more complicated and less reliable?
I admin a dozen domains professionally, and run a couple mail servers for volunteer orgs and all of them will get it.
-Brian
"...on the other hand, the standards bodies have proven themselves helpless and hopeless when it comes to providing solutions."
E-mail is supposed to do a certain job, and it does that job well, at least from a technical standpoint. The problems with spam are identical to similar problems in every other arena, it's just that they seem worse because of the level of automation. Even if it wasn't automated, spam would still be a problem. With idiots knocking on my door every other week with a hard sale for everything from oil changes to chinese food, I'm starting to almost regret the do-not-call list, because I didn't have to worry as much about these degenerates (if you don't take "No" for an answer and walk away immmediately, you are a degenerate in my book, and very door-to-door jerkwad so far has been one) giving my wife a hard time.
Standards bodies can't do anything to fix human behavior, unfortunately.
A far beter approach (which I think I saw on Slashdot but can't remember) is to use an extension which says whether IP addresses are allowed to use a domain.
This extension was based on DNS and basically allowed the mail server to query whether the IP address of the mailer was allowed to send on behalf of the domain.
Yes - this would be open to IP spoofing. Perhaps this DNS extension should be combined with the Yahoo method. If Yahoo, Hotmail and a couple of other providers adopted it could have massive effect.
To intially put live perhaps they could have an authenticated vs non-authenticated flag/filter in their web-mail client.
abuse@yahoo.com (purposely unmunged) claims that 419 spam from their servers didn't come from them. Gee, what's web108.biz.yahoo.com then? Some magic realm where the Nigerians have taken over Yahoo's network without their knowledge? That box relayed the spam to my MX, so it came from them, period.
Given that level of cluelessness, I assume that any "anti-spam" technology from them is going to be brain damaged from the start.
Crap like that is why yahoo.com is now on a "block all, except some" ruleset here. Other freewebmail services are getting there, too.
Look at it in very simple terms: what's to lose when you abuse a free e-mail account? Oh no, they cancelled my free account! I'll just have to make another! This is just going to ruin my day!
Until there is a real penalty for screwing around and getting an account cancelled, I don't want any mail from them. The revolving door of accounts needs to stop.
Anything to stop these parasites from smuggling our precious productivity.
:)
I say go for it. I mean, SOMEONE has to try something. I don't care HOW it gets done. JUST DO IT
While I applaud Yahoo for taking the initiative, this is just one more method to combat SPAM that will only work if everyone does it. I'd have to accept mail at my gateways from non-signaturized email anyway cause all the users would gripe that their friends couldn't send them the joke of the day.
Even then, these header signatures could be easily forged by the spammers.
Pessimists.net - as if life wasn't depressing enough.
There were alot of vital ascpects to this point made in the previous article some of which are quite thought provoking!
If you missed the previous thread, I hgihly recommended reading or even reading it.
Never try to beat a professional at his own game!
"Yahoo's plan is to write open-source software for popular e-mail server programs such as QMail and SendMail that would check all incoming messages to ensure they're coming from real Internet domains."
But SPF checks that the domain from which the email purports to come declares its mail will come from the server from which it has come.
Adding crypto sounds good, but I don't actually see that the encrypted token is doing anything, surely one will still have to look it up somewhere, which might as well be DNS.
If it wasn't offered free, I'd wonder if it was the old trick of finding something free, adding something that only you can do to it, convincing people they needed that, and then selling it to them.
I'm happy to look at any fix for spam, and implement any that are effective and within my powers, but in the end this is a social problem, and the neighbours of the dingbats that are behaving badly should express their disapprobation.
As reported earlier, Microsoft has been working on it's own anti-spam technology. Yahoo is trying to get their solution adopted by being the first one out of the gate, but since Gates and Co. have a lot of weight to throw around in any standards war, I don't see Yahoo's plan being a success.
In my opinion, Microsoft's plan seems a bit more feasible and crack-proof, too. By linking email to processing power, it makes it really expensive to send millions of emails.
I'm serious - spam eradicated over night, requires less than this initiative but is 100% foolproof.
If anyone wants to fund me on this then email me alan dot c5 at ntlworld dot com
All I'm really looking for is the funding for patents, any big company would use this once its disclosed.
Anyone interested?
I'm all for a spam solution coming from private enterprise as opposed to legislation- in fact, I think the former is the only method that has a chance of working. Maybe Yahoo's attempt will help, maybe they'll waste a bunch of money trying, but I guaruntee it's less money and less waste than Congress or the FCC doing the same thing.
We should use the resources that are already in place: Registrars.
Why isn't there a way for me to login to my GoDaddy account, and securely edit a list of valid IP addresses that email for my domain can be sent from.
Any email that isn't on this list is forged and can be deleted. Yes, people will need to setup and require SASL for sending mail, but that is easy.
The infrastructure is already in place, just release an updated RFC with a Jan 1, 2005 deadline for compliance (by registrars and sysadmins).
January 2nd our spam will drop by 99%...
That way, there's no question where the email came from, and exactly which account sent it. Plus traffic goes way down by not passing the content all over the place.
In addition millions of copies of the same email would not have to be held on recipient's servers, they would just sit on the originating server until received or until some time limit expired.
I guess this would prohibit using a (ISP's) email server as a repository, you would have to download everything you wanted to keep, but hey, no more email size limits! - send me the world - if I want it, I'll come and get it!
Could this help in the spam wars?
Spam is not a social problem, a spammer is a social problem. Sort of like an alcoholic being a social problem at a wedding - not the alcohol.
From excellent karma to terible karma with a single +5 funny post...
1) Free of ownership
2) Easy to implement on any platform
3) offers a valid chance of actually working
With those three met, I think it has a chance, especially with one of the more visible players helping it along. Though they might want to participate in some open-source deveopments (mozilla, etc) and contribute the necessary code to also help push along the effort.
"Enjoy what you're doing! If it becomes drudgery, you're doing it wrong!" - Jim Butterfield
Spammers wouldn't do it unless they got
paid.
What I want to know is: Who the heck buys stuff from
spam?
And how about we fine them?
I'm agin it. Cause problems. Will not fix SPAM. I have however added SPF records to my DNS. More flexible solution. I'll get around to patching my MTA to reject invalid incomming in good time.
Development of a workable solution, that is.
There have been a few times in the past where an entrenched technology has hit a wall in functionality, but because it was entrenched no one really did anything about it.
Then, someone said "Fuck standards - I have to DO something about this!" and started pushing thier solution. Other saw that someone was willing to take the first step, and took a step themselves. After some shakeouts, a new, more functional standard emerged.
My hope is that Yahoo has started the "SPAM proof MTA" development war for real this time. I want my e-mail system back.
Soko
"Depression is merely anger without enthusiasm." - Anonymous
As I look to my left on the main page, I'm greeted with an OSDN Personals ad. ...
HAAAHAHAHAHAHAHA.
In all seriousness, did anyone actually CONSIDER that brain fart before passing it?
I don't think so. I think a bad and poorly designed solution is worse than no solution. Especially when there is other competing solutions, which are argueably better, or at least equal to Yahoo!'s domain keys system, such as RMX. IMHO, Domain Keys offers no significant improvements to the spam problem, but rather adds a crypto overhead to the sending and receiving of every message. I think it is great that Yahoo is trying to innovate to stop the SPAM problem, but being cavalier and going at it by themselves is not the answer, especially when they have a great Anti-spam alliance with AOL and MS.
So that's what this was about. Spammers aren't adding gibberish to fool Bayesian filters; instead, that's the result of the spammers' lame-ass attempts at brute-forcing Yahoo's new crypto sig headers.
(As to why the nonsense stuff is usually in the body instead of the headers... hey, what can I say, that's spammer logic for ya.)
Please Help a Schizoid Genius!
We also need to stop the merchants from making money! I SMELL DDoS!!! Kill 'em all!
A blog about stuff.
Bad analogy, the alcohol is still a problem in this case because Spam is never a good thing, even "if done in moderation."
The unofficial
Totally against for privacy reasons. The best thing to do is to TRACK SPAMMERS, bring them to justice and make sure they don't do it again.
But noooooohhhhh, that's of course not what USA does. Commercial money has bought their legislature so instead of protecting civilians it now protects penis-pill companies and porn-spammers. Way to go suckers...
What is wrong with this world... It is getting more and more fucked up by the day. Now I face being tracked, because some assholes thought money was more important than serving the civilians.
Bah!
You mean like "reverse MX" records... google for RMX, SMTP+SPF, DRIP, DMX. (SPF seems to have momentum at the moment)
However, reverse-MX solutions will not kill off spam (a common mis-conception). The goal of reverse-MX proposals is to stop domain forgery where spammers are able to, with complete impunity, to tack on any old domain name to their spams. Which means that the unfortunate organization who is forged gets to deal with the thousands of e-mail bounces and the irate phone calls / e-mails from people who think that the organization was the source of the spam. As a mail admin, I'm able to control which servers handle inbound e-mail for my domain through specifying MX records. Reverse MX allows me to have the same amount of control over outbound e-mail from my domain.
What will happen instead, once reverse-MX systems (or Yahoo!'s system or other sender-authentication systems) come into play. Spammers will have to change tactics and resort to either forging one of the remaining domains that don't have reverse-MX information published, or they will register throw-away domains by the hundreds. It will drive up their costs a tiny bit (much like the impact of bayesian and other filters requiring them to use randomization techniques).
But the real nice side-effect of reverse-MX, etc., is that you'll be able to more reliable whitelist based on domain name. And your bayesian filters will be able to assign high ham values to domain names.
It also puts a crimp in e-mail worms that attempt to use a built-in SMTP engine to avoid detection. Unless the worm forges a domain with no reverse-MX info published, the worm won't spread (most MTAs will drop the connection). Instead, the worm will have to route through the user domain's SMTP server, where the mail admin is more likely to catch the traffic (virus scanner on the SMTP server, or rate limiters).
Wolde you bothe eate your cake, and have your cake?
Well sometimes Victoria Secret spam piques my interest....
From excellent karma to terible karma with a single +5 funny post...
One of the best solutions I've seen has been the SPF (Sender Permitted From) idea previously mentioned here and here.
It's on the agenda for my next mailserver deployment. Hopefully others will implement it as well. Seems like a really good, vendor and ISP neutral idea that could really help make a difference. And it has (or had when I last read it) a good deployment plan that allowed for phased deployments and letting each receiving site determine the strictness of the implementation for receiving email from other sources.
If that's what Yahoo is rolling out, even better. If a critical mass can just get behind a single solution such as SPF, then it has a chance to make a difference. It we keep deploying vendor-based solutions, we don't make any progress.
. 62,400 repetitions make one truth -- Brave New World, Aldous Huxley
I figure a standard has to come from somewhere. As long as it's truly open, and there's no chance of it closing later (ala the whole GIF fiasco).
The java community can look to eclipse for proof of this. I've heard accounts of how they bypassed the whole JCP, but IMHO managed to produce something that, preferences aside, functions according to spec and contributes to the community as a whole.
More power to anyone who sees a potential solution to a problem and pursues it, not to make a buck, but as a solution to a common problem.
Well, that's my 2cents.
if thats really your style...
The unofficial
This is down from dozens....
The problem is the standards bodies haven't done a whole lot to curb the problems with SMTP. The implicit trust it conveys is the WHOLE problem with pam and it's time to toss it out and come up with an alternative.
Hopefully whatever the alternative is, it'll allow administrators to verify the sending party or at least the relaying party and convey some level of trust and authenticity. With billions and billions of junk messages per day, email is well on the way to becoming just too much trouble to use.
This is just a stop-gap attempt to migrate closer to what is ultimately the only way to control spam: trusted hosts (also known as whitelisting).
We might as well just admit it. SMTP relays need to be licensed and regulated. This would stop spam. Implementing customized protocol-based front ends just slow things down and aren't horizontal in their implementation. And the idea of some handshake mechanism that denotes an acceptable SMTP source has to have spamming hackers salivating. They'll crack it within a week.
I would probably implement all of this on my mail servers except for one critical flaw, they only mention sendmail and qmail support (and presumably exchange as well). I use exim b/c I like the filtering options (and a friend of mine highly recommended it).
If they don't support exim, then I can't use it. Exim developers may implement it, but yahoo can't resonably say that they would start blocking before other projects have a chance to make their own versions.
On the other side of things, I'm going to start with the spf's shortly.
-CPM
---You're all I need, When the water runs deep, You're all I need, Now I cry my soul to sleep -- Collective Soul, Needs
"but hey, no more email size limits!"
Spammers don't send massive e-mails because it takes too much bandwidth to bulk send.
E-mail size limits come from mail servers that don't want individuals e-mailing massive attachments. It takes up bandwidth and storage while it sits waiting for the user to retrieve it.
And your method has already been implemented. It's called a news server. Technically there's nothing stopping you from using one as a primary e-mail address. Unless you can't set it to be post only (like SMTP) except for those with a user and pass to download the messages.
Kind of interesting actually.
Ben
Work Safe Porn
Anyone can google - I like to go ogle...
From excellent karma to terible karma with a single +5 funny post...
Pain is a powerful motivator...
Reverse MX and Yahoo!'s proposal, however, don't require widespread adoption at the start. In fact, the tipping point is probably only a few percentage points of the domain namespace.
After all, for just a few minutes worth of work (more if you don't already provide SMTP AUTH, or require users to VPN in to send e-mail already), you protect your domain against joe jobs and forged e-mail bounces. So there's a low cost-of-entry. (Yahoo!'s proposal requires more work then the simpler, less CPU-intensive SPF proposal.)
What happens next is that domain admins that publish keys/SPF information find that they're no longer getting joe-jobbed and they're able to block a higher percentage of spam then they used to. Word gets out and more folks sign on (second wave adopters).
Sometime after that, the big ISPs require your mail servers to publish SPF/keys if you want your e-mail to be delivered to their users. (FYI, this is very similar to AOL's whitelisting program, which is essential a privately-administered reverse-MX system where you tell AOL what IPs your e-mail is allowed to originate from.)
As a WAG about rate of pickup, early adopters have started, second wave folks will probably sign on in the spring/summer, and I wouldn't be surprised to see ISP-blocking by the end of the year.
Wolde you bothe eate your cake, and have your cake?
One day we will look back at something everyone does the same and say, oh, that's because it was impletmented first that way in OSS project Foo. Actually, I'm sure we could find an example of that day already being past.
from the article:
KLAATU, BORADA, NIh*ahem*
I use mailblocks. I get NO spam. It works.
-Nuke the moon
I have some spam-trap addys that automagicly block smtp servers. Near as I can tell, most of Yahoo's allegedly legitimate smtp servers are now blocked. When Yahoo! stops being a spam source, I'll listen to them. Until then, I'll assume that they're just trying to find a way to get their spam through filters.
now wait a minute -- you set up an OPEN MAIL RELAY
and send who knows how many spam messages out to
the world.
Why does yahoo have any reason to believe you
won't just "forget" and do the same thing all
over again?
sounds like you're the idiot to me -- why in the
world would you let your only ip address be used
as a mail relay? Maybe you should hire an admin?
you do realize that you're completely wrong?
This system, reverse-MX systems, and other systems will not be temporary.
The problem today with SMTP spam is that it's like being able to collect-call your target and they have to receive the call. Worse, the spammer is able to forge the caller ID (FROM:) information so that you can't simply use the caller ID info to decide whether to accept/reject the call/message.
This is the techical equivalent of a law requiring that caller ID information be accurate. It doesn't stop the telemarketing calls, but does let the receiver make a more informed decision about whether or not to accept the collect call.
Wolde you bothe eate your cake, and have your cake?
Nothing new needs to be invented here. What we should all be pushing for is signed email. There are many advantages to signed email, but here are the most relevant:
(A) Signed email signs not just the message headers, but also the message body. No chance of header substitution.
(B) Signed email associates signatures with some certificate chain and, presumably, a CRL (Certificate Revocation List). Abuses can lead to certificates being revoked.
(C) Because of the certificate chain, there is a chain of trust. There is always SOMEONE to sue!
(D) It is a simple measure to simply throw out any email that is not signed.
(E) Because of esign legislation, signed emails can be considered legally binding. In other words, lies, misrepresentations, libel, etc... in signed emails provides you with grounds for prosecution in courts of law - as if the signer wrote you the document and signed his name at the bottom (and yes, they can also be used for legally binding contracts and whatnot).
There is an issue with "Crossing the chasm" with signed email, of course. It would require a body such as AOL and/or Yahoo rising up and providing signature filters on incoming email to force such a solution into the mainstream. But once this is done, SPAM will practically dissappear. And any SPAM that comes in through signed channels can be dealt with in a satisfactory way.
I do not believe this harms any of us, btw...
You want privacy? The same techniques that allow you to sign email also allows you to encrypt email to your destination.
Worried about anonymity? Certificates can be issued that authenticate an email address without full disclosure of the owner of that address (but this may not be satisfactory for stopping abuses). Anonymity and stopping SPAM may, unfortunately, be mutually exclusive goals.... Any thoughts?
The reason that it can be true that 1+1 > 2 is that very peculiar nonzero value of the + operator
This works in Exim 4.x. This goes in the acl_smtp_rcpt:
deny message = Fake Yahoo, so you must be spam.
log_message = Fake Yahoo
senders = *@yahoo.com
condition = ${if match {$sender_host_name}{\Nyahoo.com$\N}{no}{yes}}
I also have rules for AOL, MSN, and Hotmail. the only differences for each is the string to match on (and the deny/log messages).
AOL: mx.aol.com$
MSN: (hotmail|msn).com$
Hotmail: hotmail.com$
-- Will program for bandwidth
working. Any more libido and no beast would be safe, let alone a human female.
If you know of anyone who has ever responded to a spam email, slap them with a wet slimy fish for me.
Clearly email (and pr0n) is what made Joe Average look at the internet. Email still remains the killer app of the internet (for those of us with friends) and we really must do something NOW!
This will hopefully send shockwave through the OSS community and get us into first gear!
People talk about charging, and most people write it off... But what if we did it slightly differently. You pay a worldwide organisation a small fee (say GBP1/$1) per email address you want registered, and the profits go to charity. They then authenticate that email address for all emails. If it turns out spam is being sent from that address, the account is terminated/suspended. SPAM would still be a problem, yes, but only to a lesser extent, and those spammers who don't get registered addresses, well, they go straight to the purge button...
It could also work on small systems, where you might not want to pay this organisation - you simply tell all your friends to add your email address to their allow list. Sure, you can't jsut send emails then, but it'd be a way around. What's stopping me (or someone who might want to start this off) from doing this? Non-profit, completely, probably with the profits going to some neutral cause (like a disaster fund) rather than some evil pharmaceutacal firm, "researching" cancer drugs. But that's for another time...
Seriously, could it work? BTW, I'm guessing I just released that idea into the public domain yeah? Is there such thing as a GPL for ideas? I'd have dozens...
Especially considering how promising the OSS model is, why can't we create a solution? We talk about the complexity of the problem, the importance of not breaking standards, etc. Who FUCKING cares if I can't check my email because it totally FUCKING BURIED in unsolicited junk...
I don't mean to come off as the thundering asshole, but this situation has grown so slowly its like watching a car crash spread out over the past 15 YEARS.
Please, experiment. Break things. I don't give a shit, but don't let us sit here moaning like helpless children while spammers sit back (laugh) and rake in MILLIONS.
Get fucking aggressive.
And if I hear one more idiot talk about how you have to cut spammers off by not buying their products I'm going to cut him off at the knees! If that would work you and Noah could be shooting dice right now and we'd have a hell of a lot less to worry about.
Programers still know how to experiment, right?
Quack, quack.
Read his comment before you reply you dumb fuck. It was an open proxy not an open mail relay. Sheesh. You probably run FreeBSD too.
I've taken the trouble to set up a web-warp link and post this message from 2034, in the hopes you'll be able to do something about this problem before it reaches the current state.
From 2004 to 2015, spam and filtering technology continued to battle aggressively. Both sides used the most advanced statistical and artificial intelligence methods available. By 2012, spam supercomputers (some among the top fifty supercomputing centers in the world) were crosslinking hundreds of minute details about you available from purchase records and anything else they could glean from the web, including several insidious spyware/virus products. You would get a conversational email purportedly from one of your relatives, full of personal details and chattering about some innocent topic, into which would be inserted a casual plug for some product. Spam filtering software began to lose the battle. Whitelists were the only way to maintain email contact with valid contacts, and the list was usually kept on paper in order to minimize the damage if someone in your circle of friends got a spyware break-in. The computerized contact list was no longer feasible on a home system; computerized email lists were kept under the tightest security. And for good reason: within minutes of your email address being exposed to a roving spambot, you would receive thousands of spam messages.
The problem came to a head in 2015, when voice-over-IP became so widespread that traditional land and cellular lines were all but phased out. Spammers were getting hit hard by the paper-whitelist solution, and profits were dropping to near nothing. With the wide availability of VoIP communications, customers were no longer paying phone bills, but a general bandwidth charge. Mass VoIP advertising was now inexpensive; hundreds of times more expensive than email spam, but still very cheap. Progresses in voice synthesis and artificial intelligence produced the ultimate annoyance: unending phone calls from all corners of the globe, running conversations based on gleaned information to craft a chatty personality to sell you something.
It was the last straw. Up to this point, goverments had been sluggish as usual, talking about stopping spam, but never accomplishing anything. Spam had essentially destroyed all the advantages of email, and accounted for the majority of bandwidth use. Now everyone was finding it nearly impossible to communicate at all. An international coalition of nations decided to remove spam by force, if necessary. Task forces were deployed to seek and destroy all spam installations.
Unfortunately, the spammers had opened a Pandora's box by then. The artificial intelligence spread itself into millions of computers and launched a massive attack on military computers, eventually gaining access. The entire world has been held hostage, slaves who must maintain computers and read all spam sent to them, under nuclear threat. There are rumors that some cities are being forced to build high-tech production plants for some kind of mind implant device. Most likely we will be forced to receive spam at all hours of the day, while struggling to plant food to keep ourselves alive. It's been said the AI gets power from the sun...maybe if we darken the sky we can starve it out....
...
This would a spammers wet dream.
They would write their own mail servers where more than one recpient would be linked to one post on the server. This means that they can send a small header it to a gazillion people and only spend 400 bytes on actually storing the message on their server since they only need one copy of a particular Email.
Bandwith is only wasted when a user comes to look at the mail, which also verifies that that user exists (double spam for you my friend).
So, this would make spam worse.
so in short
1) spammers could send at least twice as much spam as they can now.
2) they will get much better verification that the mail address they had is correct.
The Internet is full. Go Away!!!
A white list. Yes, that's what I do both at work and at home. When I tell people my email address, I ask them to add something to the subject line for the first email they send me. Otherwise, their emails are lumped together in my in^H^HSPAMbox and risk immediate deletion. After I've received their first email, I add their address to the whitelist, and from then on all their incomming emails are moved to the "valid" folder.
This method is a small concern to others, easy to administer, gives me 100% control, and requires no external filtering software besides your average email client. The best part is that I spend less than a minute a day dealing with SPAM.
....ask the spammers to stop......I mean _really_ nicely?
The problem with this technology is that it is going to further tax the receiving mail servers. With the amount of email that comes in (much of which is spam) mail servers are taking longer and longer to accept/deliver messages.
I have to say the SPF sounds like a good idea since it can be cached, but processing keys is a whole other beast.
Granted, I am assuming that there is some validation process to the keys that is based on IP or something, but if there isn't then the whole idea is worthless anyway.
-Jackson
Can I say as someone who has been suffering with my domain name being forged (hint picking a domain name starting with an "a" is a bad idea), I am really looking forward to some kind of Reverse MX system. The advantage of a signing system and key is the reverse does not have to be tied to a particular IP. The public part could aways be delivered by DNS, to stop "throw away" domains, we just need a way to tell the age of a domain, anything less than a month isn't accepted. if keys were centrally issued (by your registry), they could be say only issued after a month.
James
Anyone with experience with these standardization bodies knows that all of the complaining has to do with who's ideas win and who's name ends up on the standards documents. It's a particularly virulent form of academic arrogance. Solutions for signed email to stop SPAM are almost as old as email. Trust me, nothing is ever going to happen if one of the big guys doesn't put their ass on the line.
While the guys at the IETF fight for who has the biggest, ahem..., pen, the known email universe is collapsing under the weight of SPAM.
Let Yahoo hack and slash their way to a solution that works and then the standardization megalomaniacs can claim credit for inventing that idea 15 years ago while undergraduates at Stanford, Cambridge and MIT...
In the meantime, maybe we can have some peace...
The reason that it can be true that 1+1 > 2 is that very peculiar nonzero value of the + operator
Good update/answer.
The trouble with spam is the forged return addresses. If spammers were forced to use real email addresses:
;) (Kidding... his is way eaiser to update than mine) I think that implementing something like this on every ISP in America would immediately kill spam as we know it.
1) It would be much easier to block spam
2) It would be much easier to get their accounts revoked.
A friend of mine runs a script which ensures every email he reads is a real address. Essentially, he's got a cure for spam.
He has a script running on his mail that replies to every email he gets with a confirmation code. When the end user replies with that confirmation code (all it takes is hitting ctrl-r and ctrl-enter) that email address is adding to his "verified email address" list, and the original email goes through.
He doesn't even look at emails that aren't confirmed yet.
If we could get this implemented on a systematic level (such as via confirmaiton reciepts automatically & transparently handed by the Mozilla mail client) it would essentially end free for all spam as we know it. And it doesn't require rewriting the RFCs or adding new headers, or whatever. It would work with any mail reader... though adding in transparency would require updating people's mail clients.
The downsides:
-Two extra emails for every one original email are sent... but only the first time. After the email address is verified, it doesn't need another confirmation. If this is implemented system wide, the savings in the reduction of spam messages would greatly outweigh the extra cost on the network.
-People who do not confirm don't let their email get through. This happened to me the first time I mailed him after he installed his system. I send him an email, and went home for the day. Didn't see he didn't recieve it until I checked my mail again. Mail clients that handle confirmation transparently would (nearly) solve this problem.
As someone who has experience writing spam filters (I wrote a pretty good neural net spam filter way before that Graham fellow wrote his bayesian filter, that publicity hog!
Shame they move so slowly... and never can agree on how to implement anything...
-Bill Kerney
I probably should've been clearer... (I do realize that Yahoo!'s is open-source).
The original poster was saying that the Yahoo! system would need to be closed source in order to be secure. You and I both agree that a closed-source system does nothing to make Yahoo!'s system more secure and I was trying to point that out to the OP.
It's been highly educational (and sickening) to watch the machinations within the IETF and ASRG mailing lists. (I came to the conclusion last May that the ASRG probably would never actually fix anything...) Nothing surprising if you've ever worked around middle-management in a large corporation though.
Wolde you bothe eate your cake, and have your cake?
1) tell everyone in the world to stop buying what the spammers sell. This will make it useless to spam. You probably have to tell morons twice though.
2) Tell everyone in the world to lead spammers on as far as they can without actually spending any money. This will waste the spammers time to a point where it isn't profitable.
3) The combination of 1 and 2. I actually do both of these but as you can tell by looking at your inbox I'm loosing this battle.
The Internet is full. Go Away!!!
A simple turing test for accepting email from unknown senders wouldn't stop spam, but would prevent people sending out millions at a time. That's really all we need, require a person to be behind the scenes.
Infact, all that would be needed is a website anywhere that could issue a test and return a digitally signed tag that could be cut and pasted into any email.
Here is how we can solve the spam problem once and for all.
Turn on finger. Yes you heard me. Let's re-implement finger. Here is how it works.
My SMTP server gets email from joeblow@123.com. I finger joeblow@123.com. If 123.com says joeblow is a real user I then accept the email, other wise I can it.
Voila! No more forged headers, no more spam.
This very simple simple solution would also allow legitemate businesses to send spam to the people who have opted in.
War is necrophilia.
I cannot send legitimate mail to at least 4 relatives and friends due to the no-residential-IPs blocking.
Unless these schemes are set up so that anyone can generate the right keys and become a valid mail sender (at least until they spam) I don't see much future in them.
If you think about it, the whole trend of the internet has been away from central authority type solutions. If having each person registered and numbered and trackable in order to send email, why aren't we all on Prodigy, the Well, tenet, or any of the other well administered, no riff-raff systems that died when everyone fled to the internet ?
Unless I've totally failed to grasp the concept of SPF, it seems that in an "SPF-protected" world the spammers will ensure that they only spam others using your actual email address, not just some made-up email address from your domain. Hooray for progress. Meanwhile, be sure to ask your doctor to prescribe a whole spectrum of antibiotics for your next minor viral infection, to ensure that the rise of antibiotic-resistant bacteria continues unabated.
I will say that the spirit of the SPF concept is 100% AOL (not counting the former Netscape).
I take it that you don't actually know how to properly verify message headers.
Not sure if this would work but why don't they just upgrade all the ISP's mail servers to reject mail based on whether it it contains false headers. I think this would get rid of a lot of spam.
I have been searching all over, but I cannot find any specifics about how this will be implemented. Could I see an m4 snippet to add to my Sendmail configuration? Could I see an example zone file for my DNS server? Anything, please!
Seriously, are there any links at all to some technical specifics?
First of all, let me put into perspective: I hate spammers. I hate them because they and their likes, the virus/worm/etc writers, the child-porn freaks, terrorists, are forcing the rest of us to dismantle a lot of the features that we build into the Internet (ie to close down our machines with firewalls and anti-virus software).
This proposal may or may not be good for reducing spam, but it seems to me like a very good way to get 'rid' of privacy on the Internet. Using assymetric crypto techniques to identify bad guys means you'll be able to identify everybody, too. If this catches on, expect it to be extended to every tcp, or even ip, protocol. (after all, don't we want to get rid of im spammers, blog spammers, etc. too?)
I love the Internet because I can say anything and get away with it, 99% of the time (that is if you don't go contrary to evil laws like the DMCA or the <name your favorite nation here> anti-hate-speech laws). This has a lot to do with the fact that it is still largely out of control of a single government, multi-or-extra-national organization, or corporation. If a single, tracable identification measure follows you throughout the Internet, it is inevitable that it will be taken over by one such organization it the medium term.
Personally, I got rid of 95% of spam with Mozilla. And I still get the spam I want, like amazon or chapters.
In all seriousness. How much spam can you possibly be getting?
I keep hearing horror stories about people getting 100+ spam emails per day. This leaves me with the question, HOW IS YOUR EMAIL ADDRESS GETTING INTO THEIR HANDS!?!?
I don't sign up for every "free" offer that I come across. I don't have business cards made up with my email address. I have two email addresses, I might receive 10 spams per week between them.
WTF are all of you doing to get on so many spammers' lists?
LK
"Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
I too admin mail systems for dozens of domains. None of the domains I admin will use this and I will blacklist any and all domains that utilize this system. I will not under any circumstances support any non-standards compliant email implementations. I configure my MTAs to flat out reject all non-RFC2822 compliant email messages already. The Internet email community DOES NOT NEED ANOTHER CISCO. Standards are what ties us together. Half-assed and poorly thought out implementations serve no one in the end.
I subscribe to some of the debian debug/discussion lists. I've noticed 2 things:
A) They get spammed
b) The address I used there gets spammed a lot - not because they sold me out but because (I'm assuming) spambots picked me email off the HTML archives - and somebody on the list seems to be infected with a virus (windows virus, go figure).
Luckily I use an alias so I can be special extra filtering etc on that address, but this really has to stop. Spammers are perverting every useful form of email on the internet, and pushing into non-email formats too (popups/popunders/etc). I'm considering unsubscribing the group and killing that alias simply because the spamming is more than expected.
Anonymity and stopping SPAM may, unfortunately, be mutually exclusive goals.... Any thoughts?
Yes, they would be mutually exclusive. If spammers can generate disposable keys, then you might as well be filtering by the from header. I've been shouting this myself lately. Verisign has a fairly in depth whitepaper on the subject. This seems to be the most obvious answer, and more likely to actually succeed than all the hash cash/taxation schemes I've heard people kicking around.
Crap.
~ Aero
I remember a day when e-mail was nearly Spam-free, and Spammers only got away with it once. That was back in the mid-90s on the Prodigy Interactive Service, before they had opened their mail system to the Internet. When there was a closed system that required a vaild credit card to open a master account, and accounts who abused the e-mail system could be terminated without any appeal, spam existed but was very rare and quickly dealt with whenever it sprouted.
If Yahoo, MSN, and Earthlink all joined together to form an "invitation only" e-mail club, and each took responsibilty for patroling its own user base, the world would be a whole lot closer to a spam-free place. "Pink contracts" would not be tolerated, as the entire ISP would risk being expelled from the club, and therefore not be able to offer functional inter-network e-mail service. Remember, the Internet is nothing but a network formed by joining other networks... nobody has to honor the requests of other networks, however.
It's really not hard at all.
1. Create a new e-mail account.
2. Give this address out only to close friends and associates whom you trust, asking them kindly not to give it out to others.
3. Do not use this new address when making online purchases, filling out registration forms, etc. Use a junk address for this.
4. Create yet another account for mailing lists. Should it someday become overloaded with spam, delete this mailing list account and make a new one.
5. Enjoy spam-free e-mail.
My old e-mail addresses (chris@insert_one_of_my_domains_here) has been around since 1995 or so. It can be found all over the Google, mostly in old postings to mailing lists. This address gets an unfuckingbelievable amount of spam--around 3 per minute--and is no longer usable. I used the above method to get myself a new, usable address and I haven't seen a spam in months.
It's a value judgement... and according to my values, I think this is not a great idea.
First, I think the benefits of having free and semi-anonymous e-mail outweigh the disadvantages of having to use and maintain spam filters. Obviously, many people disagree with me here, and more all the time.
(Here's a conspiracy for ya: what if some Big Brother is trying to kill the free exchange of ideas in e-mail by burying the whole system with spam? I don't believe it's true, but it's worth wondering about before jumping to non-free solutions!)
Second, even if I thought that killing spam was worth the cost of crippling some of e-mail's better and more distinctive features, I think going about it in a non-standards-based way is likely to be a road to chaos.
The best solution, I think, would be to supplant e-mail with something new that works in a more trusted and accountable way. If someone really hates spam, they can use only the new system; if they want anonymity and freedom at the cost of spam, they can use the current mail system. The systems could coexist much like Usenet and the Web; each is useful for different things.
With reasonable men I will reason; with humane men I will plead; but to tyrants I will give no quarter. -- William Lloyd
As ShakaUVM stated in a previous post, the problem with spam is forged return addresses. As another poster mentioned, spam is really a social problem. The problem is, one or two dumbheads loose in the world can cripple a great technology (email).
So, spam is a social problem - a few people are a nuisance. But the problem is, right now - even if we pass great anti-spam laws, we really have no good way of knowing who is sending a message. So what if it came from ip address 3.14.15.92? Spammer joe can disconnect from that address right after he sends said spam, and nobody wants ISPs' logs to be able to be subpeonaed, do we?!?
So spam is a social problem, but we have no way of tracking the offenders. I think an authentication-by-encryption scheme is a Good Thing, but wait - I think there are such standards already out there.
The STARTTLS extension for SMTP, in RFC 2487, allows SMTP traffic to be transported over a TLS (SSL) connection - also allowing for the same type of CA-signed certificates that HTTPS is famous for. So now we can tell exactly what mail server mail is coming from - and we can refuse mail from uncertified hosts, or prosecute abusive hosts.
Anyway, correct me if I've misunderstood anything; what think ye all?
Microsoft hates spam clogging their servers
Yahoo hates it
Your ISP hates it
We hate it
Big business hates it
Who likes it? I think that spam is more detested than telemarketing... at least with telemarketing I can track down who called me easier or at least yell a bit at the pleb calling me.
So if everyone pretty much hates spam, why isn't they a jointed effort against it? It is true that businesses don't often co-operate, but it's not unheard of for large companies to join forces against something that both strongly support/oppose - nothing I could think of moreso than spam.
For the record, I think it's a great idea.
-russ
Don't piss off The Angry Economist
Spam is a classic case of the tragedy of the commons.
As long as sending millions of e-mails relatively cheaply is possible, spam will NEVER cease to be a serious problem.
You have to break the economic back which supports spam.
It has to cost something to send an e-mail.
True, it will not disappear, but the volume will drop dramatically perhaps even to the point where e-mail will become useful again.
Absolute statements are never true
I don't mind downloading the spam because I have broadband. Getting mail is no big deal, but sorting it is.
The solution I use requires that one owns a domain. Simply provide specific addresses to people/places/things depending on your expectation for spam. Filter on the client name based on the to: field and most of the crap drops into the crap folder where it belongs.
This combined with a bayesian filter keeps the spam to a very reasonable level. One added bonus:
You can know who sold you out and pass the word to others.
I use gandi.net for this. They provide e-mail redirection for free with a grab bag for unspecified addresses. 12 euros per year with nice online admin tools combined with very reasonable legal terms makes the service well worth it.
As for the e-mail problem, it is going to come down to trusted mail servers. I believe we all should be able to run mail out of our homes, because that is part of being peers on the Internet.
So, anyone can send mail, but if you expect anyone to actually read it, you need to be trusted by at least someone
Blogging because I can...
If you can send an e-mail anonymously, so can spammers. If spammers can't send e-mail anonymously, neither can you.
The price of spam doesn't come anywhere near the value of privacy and freedom of speech. I happen to like the idea that should a need arise I can easily send an untrackable e-mail. I'm sure plenty of people in more intrusive countries already enjoy this ability.
Click on the link in my sig for my method of dealing with spam which is highly effective that doesn't destroy the privacy of the sender or cost money.
Ben
Work Safe Porn
Maybe Yahoo's idea will work, though it seems to be quite porous and more of a surveillance tool than an antispam measure.. in fact it is quite plausible that this is Homeland Security's wet dream and is being sold by Yahoo on their request (though that is more paranoid than we have to be).
I have a concrete proposal at the end of this post so please read on.
Anyway someone mentioned the tipping point and I am reading this after cleaning a thousand spams out of my mail folder so I am ready to consider lots of things.
But one thing is definite about all this. If these guys were terrorists planning some horror and not just an army of rotten people bent on selling viagra and insurance, they would be shut down in a heartbeat. You can follow the money! (As many people have.)
Note these datapoints:
- Telemarketers don't like getting phone bombed, as Dave Barry launched retaliation against an association of them.
- Spammers are in it for the money
- Their clients pay because they want to sell something.
- Their clients are living in meatspace and are allergic to publicity.
- Spam is by definition, easy to get since so many are sent from each machine. (In fact I get too many to even reply with "unsubscribe" to them all).
- We all see spam, but can't stop it because the spammers are laughing at us by endlessly transforming their campaigns. The helpless feeling I suppose is similar to terrorism in that there is a feeling of a nebulous enemy profiting by your openness, there is nothing to grab hold of.
- People are willing to pay money to stop spam.
- Homeland security (probably) and the NSA and similar national organizations (definitely), and telcos and isps (of course) are sitting in front of the big routers around the world. This information can be coordinated.
- Some big organization wants a steganography analyzer built quickly (recent slashdot story)
From this and a bit of blue skying and paranoia, I get:
1. Spam, which is subtly personalized and includes photos and hyperlinks, could be used as a communications network by terrorists, so definitely falls under the national security bailiwick. Ditto for viruses and worms, though they are maybe too visible.
2. Though maybe it is better to unlock the messages than to stop spam, from a security standpoint.
3. Certainly it is possible to make transparent who exactly is sending spam, and how the money flows from their clients. Both by surveillance and of course just trying to buy some of their services.
4. If it isn't illegal, they can't be put out of business and so long as they have clients, it is a "business opportunity".
5. But by focussing the anger of thousands of people on each client and detected spammer, this lucrative business can be turned into a financially losing proposition.
6. Finally, if we make it impossible for their clients to sell their wares, there will be no point to spamming. This suggests that rather than trying to secure all of the honest email, we should focus on removing spam from the network. I don't think blackholes work, however it is quite possible that a finer granularity and more intelligence might work. (See below)
So I welcome technical fixes against spam but think they should more involve information sharing than an attempt to cryptographically secure the email network, since the power of email is fundamentally that it is so easy to use.
I would propose that a group of people are selected around the world to manually go through their incoming email and note which emails are spam, preferably qualifying what type it is and using some simple tools to also note whether this is the work of nefarious arch-spammer types that play tricks on you, as opposed to honest mailing lists. It should be an open architecture which allows more than one organization to do the grading. Perhaps one will only filter porn, etc. I believe some large antivirus companies do something a little bit like this on an automated level to learn about thre
Yes, you have failed it.
In the world of spam, 50% effective is really a brief pause, followed by 0% effectiveness. This won't even make a dent.
Yes, you've missed the mark a bit...
Under SPF (or other reverse-MX proposals), e-mail that is purporting to be from domain X has to come from a limited set of IP addresses (typically the official, mail admin monitored, virus-checking, maybe rate-limiting, SMTP servers for a domain).
So in order for a spammer to spam someone using your actual e-mail address they have to:
1) hack into your domain's outbound mail server and send e-mail from there (nothing new in this risk)
2) hijack/trojan your machine or a machine in your organization and then route e-mails through the official SMTP server (same as what happens now, except that the mail admin is more likely to notice that customer 32432's account is sending gobs of e-mail)
3) poison the DNS SPF information (tough attack to pull off, can be combatted and might lead to new security in the DNS system)
4) spammer goes in search of a domain that doesn't have reverse-MX info and forges that domain onto their e-mails
5) spammer starts to use throw-away domains at $X each
#1 and #2 are the keys... SPF is designed to make it much more difficult to do domain forging or joe-jobbing.
Wolde you bothe eate your cake, and have your cake?
If you've tried any of the great spam filters out there, you would have found out by now that they work great at filtering out the vast majority of spam. I use SpamAssassin. Others use PopFile. There's another half-dozen good Open Source spam filters out there which will get your email back. Maybe others can chime in with the filters they like.
That's indeed your prerogative.
However, the standards body (if you mean IETF) will probably take at least 2 years (if not 5) before they settle on a standard. At which point, the majority of folks will probably have implemented solutions that are "good enough".
Have you been bugging the standards folks to get their ducks in a row faster? The ASRG's charter just changed again a week or two ago (pretty much without a vote and/or much discussion on the list).
Not saying that Yahoo!'s proposal is good/bad, but as domains take steps to protect themselves against joe-jobbing, domains that don't will increasingly be forged onto spam e-mails.
You see? Thats really pretty lame. While we sit around with our proverbial thumbs sticking up our asses spammers are STILL laughing all the way to the bank. The next interesting solution is a variation called TMDA (Tagged Message Delivery Agent). It looks like a great idea, only no ones really using it. Oh and its hard as hell to setup and configure which might explain at least marginally why its not being used more. And of course there's Spam Assassin and its Bayesian buddies which is so far from an answer they are the next best thing to doing nothing!
I know I'm ranting, but honestly, hasn't this gone on long enough? I think we need to change the way we look at email. Look at IM services or something else to provide a model. Not everyone should be able to send me their Barnyard Bonanza websites or their Raped 13 Year Olds video offers. Its fucking too much. I want a public address? Fine, let me mark it public. Let me set up a special account that can absolutely swim in pornography and viagra adds. But as a defacto standard? I'd have to be stupid, but oh well.
I'm sorry if I'm stepping on anyones toes. Honestly. But this has gone on long enough.
Quack, quack.
Is this a bad joke? It like a hokey version of one those white separatists saying the holocaust was an exaggeration. All it takes is one list, which gets traded or sold to people who trade and sell.. Use your imagination.
Just because it hasn't happened to you doesn't make THAT news.
Quack, quack.
These guys support Exim, qmail, Postfix, Courier, and Sendmail. And as far as I can tell it IS the next best thing to sliced bread. It might not be a perfect answer, but its a hell of a lot better then nothing. If you offer hosting let me know, I'd love to move my site over to a service that uses something more effective then Spam Assassin or Bayesian filters.
Quack, quack.
This has been true of any enterprise, be it ship building, railroads, telephones, or computers.
Before you start whining about monopolies next time, just thing about the fact that the same telephone can work anywhere in the US, or that all electrical sockets in this country can likewise be used anywhere else in the country. Also consider the usefulness of anyone being able to go to a store and buy a piece of software for their computer.
Standards bodies take way too long, and often dont produce useful results. Look at how long it took to make CD standards, or DVD standards, or (re)writable DVD standards. It may be annoying to early adopters, but its often best to just let the market decide. It worked for VHS- how many people use Betamax?
Manipulate the moderator system! Mod someone as "overrated" today.
The ultimate solution to fighting spam is realizing that there is no perfect solution. We all know that no matter what we, spammers find a way around it. So the issue is to stop looking for that so-called "ultimate solution" that's supposed to get rid of spam forever. If anything, it's going to take several different methods to eliminate spam and there's going to be some trial and error.
And spam filters are a bandage over a sore that's being seriously neglected. I think the problem that people don't realize is that with spam, the client is limited to what he/she can do.
Yahoo might be going against standards, but they are on the right track by trying to tackle the problem from server side.
I think using AI would have some real benefits on mail servers. AI has the ability to learn. Filters on the other hand require reconfiguration to combat the ever changing spamaflouge.
I don't remember who this quote is from, or whether I remember it 100% correctly, but it's great:
"To every challenging problem, there is a solution that is obvious, easy, and wrong."
Proprietary stuff like this one usually is that solution, because not enough eyes looked at it. That's why so many software projects fail, and that's why peer-review is so important in science.
Yahoo can't even teach their mailservers to play nicely with the rest of the world (they bounce when they should have rejected). I don't trust them an inch to patch sendmail or solve the spam problem.
Assorted stuff I do sometimes: Lemuria.org
mod this troll if you want but this problem is very serious. If some idiot decides they can make money by poluting the water supply, you stop them. If they don't stop, you incarcerate them. To incarcerate them, you need an uncorrupt federal system and international cooperation - two oxymorons. If they're still poluting your drinking water, you shoot them. Lynch mobs are the only solution. If just one spammer in the US was (hideously and publically) strung up, the level of spam would fall. For Europe, the same. In Asia and Russia, just make a cash deal with the mob(s).
Mail servers that have the "nerve" to bounce mail do so in a predictable manner. Normally with a phrase such as "could not be delievered" or "rejected."
Instead of freaking out, take the time to actually look at bounced messages and find tells so you can filter them out. Those 100% unqiue tells are there.
"I'll never see the bounce."
You will if you allow the tells your mailserver uses to pass through. Or give it a unique bounce message that gets past your filter.
Trackable e-mail requires that everyone or no one do it. I'm certainly not going to. I have better ways to deal with spam. If you do it, you'll still be getting bounces from mail forged with your domain sent to mail servers that don't check.
Like it or not, you need to deal with it. If you don't have enough control, fire up your own mail server that you do have control over.
Ben
Work Safe Porn
Here's one system that I think could work:
Each E-mail sent can optionally contain a micropayment, cryptographically tied to the receiver's E-mail address and the contents of the E-mail.
When I receive E-mails, I can choose to ignore or simply spam-filter any E-mails with a value of less than X (I decide what X is).
The default action is to return the micropayment to the sender, if nothing is done within a week (or a few days) of sending the E-mail. This way, sending payments to someone who is not part of the system will effectively be a no-op.
The receiver has several possibilities:
Ignore the payment (the sender eventually gets his deposit back)
Return the payment immediately
Collect the payment
The way I would use this would be to collect the payment on any unsolicited commercial E-mails that I read (thus making sending SPAM cost money) and return/ignore all the payments from friends & other valid sources.
You could still send E-mails with no monetary value, but they would be subject to strict filtering.
I would probably set a filter limit of 5-10 cents/E-mail and only collect the money (if any) on real spam.
The system would provide income to those who run the banking, because they would get the interest on the deposits made by E-mail users.
At first, implementing something like this would have little impact on our E-mailing, because only a few people would be using the system. If it ever became widely adapted, we would have an E-mail system where sending spam is too expensive to be worthwhile and where regular E-mail would still be free (except for the loss of interest on the deposit made to send micropaid E-mails).
I don't believe this is proprietary. Yahoo is releasing a patch for Sendmail. AFAI can tell, while they're funding the dev work (because the spam rate is killing them), they aren't trying to milk this for more money.
One major problem with standards groups is that people like Verisign are on most security standards groups. Verisign has extremely strong motivations to ensure that email uses a Web-like interface, where one purchases an (expiring) Verisign cert for each email server one runs. They have strong incentive to block competing solutions. If you want to come out with a good system that prevents existing folks from milking a market, both industry consortiums and standards groups are pretty much useless. You need to do what happened with PNG -- have a bunch of talented, aggravated engineers sit down, write up a technically good spec, and put out reference code. Later on, let standards committees follow what's in place.
I can't figure out why replay attacks are an issue. I, personally, would suggest, off the cuff, including any To: or CC: lines in the message body (just for signing purposes, not actually sending either header in the body). This way, a replay attack would only allow resending the same email to the same destination from the same source. It's also pretty easy to include a timestamp, if folks are *really* concerned about replays.
Yahoo is pretty much doing what ESR and RMS have been hoping for for years -- contributing to open source systems because there's an itch that needs scratching.
Paul Vixie (disclaimer -- I don't move in his circles, and what I know about him is entirely secondhand) seems to be involved a great deal in politics, rather than technology. He leaves a bit of the same bitter tang in the mouth that Verisign does. He is, apparently, the source of at least some of the IETF objections. Vixie has also made a number of antispam statements that I tend to disagree with, including advocating mass blocking of mail servers on home email connections by netblock.
May we never see th
This is called SPF. It has a number of security flaws and shortcomings. You can find my comments on it during the last few Slashdot SPF stories. During the last Slashdot discussion, someone brought up a new DoS attack that could be executed using it.
I would *strongly* advise against implementing SPF. I consider the system fundamentally flawed, but even if someone can deal with that, at least some of the more glaring problems, like using DNS as a transport mechanism, should be fixed before anyone considers using it.
The Yahoo approach (apparently PKI, need to read up on it) is probably more work to implement, but also probably fixes the problem properly.
At the best, SPF is another hack that will grant a decrease in spam for a few months (and then leave cruft and mucked-up mailservers around for years and years to come).
May we never see th
I can't agree that SPF is particularly useful here.
So in order for a spammer to spam someone using your actual e-mail address they have to:
1) hack into your domain's outbound mail server and send e-mail from there (nothing new in this risk)
Okay, fair enough.
2) hijack/trojan your machine or a machine in your organization and then route e-mails through the official SMTP server (same as what happens now, except that the mail admin is more likely to notice that customer 32432's account is sending gobs of e-mail)
Note that the only way a spammer would be forced to go through the local SMTP server is if *everyone* is properly implementing SPF everywhere. It only takes a single misconfigured server. Frankly, the problem of making SPF work Internet-wide is a superset of solving the open relay problem (i.e. ensuring that all mail servers that can send you mail are properly configured not to allow non-customers to dump mail through them). Solving the open relay problem provides all the benefits that SPF does. SPF hides the actual costs of its implementation with a lot of discussion of interesting features, but ultimately, it's not a particularly useful proposal.
3) poison the DNS SPF information (tough attack to pull off, can be combatted and might lead to new security in the DNS system)
This may be used in a positive (authorizing additional servers) or negative (deauthorizing authorized servers) manner. In the negative manner, it takes the form of a DoS. It is only hard to pull off for heavily-used mail routes (since only one lookup in a bazillon will actually generate a DNS query). It's still possible, and difficult and expensive to defend against, and a single success can have catastrophic results. It may also be used in a positive manner, to falsify SPF information. This is not particularly tough to pull off, as tools to automate the procedure will inevitably pop up shortly after folks start using SPF.
spammer goes in search of a domain that doesn't have reverse-MX info and forges that domain onto their e-mails
Another flaw in SPF. Much like the open relay problem, it requires *correct implementation Internet-wide* to work without holes. Every time someone's proposed a security system based on this, it has failed.
5) spammer starts to use throw-away domains at $X each
Trivial issue to bypass. Spammers frequently lose their accounts after a spam incident, which means they have to pay for a throwaway account. That's $20-$40. A throwaway domain adds only $10 to that cost. It just isn't significant -- spammers make more money than that per spam run.
May we never see th
The spam issue must be solved, whether by social, technological, legal or whatever else means, or a combination of these.
The sad truth is, there will always be jerks willing to engage in self-profitable activity at the expense of others, and to some extent this activity is what we call crime. There are three prerequisites for it, which are:
- intent (you know it's bad, but you don't care)
- gain (outweighing the cost / risk)
- occasion
This last one you completely overlooked. Why do you think locks exist ? Why do you think most countries ban civilians from owning firearms ? Because that will reduce the number of occasions someone has to commit crime.
Maybe we deserve this world ?
This approach has already been proposed in the form of SPF. While theoretically, with some additional infrastructure (modifications to both DNS and more significant modifications to mail servers), it's probably possible to do this in a secure manner, the approach SPF advocates is easily defeated. It also introduces reliability and performance issues.
May we never see th
Huh?!? AFAIK, CSS/Text was invented by Hakon Wium Lie, now CTO of Opera, in 1994. Or what do you mean by CSS/Text?
Employee of Inrupt, Project Release Manager and Community Manager for Solid
Frankly, I got about 1/3 of the above.
I use PostCastServer from my static-IP DSL to send Emails thanks to a flaky ISP E-mail server. I'm just concered that I'll be shut out of being able to, quite legimately, send E-mails and end up having to go back to getting 50% of my E-mails returned after 3 days as undeliverable.
On recieving em, my whitelist and baesian filter does fine.
First let me say I agree with your premise. I have never received an anonymous delivery, email or otherwise, that I desired.
But let me show the fallicy of yahoo's actions.
Yahoos step 1 is to reject forged headers. Forged headers was just made illegal by the Bush administration IIRC. I completely approve.
Yahoos step 2 is to force a signature on every email by the server. Interestingly, Step 2 removes the need for step 1 and makes you wonder if step 2 is their real desire. Note that a solid step 1 also removes the need for step 2, given that open relays are shut down.
This is where I disapprove.
This proposes the same problem as DRM. Who controls which signatures are accepted? Once again we are right back with Verisign, et al. So unless your server has a PURCHASED KEY from verisign, or the like, your server won't be sending email to yahoo or any of the ISPs that adopt this.
I promise they won't be suggesting PGP either And so the spiral begins. Yahoo sells the rights to the certificates it will accept on a yearly basis. Verisign subsells this right in the form of the infamous certificate chain.
So what if the code is free, the certificates are not!
And most important: Once SPF is widely adopted mail administrators can disallow mails from domains with no reverse-MX. If e.g. Yahoo, Hotmail and AOL chose to do this the remaining unwilling sysadmins will be forced to upgrade.
-- kryps
Ignores standards bodies - that's the first good thing - get the politics out of spam control... especially since a lot of the spam I do get is from the people trying to sell their anti-spam wares...
I get about a dozen emails per week from McAfee... subject may as well read 'Tired of getting spam from us? Pay up and we'll stop!'
Ok, I admit I don't know much about this subject, but maybe this thought has some merit, maybe not.
To send an email to someone, you must put a certain word or words in the subject line. The twist is that the word is obtained by looking at a picture that describes the word. So say there is an email address on a website, above that email address is a cartoon picture of a dog, to get the email to that person the word "dog" must be in the subject line. For any email that does not contain the word "dog" in it, an automated response is sent back to the sender with a message showing the picture of the dog and asking that this word be placed in the email subject to get the message through successfully. This works on the principle that computers have yet to attain certain abilities humans do, like pattern recognition of images.
The price of spam doesn't come anywhere near the value of privacy and freedom of speech
Could not have stated it more elegantly than that....
1) Make new address.
2) Give it to trustees.
3) Dumbass trustees send you a SpamGateGreetingCard using your new address (because the picture is so cute, and they get a freebie animated GIF).
4) SpamGateGreetingCard propagates your address to the scum of the planet.
5) Sucks to be you!
"Good news, everyone!"
(D) It is a simple measure to simply throw out any email that is not signed.
Yes, but the ISP is still paying for the bandwidth to receive (and then bounce) said email.
If you don't bounce, spam still costs the same amount. If you do bounce unsigned emails (as the RFCs say you must), then the cost of spam to you doubles.
Then you have to take into account that spammers will just buy a cert and keep spamming. So what if the cert gets revoked (IF it gets revoked) - they'll just buy another one.
As a mail server admin, I don't think I'll be implementing this any time soon. It will be a pain in the ass, will cost me money, and won't stop spam.
Anyway, I was just thinking of a scheme last night to verify the origin of emails.
The idea is that a domain holder runs an server which maintains an index of valid emails for that domain, which receiving servers may verify a message against.
In summary, this is how it would work:
Now to expand on the above points.
It is, of course, vital that the checksum be added to the index *before* the mail is sent.
The checksum should be held on the index server for a certain amount of time, maybe 5+1 days, or whatever the RFCs say about max deliver attempts.
The end-user *could* carry out the authentication himself, should his ISP not support it for example, but this would not be ideal.
When verifying, the index server would be queried with the checksum, and optionally the current datetime.
The option to include the current datetime is to allow end-users' email clients/spam filters to carry out the check. Since somebody may have been on holiday for a week, the checksum will have expired before the client checks it.
For this reason, the client should only check those emails which have a date less than the expiry time. Should the local clock be slow however, this would lead to expired mails being checked. By including the current datetime, the index server can detect an offset, and respond with a positive should the client be too far behind. It is of course imperitive that the index server have the correct time.
Mailservers would normally have no need to worry about this, so the datetime would not be included in their request,
Vs lbh pna ernq guvf, ybt bss abj. Tb bhgfvqr. Syl n xvgr.
The next level for filtering is looking at the body of the incoming messages and excluding any html formatted messages (first), any containing graphics in the body (second), and then applying common word filters (such as .biz, PeNiS, enlargement, pills, meds, nigeria, etc.
Works almost to the extreme. Just a few sendmail hacks and you are there.
I SMELL DDoS!!! Kill 'em all!
:)
Why not? They're sending you unrequested data, so, just send them unrequested data.
You're being as wrong as they, but at least, they asked for. If servers that allow spam gets falling, less and less servers will be so misconfigured to allow it
Just DO IT already and switch to IPv6. Everyone who has a net connection gets an IP address that is theirs. They send spam, it is identifiable as being from Joe Blow's IP address. Go have a talk with Joe Blow, perhaps with a baseball bat (to emphasize certain talking points, you understand).
In Bushworld, they struggle to keep church and state separate in Iraq as they increasingly merge the two in America.
"Instead of sending the whole email content - and with it the ability to falsify email header information, why not just send the email header only - and require the originating server to hold the email content?"
I thought about this once. How do you ensure that the person coming back to get the mail off the originating server is indeed the person to whom the mail was sent? I don't think any certificate-based scheme is the answer, because key management is so complicated. At least in their current incarnations. (If there was something as simple as ssh-keygen that also posted the public key to the ISP, it might be workable. But there isn't, and there's the question of mail client support.)
Regarding the "It shows your IP address" that mabu pointed out, wouldn't it be trivial to proxy the "get the content" message through the ISP's MTA?
__CmdrTHAC0__
In Soviet Russia, Spanish Inquisition doesn't expect YOU!!
I, myself, welcome our new Yahoo Overlords. Something - anything needs to be done to combat this, and first and foremost should be a technical solution. Therefore, if standards bodies are unwilling or unable to stop the endless flow of offers of penile enlargement or whatever the hell that shit is (I don't know, Outlook 2003 does a nice job of breaking HTML messages), someone should be. And by delivering software that plugs into sendmail, yahoo isn't trying to make money on shipping some new revolutionary spam filter, they are trying to help the community as a whole.
those are my 2 cents
The SPF guys are pretty clueful. There's a large number of smart people who have spent a lot of thought on it; they haven't missed anything obvious.
this guy is clueless or a troll. if clueless, it's simple to educate yourself about spf instead of trying to look smart when you aren't.
If everybody used a list of acceptable senders, I doubt there will be much space for spam. Most other solutions whould require some form of central control, which I'm against. :)
The only issue is with mail coming from people you don't have in the address book yet.
I guess the combination of a private and a public e-mail account is the solution we all use given the technology now.
Couldn't this be a solution for everyone ? Ok.... at least most
=v= Yahoo! has a massive problem on its hands. Porn spammers have created hundreds of Yahoo! IDs and subscribed them to an unknown number of "open membership" Yahoo! Groups. They sit there, lurking, until one day they spam the groups. The groups' moderators never know until after the damage is done, at which point they can unsubscribe the ID, but another one's already there to send the next spam.
This problem could be solved by tracking down the IDs who've done this and seeing what others are related (e.g. by creation date, IP address, etc.), and either turning them off or putting them on some sort of individual probation. Yahoo! can do this, and indeed are the only ones who can do it. But they instead pass the buck to the unempowered moderators.
So instead they're trying to deploy Yet Another key system?
Anyone going to SpamCon 2004? It's in Boston and it's free. Wonder if they have a "spot the spammer, win a T-shirt" contest.
How exactly is this better than SMTP STARTTLS, which is already standardized and widely available?
Everyone knows that tech standards should be written by Intel.
What makes something a standard is many people choosing to accept it. Most of the buses in your PC, and probably your instruction set as well, are standards essentially propogated by Intel. If Yahoo and AOL want to get together and say "Hey, this spam thing sucks, we're going to do this, and whose with us?" what's wrong with that? Probably tons more effective than a standards body trying to convince them.
If the standard sucks, then don't use it.
paintball
The fact is that anyone can raise a new standard, it will have to do something useful or it will simply be ignored, but it is hardly difficult to get the process started, by raising an Internet Draft, and in a case like this it should only take a few months to become a standard. The IETF work much more efficiently than any commercial standards body that I know of. The process is documented at ftp://ftp.isi.edu/in-notes/rfc2026.txt amongst other places, and surely must be the correct procedure to use. Who cases about ANSI, or BSI, or CENELEC, or any of these bodies that sell you a few pages of copyrighted standard for silly money? The RFCs are published for everyone to use, which is why ithe net works as well as it does, despite the efforts and intentions of some, such as the Convicted Monopolist (had to get him in somewhere..), to "de-commoditise the protocols".
There is no reason why they can't raise an Internet Draft right now and start using the thing, people can then follow the Draft at their own risk of having to do more work if it changes.
"tell those who don't want their inbox to be full of crap to 'get bent'."
If that were in any way shape or form an accurate representation of my ideas then why would I be plugging an idea which gets rid of spam?
I'm telling those who want to destroy privacy and/or cause massive collateral damage by blacklisting more innocent IPs than spammers in their pursuit to block spam to get bent. There's a big difference you're apparently too illiterate or too much of a troll to understand.
Most likely both. It's a method commonly used on the internet to flat out lie about what people have to say in the hopes that people will simply assume they aren't full of shit (which you are) and not bother to read the source material for themselves.
This is Slashdot afterall. Nobody reads the article.
"I think you should try and consider the other side's argument for a while."
You obviously have no clue which side I'm arguing. It's the sensible and effective side to blocking spam. Apparently you want me to consider the nonsensical and ineffectual methodology to blocking spam.
And I did. I pointed out it's ineffectual and nonsensical.
Ben
Work Safe Porn
I get about 20 spams/day (stopped at the corporate quarantine for email) and about 10 legitimate emails/day (almost all internal and hence easy to see). The only place that my email has been public is Ebay - and the increase in spam correlates with my purchase of a fountain pen there, so I'm almost certain that harvesting via Ebay is the main source of my spam. I have given email out to some companies, but I don't think I've gotten spam because of them. I don't have a blog or a web page, so they aren't sources of spam, and I don't think I'm on a web page with email exposed. (It could also be through Yahoo, but I don't check that much, and they're good about spam sorting).
It doesn't take much to get significant spam now - for me, once was enough. Your comments may be more true of real mail, where barrier to entry and cost of mailings are higher, but for spam it doesn't seem to take much effort/error to get a lot of spam.
I agree that certs at the domain level are probably reasonable
Would you rather pay $400/year for an e-mail sending certificate (assuming similar pricing to Veri$ign's SSL certificates) or give up the right to send MAIL FROM your small business's domain?
FYI, AOL already does reverse-MX whitelisting. If your domain sends a large volume of e-mail into their system, you have to list the IP addresses of your outbound mail servers with them. (Otherwise the mail gets dropped to /dev/null.)
AOL's probably interested in SPF mainly because it means they won't have to do all of the manual processing that they do now. At least, they won't have to manually keep track of domains and outbound IPs. (Instead, they'll just query the SPF record for the domain.)
Backing of the large ISPs will definitely do a lot to either make/break any of the source verification systems such as DomainKeys or SPF.
Wolde you bothe eate your cake, and have your cake?
It's a lot less trollish if you actually address the argument I was making instead of arguing against things I never said.
If you want to argue that I'm wrong for saying the grass is pink when I actually said it was green then you're going to have to go at it alone.
You're an idiot. You offered nothing constructive and simply accused me of saying things I didn't say.
Don't act all shocked I wasn't fooled. Unlike you, I know what I said in the article.
Did you think I forgot? I'm the one who wrote it.
Ben
Work Safe Porn
if you post your info publically.
Anything you post in a newsgroup/on-line is public information.
Nobody is forcing you to use a valid e-mail address for those things. And you have no expectation of privacy with newsgroups.
Ben
Work Safe Porn
To Fast. I could see 2 to 3 Years but just stage 1 could reduce False Postives for those users who support they tech.
I would propose that a group of people are selected around the world to manually go through their incoming email and note which emails are spam, preferably qualifying what type it is and using some simple tools to also note whether this is the work of nefarious arch-spammer types that play tricks on you, as opposed to honest mailing lists. ...
...
An ISP subscribing to one or more of these realtime email filters (only a blackhole at the single email level) would be able to refuse acceptance of the email.
If a distributed quick response system is implemented (it seems pretty simple technically) we could effectively neutralize an outgoing spam stream within minutes or even seconds of its beginning.
Sounds like you invented the DNS Blacklist. If distributed response is what you're looking for, you've got DCC and Vilpul's Razor on the free side, and Brightmail on the commercial side.
Neutralizing within seconds would probably have to be done at the egress point, i.e. the immediate upstream of the spam source. If they had their act together though, there probably wouldn't be all the spam coming out of there in the first place.
I don't think these solutions are going to close down spammers the way I suggested in my vehement post, but they (especially Vipul's Razor I believe) have a lot of the things I was dreaming about!
Vipul's human network looks great though it seems that it might miss spam which personalizes itself per user (just a theory witout trying it, which I will) and DCC looks interesting in that it has fuzzy matching that tries to evolve with spam. Brightmail's claim of 99.999% accuracy is pretty good, I suppose it would have to have humans in there somewhere. Anyway I'll check out Vipul and see what happens.
I think it would be very cool if companies were given money by governments to develop or implement antispam technologies, in addition to the other suggestions I had. Thanks again and here's to no more spam.
As a few people noted on this thread, the use of spam (specifically, the filter-cracking gibberish routinely appended to spam) as a terrorist comm channel would be an excellent way to evade traffic analysis.
If it isn't illegal, they can't be put out of business
99% of the spam I've ever seen is illegal on its face (fraud, illicit sale of prescription drugs, unauthorized commercial use of trademarks and copyrights, distribution of pornography to minors, etc).
And maybe our antispam net could benefit from time to time by a friendly security officer geek who also gets too much spam on his yahoo account at home and has gotten pissed off!
Suggestion: If you fit this description and are reading this, write a memo describing the use of spam as a comm channel immune to traffic analysis and get it into the record. This will give your agency the choice of 1)investigating spammers using their obvious violations of existing laws as leverage, or 2)potentially becoming the scapegoat it it turns out that terrorists do pull off another attack with the aid of this technique.
/. If the government wants us to respect the law, it should set a better example.