Slashdot Mirror
Yahoo and Unilateral Anti-Spam Technology?
EatenByAGrue asks: "According to this Business Week article, Yahoo is planning on distributing a toolkit for Sendmail and other mail daemons that adds an encrypted source domain key to email headers to verify where they came from. However, critics are concerned that the scheme will be easily bypassed and that it ignores standards bodies. What does the Slashdot community (representing countless email admins, I'm sure) think of this proposal? On one hand, its a commercial enterprise dictating standard technology, on the other hand, the standards bodies have proven themselves helpless and hopeless when it comes to providing solutions."
easy email tracking system will be gladly welcomed by police and other agencies...
This Is Not a Sig
I try to be as standards compliant with my mail servers as is humanly possible. Even with numerous spam filters, I get about 10 legitimate email messages a day and 100 spams. Something has got to change.
Whether it is this technology, or another, something has got to be done. I'll implement this and hope that other admins do the same.
-sirket
I think this is a good move on Yahoo!'s part. As a developer I think a solution that is available and 50% effective is better than a solution that no one has implemented yet.
Lets get the implementations out there in the wild and use the feedback to create real solutions!
These days I can't even open by inbox, it is so overflowing with spam. I'm exaggerating, but at some point email is going to become completely useless because of spam. I do a lot of business over telephone (the way I used to do it before email) and have an ftp site to which customers can copy shared files.
It's slower, but not as slow as deleted emails that I never see and can't respond to.
I have been pwned because my
It's important for standards organizations to be taken seriously if people want to actually see careful and appropriate change made. We could, I suppose, say that the W3C is completely useless because Microsoft essentially dictates what will and will not be a standard on the majority of platforms but that doesn't make the W3C any more useless. Actually, it makes it much more important to look for a body that can develop RFC's and such so that we can all look at the proposed solutions and say yes or no. When a corporation decides on something it just happens and all we have to fall upon to stop the adoption of a (potentially) damaging standard is the free market system. However, in this situation that wouldn't have much of a bearing on a system that doesn't technically bring Yahoo! any more revenue.
Web folk always moan about MSIE's poor standards complience, for instance, but forget that CSS/Text came from them -- Netscape was pushing CSS/JavaScript at the time. Now, one of those is a standard, and the other is dead.
Ultimitely, either people will like Yahoo's idea and adopt it and it will eventually become a new standard, or it will be ignored by everyone else and forgotten. Only time will tell.
The extra key could be used by anybody who wants to, and ignored by the rest. And their implementation is open-source, so it doesn't look like a way of making an end-run past other ISPs. And since many spam messages come from fake Yahoo email id's, this would be a great way to immediately filter out those ones: if it says Yahoo but doesn't carry a key-->SPAM bin
I like the idea of a major player getting on with it and DOING something.
Would we rather have MS dictating an anti-spam standard? You can be sure such a beast would be a lot less benign than Yahoo's proposal
"From" address from what your SMTP server is, in which case I don't see how it could work for you.
This may put a lot of travellers out in the cold.
A solution is badly needed, but it has to work for everybody.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
...de facto standards emerge. One need look no further than POSIX/SUS and GNU/Linux for an example.
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
Doesn't sound like this will be too effective in stopping spam for
Yahoo users, and Yahoo is already a pain
to work with.
I setup a proxy and was a spam relay (unknowingly of course) for just
under a week. I got blacklisted on a couple of email sites, my ISP
bitched and I fixed it. So sorry.
So I'm now off every blacklist I know of, and everyone loves me again.
That is except Yahoo, the evil nazi bastards. I've filled out their
stupid, "fill this out to get
un-blacklisted" form at least 30 times (twice a day normally).
It must go into a black hole because they still are rejecting my mail.
Everyone else lets me through but stupid Yahoo, who seem to have NO
admins, no technical people, and a violate once banned for life reject
policy. Grrr. So I guess, if this new system lets them drop their damn
overbearing blacklists, I'm all for it.
Now that RIAA has gotten rid of Napster and trying to crack down, what did most people do? Other programers created other way to share music. Now all of this was just so we could get free music. These spammers are making money at what they do. How hard are they going to try and find a way to mail in our inbox? What we need to do is find a way to keep spammers from making money. That would stop them.
It would be much simpler to add a record type to DNS servers to identify **outgoing** mail servers. Email proxies, where 60% of all spam comes from, would be immediately eliminated. Spammers with fixed servers and addresses are easily taken care of by the RBLs. Why introduce something that is more complicated and less reliable?
I admin a dozen domains professionally, and run a couple mail servers for volunteer orgs and all of them will get it.
-Brian
Yes but we will never have a social solution when all it takes is 0.000002% of the worlds population to be spammers.
There's always going to be pricks who will do anything for a buck.
You can't expect to wield supreme executive power, just because some watery tart threw a sword at you
"...on the other hand, the standards bodies have proven themselves helpless and hopeless when it comes to providing solutions."
E-mail is supposed to do a certain job, and it does that job well, at least from a technical standpoint. The problems with spam are identical to similar problems in every other arena, it's just that they seem worse because of the level of automation. Even if it wasn't automated, spam would still be a problem. With idiots knocking on my door every other week with a hard sale for everything from oil changes to chinese food, I'm starting to almost regret the do-not-call list, because I didn't have to worry as much about these degenerates (if you don't take "No" for an answer and walk away immmediately, you are a degenerate in my book, and very door-to-door jerkwad so far has been one) giving my wife a hard time.
Standards bodies can't do anything to fix human behavior, unfortunately.
A far beter approach (which I think I saw on Slashdot but can't remember) is to use an extension which says whether IP addresses are allowed to use a domain.
This extension was based on DNS and basically allowed the mail server to query whether the IP address of the mailer was allowed to send on behalf of the domain.
Yes - this would be open to IP spoofing. Perhaps this DNS extension should be combined with the Yahoo method. If Yahoo, Hotmail and a couple of other providers adopted it could have massive effect.
To intially put live perhaps they could have an authenticated vs non-authenticated flag/filter in their web-mail client.
There were alot of vital ascpects to this point made in the previous article some of which are quite thought provoking!
If you missed the previous thread, I hgihly recommended reading or even reading it.
Never try to beat a professional at his own game!
I'm all for a spam solution coming from private enterprise as opposed to legislation- in fact, I think the former is the only method that has a chance of working. Maybe Yahoo's attempt will help, maybe they'll waste a bunch of money trying, but I guaruntee it's less money and less waste than Congress or the FCC doing the same thing.
I mostly disagree with the parent.
I agree that spam is a social problem, but you need to qualify what you mean a little more. Technology is the enabling mechanism to this problem (that some people are willing to be jerks and abuse a medium). Computers are exceedingly good at cranking out spam, day and night, and the medium of email is exceedingly weak against protecting against this kind of abuse. The same kind of social problem exists in all communications mediums, but you don't see just anyone wardialing people to sell viagra and penis pills. Calling a million people is expensive and time consuming, spamming is not. Therefore, this is a technologically exagerated (sp?) manifestation of a very minor social problem, making your point all but useless when trying to solve it. You've got to solve the problem in this situation, which is the enabler - technology.
$45 per U Colocation Special
we will never have a social solution
Three words:
Tar and Feathers
That way, there's no question where the email came from, and exactly which account sent it. Plus traffic goes way down by not passing the content all over the place.
In addition millions of copies of the same email would not have to be held on recipient's servers, they would just sit on the originating server until received or until some time limit expired.
I guess this would prohibit using a (ISP's) email server as a repository, you would have to download everything you wanted to keep, but hey, no more email size limits! - send me the world - if I want it, I'll come and get it!
Could this help in the spam wars?
I'm agin it. Cause problems. Will not fix SPAM. I have however added SPF records to my DNS. More flexible solution. I'll get around to patching my MTA to reject invalid incomming in good time.
Development of a workable solution, that is.
There have been a few times in the past where an entrenched technology has hit a wall in functionality, but because it was entrenched no one really did anything about it.
Then, someone said "Fuck standards - I have to DO something about this!" and started pushing thier solution. Other saw that someone was willing to take the first step, and took a step themselves. After some shakeouts, a new, more functional standard emerged.
My hope is that Yahoo has started the "SPAM proof MTA" development war for real this time. I want my e-mail system back.
Soko
"Depression is merely anger without enthusiasm." - Anonymous
I don't think so. I think a bad and poorly designed solution is worse than no solution. Especially when there is other competing solutions, which are argueably better, or at least equal to Yahoo!'s domain keys system, such as RMX. IMHO, Domain Keys offers no significant improvements to the spam problem, but rather adds a crypto overhead to the sending and receiving of every message. I think it is great that Yahoo is trying to innovate to stop the SPAM problem, but being cavalier and going at it by themselves is not the answer, especially when they have a great Anti-spam alliance with AOL and MS.
You mean like "reverse MX" records... google for RMX, SMTP+SPF, DRIP, DMX. (SPF seems to have momentum at the moment)
However, reverse-MX solutions will not kill off spam (a common mis-conception). The goal of reverse-MX proposals is to stop domain forgery where spammers are able to, with complete impunity, to tack on any old domain name to their spams. Which means that the unfortunate organization who is forged gets to deal with the thousands of e-mail bounces and the irate phone calls / e-mails from people who think that the organization was the source of the spam. As a mail admin, I'm able to control which servers handle inbound e-mail for my domain through specifying MX records. Reverse MX allows me to have the same amount of control over outbound e-mail from my domain.
What will happen instead, once reverse-MX systems (or Yahoo!'s system or other sender-authentication systems) come into play. Spammers will have to change tactics and resort to either forging one of the remaining domains that don't have reverse-MX information published, or they will register throw-away domains by the hundreds. It will drive up their costs a tiny bit (much like the impact of bayesian and other filters requiring them to use randomization techniques).
But the real nice side-effect of reverse-MX, etc., is that you'll be able to more reliable whitelist based on domain name. And your bayesian filters will be able to assign high ham values to domain names.
It also puts a crimp in e-mail worms that attempt to use a built-in SMTP engine to avoid detection. Unless the worm forges a domain with no reverse-MX info published, the worm won't spread (most MTAs will drop the connection). Instead, the worm will have to route through the user domain's SMTP server, where the mail admin is more likely to catch the traffic (virus scanner on the SMTP server, or rate limiters).
Wolde you bothe eate your cake, and have your cake?
No, spam is an ECONOMIC problem, not a social problem. You're never going to get "perfect" people who always act morally.
We pretend email is free, so the spammers think they are dividing by zero--and any return on zero investment looks very impressive. This is actually a silly legacy of when the nascent Internet was a non-commercial and purely cooperative enterprise. "You help me with my email and I'll help you with yours. We just won't worry about the details of the bean counting."
Now the spammers say "You help me with my 10 million emails and maybe I'll find a sucker who'll send me $10!"
Freedom = (Meaningful - Coerced) Choice != (Speech | Beer^2), and sad sock puppets' bad mods avail them naught.
I would probably implement all of this on my mail servers except for one critical flaw, they only mention sendmail and qmail support (and presumably exchange as well). I use exim b/c I like the filtering options (and a friend of mine highly recommended it).
If they don't support exim, then I can't use it. Exim developers may implement it, but yahoo can't resonably say that they would start blocking before other projects have a chance to make their own versions.
On the other side of things, I'm going to start with the spf's shortly.
-CPM
---You're all I need, When the water runs deep, You're all I need, Now I cry my soul to sleep -- Collective Soul, Needs
But the solution is technological. Why can't we use technology and updated standards to close the gaping holes that currently exist?
Pain is a powerful motivator...
Reverse MX and Yahoo!'s proposal, however, don't require widespread adoption at the start. In fact, the tipping point is probably only a few percentage points of the domain namespace.
After all, for just a few minutes worth of work (more if you don't already provide SMTP AUTH, or require users to VPN in to send e-mail already), you protect your domain against joe jobs and forged e-mail bounces. So there's a low cost-of-entry. (Yahoo!'s proposal requires more work then the simpler, less CPU-intensive SPF proposal.)
What happens next is that domain admins that publish keys/SPF information find that they're no longer getting joe-jobbed and they're able to block a higher percentage of spam then they used to. Word gets out and more folks sign on (second wave adopters).
Sometime after that, the big ISPs require your mail servers to publish SPF/keys if you want your e-mail to be delivered to their users. (FYI, this is very similar to AOL's whitelisting program, which is essential a privately-administered reverse-MX system where you tell AOL what IPs your e-mail is allowed to originate from.)
As a WAG about rate of pickup, early adopters have started, second wave folks will probably sign on in the spring/summer, and I wouldn't be surprised to see ISP-blocking by the end of the year.
Wolde you bothe eate your cake, and have your cake?
This comment isn't insightful, it's stupid.
So if spam is a social problem, what about auto theft? Should that also be solved by economics and/or behavior? Do you think that people shouldn't lock their cars or have alarm systems? Or that they should have push-button starters with no key required? If you believe this, you're a fool.
How about hacking? Should that also be solved by economics and/or behavior? Should remotely-accessible computer systems not be password protected? Instead of having user accounts with passwords to keep hackers out, should we just let anyone log in who wants to, and use other means to punish people who abuse this? How about we connect our military systems to the internet in this way? Again, if you believe this, you're a fool.
Any time a technological measure can be employed to minimize a social problem, it should be, because relying on society to proactively halt the activities of those who prey on weaknesses in the society is foolhardy because society only acts in a reactionary manner.
Also, we've all seen discussions in projects where many people propose solutions in the abstract but to get real cred a solution has to be proposed as working code. Nothing gets implemented quite as fast as working code.
KLAATU, BORADA, NIh*ahem*
Nothing new needs to be invented here. What we should all be pushing for is signed email. There are many advantages to signed email, but here are the most relevant:
(A) Signed email signs not just the message headers, but also the message body. No chance of header substitution.
(B) Signed email associates signatures with some certificate chain and, presumably, a CRL (Certificate Revocation List). Abuses can lead to certificates being revoked.
(C) Because of the certificate chain, there is a chain of trust. There is always SOMEONE to sue!
(D) It is a simple measure to simply throw out any email that is not signed.
(E) Because of esign legislation, signed emails can be considered legally binding. In other words, lies, misrepresentations, libel, etc... in signed emails provides you with grounds for prosecution in courts of law - as if the signer wrote you the document and signed his name at the bottom (and yes, they can also be used for legally binding contracts and whatnot).
There is an issue with "Crossing the chasm" with signed email, of course. It would require a body such as AOL and/or Yahoo rising up and providing signature filters on incoming email to force such a solution into the mainstream. But once this is done, SPAM will practically dissappear. And any SPAM that comes in through signed channels can be dealt with in a satisfactory way.
I do not believe this harms any of us, btw...
You want privacy? The same techniques that allow you to sign email also allows you to encrypt email to your destination.
Worried about anonymity? Certificates can be issued that authenticate an email address without full disclosure of the owner of that address (but this may not be satisfactory for stopping abuses). Anonymity and stopping SPAM may, unfortunately, be mutually exclusive goals.... Any thoughts?
The reason that it can be true that 1+1 > 2 is that very peculiar nonzero value of the + operator
Especially considering how promising the OSS model is, why can't we create a solution? We talk about the complexity of the problem, the importance of not breaking standards, etc. Who FUCKING cares if I can't check my email because it totally FUCKING BURIED in unsolicited junk...
I don't mean to come off as the thundering asshole, but this situation has grown so slowly its like watching a car crash spread out over the past 15 YEARS.
Please, experiment. Break things. I don't give a shit, but don't let us sit here moaning like helpless children while spammers sit back (laugh) and rake in MILLIONS.
Get fucking aggressive.
And if I hear one more idiot talk about how you have to cut spammers off by not buying their products I'm going to cut him off at the knees! If that would work you and Noah could be shooting dice right now and we'd have a hell of a lot less to worry about.
Programers still know how to experiment, right?
Quack, quack.
This would a spammers wet dream.
They would write their own mail servers where more than one recpient would be linked to one post on the server. This means that they can send a small header it to a gazillion people and only spend 400 bytes on actually storing the message on their server since they only need one copy of a particular Email.
Bandwith is only wasted when a user comes to look at the mail, which also verifies that that user exists (double spam for you my friend).
So, this would make spam worse.
so in short
1) spammers could send at least twice as much spam as they can now.
2) they will get much better verification that the mail address they had is correct.
The Internet is full. Go Away!!!
Anyone with experience with these standardization bodies knows that all of the complaining has to do with who's ideas win and who's name ends up on the standards documents. It's a particularly virulent form of academic arrogance. Solutions for signed email to stop SPAM are almost as old as email. Trust me, nothing is ever going to happen if one of the big guys doesn't put their ass on the line.
While the guys at the IETF fight for who has the biggest, ahem..., pen, the known email universe is collapsing under the weight of SPAM.
Let Yahoo hack and slash their way to a solution that works and then the standardization megalomaniacs can claim credit for inventing that idea 15 years ago while undergraduates at Stanford, Cambridge and MIT...
In the meantime, maybe we can have some peace...
The reason that it can be true that 1+1 > 2 is that very peculiar nonzero value of the + operator
SMTP relays need to be licensed and regulated.
Ummm... and who do you propose is going to do the licensing and regulations? What enforcement powers will they have over relays in another jurisdiction?
What's to stop the spammers from bribing officials to get their spam-relays "licensed"?
Wolde you bothe eate your cake, and have your cake?
But Spam is more about an inappropriate use of technology. SMTP was designed on the assumption that the community at large using it would not be interested in abusing it. This was the case back when the Internet was not yet commercialized, and I remember it pretty well.
I think the only thing that will resolve the spam issue is abandonment of SMTP as we know it, and an adoption of a new protocol that enforces traceability. This is nothing new or scary - the IP numbers are all tracked and the BGP tables that run the internet all provide traceability to the source. Even though your average Joe might not be able to do it, but ISP's cooperate and exchange this info all the time on NOC-to-NOC basis to identify sources of trouble.
A similar system will need to exist for mail, that will require some sort of a registration and compliance to join the "mail provider" network, whatever that will be. As soon as the e-mail becomes traceable to the source, perhaps even if not with 100% accuracy, there will be a drastic reduction in spam.
Second problem is all those exploitable zombie Windows boxen out there, but I don't know what to suggest here...
The trouble with spam is the forged return addresses. If spammers were forced to use real email addresses:
;) (Kidding... his is way eaiser to update than mine) I think that implementing something like this on every ISP in America would immediately kill spam as we know it.
1) It would be much easier to block spam
2) It would be much easier to get their accounts revoked.
A friend of mine runs a script which ensures every email he reads is a real address. Essentially, he's got a cure for spam.
He has a script running on his mail that replies to every email he gets with a confirmation code. When the end user replies with that confirmation code (all it takes is hitting ctrl-r and ctrl-enter) that email address is adding to his "verified email address" list, and the original email goes through.
He doesn't even look at emails that aren't confirmed yet.
If we could get this implemented on a systematic level (such as via confirmaiton reciepts automatically & transparently handed by the Mozilla mail client) it would essentially end free for all spam as we know it. And it doesn't require rewriting the RFCs or adding new headers, or whatever. It would work with any mail reader... though adding in transparency would require updating people's mail clients.
The downsides:
-Two extra emails for every one original email are sent... but only the first time. After the email address is verified, it doesn't need another confirmation. If this is implemented system wide, the savings in the reduction of spam messages would greatly outweigh the extra cost on the network.
-People who do not confirm don't let their email get through. This happened to me the first time I mailed him after he installed his system. I send him an email, and went home for the day. Didn't see he didn't recieve it until I checked my mail again. Mail clients that handle confirmation transparently would (nearly) solve this problem.
As someone who has experience writing spam filters (I wrote a pretty good neural net spam filter way before that Graham fellow wrote his bayesian filter, that publicity hog!
Shame they move so slowly... and never can agree on how to implement anything...
-Bill Kerney
In all seriousness. How much spam can you possibly be getting?
I keep hearing horror stories about people getting 100+ spam emails per day. This leaves me with the question, HOW IS YOUR EMAIL ADDRESS GETTING INTO THEIR HANDS!?!?
I don't sign up for every "free" offer that I come across. I don't have business cards made up with my email address. I have two email addresses, I might receive 10 spams per week between them.
WTF are all of you doing to get on so many spammers' lists?
LK
"Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
I remember a day when e-mail was nearly Spam-free, and Spammers only got away with it once. That was back in the mid-90s on the Prodigy Interactive Service, before they had opened their mail system to the Internet. When there was a closed system that required a vaild credit card to open a master account, and accounts who abused the e-mail system could be terminated without any appeal, spam existed but was very rare and quickly dealt with whenever it sprouted.
If Yahoo, MSN, and Earthlink all joined together to form an "invitation only" e-mail club, and each took responsibilty for patroling its own user base, the world would be a whole lot closer to a spam-free place. "Pink contracts" would not be tolerated, as the entire ISP would risk being expelled from the club, and therefore not be able to offer functional inter-network e-mail service. Remember, the Internet is nothing but a network formed by joining other networks... nobody has to honor the requests of other networks, however.
It's a value judgement... and according to my values, I think this is not a great idea.
First, I think the benefits of having free and semi-anonymous e-mail outweigh the disadvantages of having to use and maintain spam filters. Obviously, many people disagree with me here, and more all the time.
(Here's a conspiracy for ya: what if some Big Brother is trying to kill the free exchange of ideas in e-mail by burying the whole system with spam? I don't believe it's true, but it's worth wondering about before jumping to non-free solutions!)
Second, even if I thought that killing spam was worth the cost of crippling some of e-mail's better and more distinctive features, I think going about it in a non-standards-based way is likely to be a road to chaos.
The best solution, I think, would be to supplant e-mail with something new that works in a more trusted and accountable way. If someone really hates spam, they can use only the new system; if they want anonymity and freedom at the cost of spam, they can use the current mail system. The systems could coexist much like Usenet and the Web; each is useful for different things.
With reasonable men I will reason; with humane men I will plead; but to tyrants I will give no quarter. -- William Lloyd
As ShakaUVM stated in a previous post, the problem with spam is forged return addresses. As another poster mentioned, spam is really a social problem. The problem is, one or two dumbheads loose in the world can cripple a great technology (email).
So, spam is a social problem - a few people are a nuisance. But the problem is, right now - even if we pass great anti-spam laws, we really have no good way of knowing who is sending a message. So what if it came from ip address 3.14.15.92? Spammer joe can disconnect from that address right after he sends said spam, and nobody wants ISPs' logs to be able to be subpeonaed, do we?!?
So spam is a social problem, but we have no way of tracking the offenders. I think an authentication-by-encryption scheme is a Good Thing, but wait - I think there are such standards already out there.
The STARTTLS extension for SMTP, in RFC 2487, allows SMTP traffic to be transported over a TLS (SSL) connection - also allowing for the same type of CA-signed certificates that HTTPS is famous for. So now we can tell exactly what mail server mail is coming from - and we can refuse mail from uncertified hosts, or prosecute abusive hosts.
Anyway, correct me if I've misunderstood anything; what think ye all?
Spam is a classic case of the tragedy of the commons.
As long as sending millions of e-mails relatively cheaply is possible, spam will NEVER cease to be a serious problem.
You have to break the economic back which supports spam.
It has to cost something to send an e-mail.
True, it will not disappear, but the volume will drop dramatically perhaps even to the point where e-mail will become useful again.
Absolute statements are never true
I don't mind downloading the spam because I have broadband. Getting mail is no big deal, but sorting it is.
The solution I use requires that one owns a domain. Simply provide specific addresses to people/places/things depending on your expectation for spam. Filter on the client name based on the to: field and most of the crap drops into the crap folder where it belongs.
This combined with a bayesian filter keeps the spam to a very reasonable level. One added bonus:
You can know who sold you out and pass the word to others.
I use gandi.net for this. They provide e-mail redirection for free with a grab bag for unspecified addresses. 12 euros per year with nice online admin tools combined with very reasonable legal terms makes the service well worth it.
As for the e-mail problem, it is going to come down to trusted mail servers. I believe we all should be able to run mail out of our homes, because that is part of being peers on the Internet.
So, anyone can send mail, but if you expect anyone to actually read it, you need to be trusted by at least someone
Blogging because I can...
If you can send an e-mail anonymously, so can spammers. If spammers can't send e-mail anonymously, neither can you.
The price of spam doesn't come anywhere near the value of privacy and freedom of speech. I happen to like the idea that should a need arise I can easily send an untrackable e-mail. I'm sure plenty of people in more intrusive countries already enjoy this ability.
Click on the link in my sig for my method of dealing with spam which is highly effective that doesn't destroy the privacy of the sender or cost money.
Ben
Work Safe Porn
Maybe Yahoo's idea will work, though it seems to be quite porous and more of a surveillance tool than an antispam measure.. in fact it is quite plausible that this is Homeland Security's wet dream and is being sold by Yahoo on their request (though that is more paranoid than we have to be).
I have a concrete proposal at the end of this post so please read on.
Anyway someone mentioned the tipping point and I am reading this after cleaning a thousand spams out of my mail folder so I am ready to consider lots of things.
But one thing is definite about all this. If these guys were terrorists planning some horror and not just an army of rotten people bent on selling viagra and insurance, they would be shut down in a heartbeat. You can follow the money! (As many people have.)
Note these datapoints:
- Telemarketers don't like getting phone bombed, as Dave Barry launched retaliation against an association of them.
- Spammers are in it for the money
- Their clients pay because they want to sell something.
- Their clients are living in meatspace and are allergic to publicity.
- Spam is by definition, easy to get since so many are sent from each machine. (In fact I get too many to even reply with "unsubscribe" to them all).
- We all see spam, but can't stop it because the spammers are laughing at us by endlessly transforming their campaigns. The helpless feeling I suppose is similar to terrorism in that there is a feeling of a nebulous enemy profiting by your openness, there is nothing to grab hold of.
- People are willing to pay money to stop spam.
- Homeland security (probably) and the NSA and similar national organizations (definitely), and telcos and isps (of course) are sitting in front of the big routers around the world. This information can be coordinated.
- Some big organization wants a steganography analyzer built quickly (recent slashdot story)
From this and a bit of blue skying and paranoia, I get:
1. Spam, which is subtly personalized and includes photos and hyperlinks, could be used as a communications network by terrorists, so definitely falls under the national security bailiwick. Ditto for viruses and worms, though they are maybe too visible.
2. Though maybe it is better to unlock the messages than to stop spam, from a security standpoint.
3. Certainly it is possible to make transparent who exactly is sending spam, and how the money flows from their clients. Both by surveillance and of course just trying to buy some of their services.
4. If it isn't illegal, they can't be put out of business and so long as they have clients, it is a "business opportunity".
5. But by focussing the anger of thousands of people on each client and detected spammer, this lucrative business can be turned into a financially losing proposition.
6. Finally, if we make it impossible for their clients to sell their wares, there will be no point to spamming. This suggests that rather than trying to secure all of the honest email, we should focus on removing spam from the network. I don't think blackholes work, however it is quite possible that a finer granularity and more intelligence might work. (See below)
So I welcome technical fixes against spam but think they should more involve information sharing than an attempt to cryptographically secure the email network, since the power of email is fundamentally that it is so easy to use.
I would propose that a group of people are selected around the world to manually go through their incoming email and note which emails are spam, preferably qualifying what type it is and using some simple tools to also note whether this is the work of nefarious arch-spammer types that play tricks on you, as opposed to honest mailing lists. It should be an open architecture which allows more than one organization to do the grading. Perhaps one will only filter porn, etc. I believe some large antivirus companies do something a little bit like this on an automated level to learn about thre
True enough but the idea can be modified slightly like this.
.plan and the finger deamon can remove the line once the process is over.
The finger deamon can be rewritten slighly to return an affirmative if the user actually sent an email to the fingering domain. The SMTP server can drop a line in the
War is necrophilia.
The ultimate solution to fighting spam is realizing that there is no perfect solution. We all know that no matter what we, spammers find a way around it. So the issue is to stop looking for that so-called "ultimate solution" that's supposed to get rid of spam forever. If anything, it's going to take several different methods to eliminate spam and there's going to be some trial and error.
And spam filters are a bandage over a sore that's being seriously neglected. I think the problem that people don't realize is that with spam, the client is limited to what he/she can do.
Yahoo might be going against standards, but they are on the right track by trying to tackle the problem from server side.
I think using AI would have some real benefits on mail servers. AI has the ability to learn. Filters on the other hand require reconfiguration to combat the ever changing spamaflouge.
I don't remember who this quote is from, or whether I remember it 100% correctly, but it's great:
"To every challenging problem, there is a solution that is obvious, easy, and wrong."
Proprietary stuff like this one usually is that solution, because not enough eyes looked at it. That's why so many software projects fail, and that's why peer-review is so important in science.
Yahoo can't even teach their mailservers to play nicely with the rest of the world (they bounce when they should have rejected). I don't trust them an inch to patch sendmail or solve the spam problem.
Assorted stuff I do sometimes: Lemuria.org
Mail servers that have the "nerve" to bounce mail do so in a predictable manner. Normally with a phrase such as "could not be delievered" or "rejected."
Instead of freaking out, take the time to actually look at bounced messages and find tells so you can filter them out. Those 100% unqiue tells are there.
"I'll never see the bounce."
You will if you allow the tells your mailserver uses to pass through. Or give it a unique bounce message that gets past your filter.
Trackable e-mail requires that everyone or no one do it. I'm certainly not going to. I have better ways to deal with spam. If you do it, you'll still be getting bounces from mail forged with your domain sent to mail servers that don't check.
Like it or not, you need to deal with it. If you don't have enough control, fire up your own mail server that you do have control over.
Ben
Work Safe Porn
Here's one system that I think could work:
Each E-mail sent can optionally contain a micropayment, cryptographically tied to the receiver's E-mail address and the contents of the E-mail.
When I receive E-mails, I can choose to ignore or simply spam-filter any E-mails with a value of less than X (I decide what X is).
The default action is to return the micropayment to the sender, if nothing is done within a week (or a few days) of sending the E-mail. This way, sending payments to someone who is not part of the system will effectively be a no-op.
The receiver has several possibilities:
Ignore the payment (the sender eventually gets his deposit back)
Return the payment immediately
Collect the payment
The way I would use this would be to collect the payment on any unsolicited commercial E-mails that I read (thus making sending SPAM cost money) and return/ignore all the payments from friends & other valid sources.
You could still send E-mails with no monetary value, but they would be subject to strict filtering.
I would probably set a filter limit of 5-10 cents/E-mail and only collect the money (if any) on real spam.
The system would provide income to those who run the banking, because they would get the interest on the deposits made by E-mail users.
At first, implementing something like this would have little impact on our E-mailing, because only a few people would be using the system. If it ever became widely adapted, we would have an E-mail system where sending spam is too expensive to be worthwhile and where regular E-mail would still be free (except for the loss of interest on the deposit made to send micropaid E-mails).
I don't believe this is proprietary. Yahoo is releasing a patch for Sendmail. AFAI can tell, while they're funding the dev work (because the spam rate is killing them), they aren't trying to milk this for more money.
One major problem with standards groups is that people like Verisign are on most security standards groups. Verisign has extremely strong motivations to ensure that email uses a Web-like interface, where one purchases an (expiring) Verisign cert for each email server one runs. They have strong incentive to block competing solutions. If you want to come out with a good system that prevents existing folks from milking a market, both industry consortiums and standards groups are pretty much useless. You need to do what happened with PNG -- have a bunch of talented, aggravated engineers sit down, write up a technically good spec, and put out reference code. Later on, let standards committees follow what's in place.
I can't figure out why replay attacks are an issue. I, personally, would suggest, off the cuff, including any To: or CC: lines in the message body (just for signing purposes, not actually sending either header in the body). This way, a replay attack would only allow resending the same email to the same destination from the same source. It's also pretty easy to include a timestamp, if folks are *really* concerned about replays.
Yahoo is pretty much doing what ESR and RMS have been hoping for for years -- contributing to open source systems because there's an itch that needs scratching.
Paul Vixie (disclaimer -- I don't move in his circles, and what I know about him is entirely secondhand) seems to be involved a great deal in politics, rather than technology. He leaves a bit of the same bitter tang in the mouth that Verisign does. He is, apparently, the source of at least some of the IETF objections. Vixie has also made a number of antispam statements that I tend to disagree with, including advocating mass blocking of mail servers on home email connections by netblock.
May we never see th
The spam issue must be solved, whether by social, technological, legal or whatever else means, or a combination of these.
The sad truth is, there will always be jerks willing to engage in self-profitable activity at the expense of others, and to some extent this activity is what we call crime. There are three prerequisites for it, which are:
- intent (you know it's bad, but you don't care)
- gain (outweighing the cost / risk)
- occasion
This last one you completely overlooked. Why do you think locks exist ? Why do you think most countries ban civilians from owning firearms ? Because that will reduce the number of occasions someone has to commit crime.
Maybe we deserve this world ?
First let me say I agree with your premise. I have never received an anonymous delivery, email or otherwise, that I desired.
But let me show the fallicy of yahoo's actions.
Yahoos step 1 is to reject forged headers. Forged headers was just made illegal by the Bush administration IIRC. I completely approve.
Yahoos step 2 is to force a signature on every email by the server. Interestingly, Step 2 removes the need for step 1 and makes you wonder if step 2 is their real desire. Note that a solid step 1 also removes the need for step 2, given that open relays are shut down.
This is where I disapprove.
This proposes the same problem as DRM. Who controls which signatures are accepted? Once again we are right back with Verisign, et al. So unless your server has a PURCHASED KEY from verisign, or the like, your server won't be sending email to yahoo or any of the ISPs that adopt this.
I promise they won't be suggesting PGP either And so the spiral begins. Yahoo sells the rights to the certificates it will accept on a yearly basis. Verisign subsells this right in the form of the infamous certificate chain.
So what if the code is free, the certificates are not!
The fact is that anyone can raise a new standard, it will have to do something useful or it will simply be ignored, but it is hardly difficult to get the process started, by raising an Internet Draft, and in a case like this it should only take a few months to become a standard. The IETF work much more efficiently than any commercial standards body that I know of. The process is documented at ftp://ftp.isi.edu/in-notes/rfc2026.txt amongst other places, and surely must be the correct procedure to use. Who cases about ANSI, or BSI, or CENELEC, or any of these bodies that sell you a few pages of copyrighted standard for silly money? The RFCs are published for everyone to use, which is why ithe net works as well as it does, despite the efforts and intentions of some, such as the Convicted Monopolist (had to get him in somewhere..), to "de-commoditise the protocols".
There is no reason why they can't raise an Internet Draft right now and start using the thing, people can then follow the Draft at their own risk of having to do more work if it changes.