Slashdot Mirror
Yahoo and Unilateral Anti-Spam Technology?
EatenByAGrue asks: "According to this Business Week article, Yahoo is planning on distributing a toolkit for Sendmail and other mail daemons that adds an encrypted source domain key to email headers to verify where they came from. However, critics are concerned that the scheme will be easily bypassed and that it ignores standards bodies. What does the Slashdot community (representing countless email admins, I'm sure) think of this proposal? On one hand, its a commercial enterprise dictating standard technology, on the other hand, the standards bodies have proven themselves helpless and hopeless when it comes to providing solutions."
easy email tracking system will be gladly welcomed by police and other agencies...
This Is Not a Sig
Spam is a SOCIAL problem, not a TECHNOLOGICAL problem. Spam must be solved by economics and/or behavior.
Web folk always moan about MSIE's poor standards complience, for instance, but forget that CSS/Text came from them -- Netscape was pushing CSS/JavaScript at the time. Now, one of those is a standard, and the other is dead.
Ultimitely, either people will like Yahoo's idea and adopt it and it will eventually become a new standard, or it will be ignored by everyone else and forgotten. Only time will tell.
The extra key could be used by anybody who wants to, and ignored by the rest. And their implementation is open-source, so it doesn't look like a way of making an end-run past other ISPs. And since many spam messages come from fake Yahoo email id's, this would be a great way to immediately filter out those ones: if it says Yahoo but doesn't carry a key-->SPAM bin
I like the idea of a major player getting on with it and DOING something.
Would we rather have MS dictating an anti-spam standard? You can be sure such a beast would be a lot less benign than Yahoo's proposal
"From" address from what your SMTP server is, in which case I don't see how it could work for you.
This may put a lot of travellers out in the cold.
A solution is badly needed, but it has to work for everybody.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
...de facto standards emerge. One need look no further than POSIX/SUS and GNU/Linux for an example.
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety."
Doesn't sound like this will be too effective in stopping spam for
Yahoo users, and Yahoo is already a pain
to work with.
I setup a proxy and was a spam relay (unknowingly of course) for just
under a week. I got blacklisted on a couple of email sites, my ISP
bitched and I fixed it. So sorry.
So I'm now off every blacklist I know of, and everyone loves me again.
That is except Yahoo, the evil nazi bastards. I've filled out their
stupid, "fill this out to get
un-blacklisted" form at least 30 times (twice a day normally).
It must go into a black hole because they still are rejecting my mail.
Everyone else lets me through but stupid Yahoo, who seem to have NO
admins, no technical people, and a violate once banned for life reject
policy. Grrr. So I guess, if this new system lets them drop their damn
overbearing blacklists, I'm all for it.
Now that RIAA has gotten rid of Napster and trying to crack down, what did most people do? Other programers created other way to share music. Now all of this was just so we could get free music. These spammers are making money at what they do. How hard are they going to try and find a way to mail in our inbox? What we need to do is find a way to keep spammers from making money. That would stop them.
It would be much simpler to add a record type to DNS servers to identify **outgoing** mail servers. Email proxies, where 60% of all spam comes from, would be immediately eliminated. Spammers with fixed servers and addresses are easily taken care of by the RBLs. Why introduce something that is more complicated and less reliable?
"...on the other hand, the standards bodies have proven themselves helpless and hopeless when it comes to providing solutions."
E-mail is supposed to do a certain job, and it does that job well, at least from a technical standpoint. The problems with spam are identical to similar problems in every other arena, it's just that they seem worse because of the level of automation. Even if it wasn't automated, spam would still be a problem. With idiots knocking on my door every other week with a hard sale for everything from oil changes to chinese food, I'm starting to almost regret the do-not-call list, because I didn't have to worry as much about these degenerates (if you don't take "No" for an answer and walk away immmediately, you are a degenerate in my book, and very door-to-door jerkwad so far has been one) giving my wife a hard time.
Standards bodies can't do anything to fix human behavior, unfortunately.
I'm all for a spam solution coming from private enterprise as opposed to legislation- in fact, I think the former is the only method that has a chance of working. Maybe Yahoo's attempt will help, maybe they'll waste a bunch of money trying, but I guaruntee it's less money and less waste than Congress or the FCC doing the same thing.
I don't think so. I think a bad and poorly designed solution is worse than no solution. Especially when there is other competing solutions, which are argueably better, or at least equal to Yahoo!'s domain keys system, such as RMX. IMHO, Domain Keys offers no significant improvements to the spam problem, but rather adds a crypto overhead to the sending and receiving of every message. I think it is great that Yahoo is trying to innovate to stop the SPAM problem, but being cavalier and going at it by themselves is not the answer, especially when they have a great Anti-spam alliance with AOL and MS.
FYI, the odds on the street of IETF approving a new DNS RR type (as discussed today on the SPF mailing list) is that it would take at least 2 years and more likely 5 years.
It's not like the spam problem cropped up overnight either, it's been around for at least a few years and the IETF, et al, are still discussing the issue.
Pain is a powerful motivator...
Reverse MX and Yahoo!'s proposal, however, don't require widespread adoption at the start. In fact, the tipping point is probably only a few percentage points of the domain namespace.
After all, for just a few minutes worth of work (more if you don't already provide SMTP AUTH, or require users to VPN in to send e-mail already), you protect your domain against joe jobs and forged e-mail bounces. So there's a low cost-of-entry. (Yahoo!'s proposal requires more work then the simpler, less CPU-intensive SPF proposal.)
What happens next is that domain admins that publish keys/SPF information find that they're no longer getting joe-jobbed and they're able to block a higher percentage of spam then they used to. Word gets out and more folks sign on (second wave adopters).
Sometime after that, the big ISPs require your mail servers to publish SPF/keys if you want your e-mail to be delivered to their users. (FYI, this is very similar to AOL's whitelisting program, which is essential a privately-administered reverse-MX system where you tell AOL what IPs your e-mail is allowed to originate from.)
As a WAG about rate of pickup, early adopters have started, second wave folks will probably sign on in the spring/summer, and I wouldn't be surprised to see ISP-blocking by the end of the year.
Wolde you bothe eate your cake, and have your cake?
Instead of sending the whole email content - and with it the ability to falsify email header information, why not just send the email header only - and require the originating server to hold the email content?
Neat idea... in theory. There are a few problems with it:
1. It would reduce overall bandwidth being burned on the Internet and cost the very influential backbone ISPs lots of money that they're charging smaller providers for bandwidth, so they'll hate the idea and lobby against it.
2. The flow of information on the Internet would heavily tilt more towards prime time, creating additional bottleneck issues. Users would be downloading expentially more data during business hours and much less in the off time. Server resources would need to be beefed up and there is no guarantee that the requested mail could be retrieved upon request (an e-mail based "slashdot effect")
3. If you think e-mail headers are misleading now, under such a system things would be a lot worse. You'd be lost in a sea of misleading e-mail you could only verify by exposing yourself to the spammer.
4. When you went to retrieve the e-mail message, you would expose your personal IP address. It would be the equivalent of having a web-page bot allowing spammers and other systems to associate a fixed location in cyberspace with your identity, email and any other info in the e-mail. Serious privacy invasion issues abound.
Especially considering how promising the OSS model is, why can't we create a solution? We talk about the complexity of the problem, the importance of not breaking standards, etc. Who FUCKING cares if I can't check my email because it totally FUCKING BURIED in unsolicited junk...
I don't mean to come off as the thundering asshole, but this situation has grown so slowly its like watching a car crash spread out over the past 15 YEARS.
Please, experiment. Break things. I don't give a shit, but don't let us sit here moaning like helpless children while spammers sit back (laugh) and rake in MILLIONS.
Get fucking aggressive.
And if I hear one more idiot talk about how you have to cut spammers off by not buying their products I'm going to cut him off at the knees! If that would work you and Noah could be shooting dice right now and we'd have a hell of a lot less to worry about.
Programers still know how to experiment, right?
Quack, quack.
This would a spammers wet dream.
They would write their own mail servers where more than one recpient would be linked to one post on the server. This means that they can send a small header it to a gazillion people and only spend 400 bytes on actually storing the message on their server since they only need one copy of a particular Email.
Bandwith is only wasted when a user comes to look at the mail, which also verifies that that user exists (double spam for you my friend).
So, this would make spam worse.
so in short
1) spammers could send at least twice as much spam as they can now.
2) they will get much better verification that the mail address they had is correct.
The Internet is full. Go Away!!!
Most of your reasons are in fact why signed email WON'T work. .001% of transactions where conventional forms of contract aren't good enough. Most people wouldn't sign a binding contract without legal advice, at which point they have access to a notary, etc., and the signature feature on email has no value.
B. CRLs don't scale. Period. There's a reason why PKIs hardly ever get past 100K users.
C. Someone to sue...only in the US is that an attractive feature.
D. Sure, but most users are unlikely to get savvy enough to understand the distinction. The proposed scheme takes that decision out of the user's hand.
E. Sure, for that
My take is that this is a problem that is hard enough to address even partially---adding the burden of a massive worldwide PKI deployment would make it impossible. Verisign or Thawte would love it.
Premature optimization is the root of all evil
The trouble with spam is the forged return addresses. If spammers were forced to use real email addresses:
;) (Kidding... his is way eaiser to update than mine) I think that implementing something like this on every ISP in America would immediately kill spam as we know it.
1) It would be much easier to block spam
2) It would be much easier to get their accounts revoked.
A friend of mine runs a script which ensures every email he reads is a real address. Essentially, he's got a cure for spam.
He has a script running on his mail that replies to every email he gets with a confirmation code. When the end user replies with that confirmation code (all it takes is hitting ctrl-r and ctrl-enter) that email address is adding to his "verified email address" list, and the original email goes through.
He doesn't even look at emails that aren't confirmed yet.
If we could get this implemented on a systematic level (such as via confirmaiton reciepts automatically & transparently handed by the Mozilla mail client) it would essentially end free for all spam as we know it. And it doesn't require rewriting the RFCs or adding new headers, or whatever. It would work with any mail reader... though adding in transparency would require updating people's mail clients.
The downsides:
-Two extra emails for every one original email are sent... but only the first time. After the email address is verified, it doesn't need another confirmation. If this is implemented system wide, the savings in the reduction of spam messages would greatly outweigh the extra cost on the network.
-People who do not confirm don't let their email get through. This happened to me the first time I mailed him after he installed his system. I send him an email, and went home for the day. Didn't see he didn't recieve it until I checked my mail again. Mail clients that handle confirmation transparently would (nearly) solve this problem.
As someone who has experience writing spam filters (I wrote a pretty good neural net spam filter way before that Graham fellow wrote his bayesian filter, that publicity hog!
Shame they move so slowly... and never can agree on how to implement anything...
-Bill Kerney
Here is how we can solve the spam problem once and for all.
Turn on finger. Yes you heard me. Let's re-implement finger. Here is how it works.
My SMTP server gets email from joeblow@123.com. I finger joeblow@123.com. If 123.com says joeblow is a real user I then accept the email, other wise I can it.
Voila! No more forged headers, no more spam.
This very simple simple solution would also allow legitemate businesses to send spam to the people who have opted in.
War is necrophilia.
If you can send an e-mail anonymously, so can spammers. If spammers can't send e-mail anonymously, neither can you.
The price of spam doesn't come anywhere near the value of privacy and freedom of speech. I happen to like the idea that should a need arise I can easily send an untrackable e-mail. I'm sure plenty of people in more intrusive countries already enjoy this ability.
Click on the link in my sig for my method of dealing with spam which is highly effective that doesn't destroy the privacy of the sender or cost money.
Ben
Work Safe Porn
Maybe Yahoo's idea will work, though it seems to be quite porous and more of a surveillance tool than an antispam measure.. in fact it is quite plausible that this is Homeland Security's wet dream and is being sold by Yahoo on their request (though that is more paranoid than we have to be).
I have a concrete proposal at the end of this post so please read on.
Anyway someone mentioned the tipping point and I am reading this after cleaning a thousand spams out of my mail folder so I am ready to consider lots of things.
But one thing is definite about all this. If these guys were terrorists planning some horror and not just an army of rotten people bent on selling viagra and insurance, they would be shut down in a heartbeat. You can follow the money! (As many people have.)
Note these datapoints:
- Telemarketers don't like getting phone bombed, as Dave Barry launched retaliation against an association of them.
- Spammers are in it for the money
- Their clients pay because they want to sell something.
- Their clients are living in meatspace and are allergic to publicity.
- Spam is by definition, easy to get since so many are sent from each machine. (In fact I get too many to even reply with "unsubscribe" to them all).
- We all see spam, but can't stop it because the spammers are laughing at us by endlessly transforming their campaigns. The helpless feeling I suppose is similar to terrorism in that there is a feeling of a nebulous enemy profiting by your openness, there is nothing to grab hold of.
- People are willing to pay money to stop spam.
- Homeland security (probably) and the NSA and similar national organizations (definitely), and telcos and isps (of course) are sitting in front of the big routers around the world. This information can be coordinated.
- Some big organization wants a steganography analyzer built quickly (recent slashdot story)
From this and a bit of blue skying and paranoia, I get:
1. Spam, which is subtly personalized and includes photos and hyperlinks, could be used as a communications network by terrorists, so definitely falls under the national security bailiwick. Ditto for viruses and worms, though they are maybe too visible.
2. Though maybe it is better to unlock the messages than to stop spam, from a security standpoint.
3. Certainly it is possible to make transparent who exactly is sending spam, and how the money flows from their clients. Both by surveillance and of course just trying to buy some of their services.
4. If it isn't illegal, they can't be put out of business and so long as they have clients, it is a "business opportunity".
5. But by focussing the anger of thousands of people on each client and detected spammer, this lucrative business can be turned into a financially losing proposition.
6. Finally, if we make it impossible for their clients to sell their wares, there will be no point to spamming. This suggests that rather than trying to secure all of the honest email, we should focus on removing spam from the network. I don't think blackholes work, however it is quite possible that a finer granularity and more intelligence might work. (See below)
So I welcome technical fixes against spam but think they should more involve information sharing than an attempt to cryptographically secure the email network, since the power of email is fundamentally that it is so easy to use.
I would propose that a group of people are selected around the world to manually go through their incoming email and note which emails are spam, preferably qualifying what type it is and using some simple tools to also note whether this is the work of nefarious arch-spammer types that play tricks on you, as opposed to honest mailing lists. It should be an open architecture which allows more than one organization to do the grading. Perhaps one will only filter porn, etc. I believe some large antivirus companies do something a little bit like this on an automated level to learn about thre
It already costs to money send spam.
The problem is that cost is not sent directly to the originator. Perhaps it's time to create legislation that confers civil and criminal penalties upon someone who uses an uninformed person's equipment as a relay for unsolicited commercial communications...say...$.03 per offense?
How long do you think it would take for not only OS/App vendors to lock their stuff down tighter than mother Theresa, but that someone (many someones) would come out with 'free' software for the average Joe to install on his computer to track and log spammers trying to send/use him as an open relay (letting said spammer do it for , oh, say...a few million emails first?)
1) dictionary attacks
2) e-mail addresses in public records
3) common e-mail addresses that you have to monitor (john@domain, webmaster@, abuse@, postmaster@, root@)
4) friends who have posted your address online (good intentions...)
5) corporate espionage where someone makes a copy of a maillist for a spammer for $$$
6) spammer got lucky
Wolde you bothe eate your cake, and have your cake?
I think 100k mail domains worldwide is a little on the low side. One ISP to my knowledge hosts over 30k mail domains. Of course, that could just be our cool software :-)
Your friend sends you a 'funny' e-greetings flash card email via e-greetings card website. "Click here to send this to a friend!"
e-greetings card website sells your email address to spammers.
Lots of variations of this one are around. Check out evite.com and their 'privacy' statement. It only exists to capture your email and browsing habits and web-bug you with invisible pixels with cookies.
--jeff++
ipv6 is my vpn
Here's one system that I think could work:
Each E-mail sent can optionally contain a micropayment, cryptographically tied to the receiver's E-mail address and the contents of the E-mail.
When I receive E-mails, I can choose to ignore or simply spam-filter any E-mails with a value of less than X (I decide what X is).
The default action is to return the micropayment to the sender, if nothing is done within a week (or a few days) of sending the E-mail. This way, sending payments to someone who is not part of the system will effectively be a no-op.
The receiver has several possibilities:
Ignore the payment (the sender eventually gets his deposit back)
Return the payment immediately
Collect the payment
The way I would use this would be to collect the payment on any unsolicited commercial E-mails that I read (thus making sending SPAM cost money) and return/ignore all the payments from friends & other valid sources.
You could still send E-mails with no monetary value, but they would be subject to strict filtering.
I would probably set a filter limit of 5-10 cents/E-mail and only collect the money (if any) on real spam.
The system would provide income to those who run the banking, because they would get the interest on the deposits made by E-mail users.
At first, implementing something like this would have little impact on our E-mailing, because only a few people would be using the system. If it ever became widely adapted, we would have an E-mail system where sending spam is too expensive to be worthwhile and where regular E-mail would still be free (except for the loss of interest on the deposit made to send micropaid E-mails).
I think signed e-mail is a really good solution for controlling spam in general. However, there is one problem I haven't worked out yet.
Spammers are unscrupulous, but they're not the only ones. Let's say that you have someone who creates an online identity and uses tjeore-mail account for a few months and builds up a reputation as non-spam-sender. Great, they're not sending spam, and the system is working, allowing their messages to get through.
But then let's say that person manages their finances badly. They go out and buy all kinds of crap they don't need (a big screen TV, a car, etc.), and they charge up lots of crap on their credit card. They have no savings, and then they get laid off from work. Suddenly, they're desperate for money. Their electricity is about to get shut off, and they are a couple months behind on their rent. And then a spammer approaches them and says "I'll give you $5000 if you just give me your private key. All you have to do is get a new e-mail account, let your friends know your new address, and then hand over the keys to your old account to me."
Presto, the spammer pays his $5000 and gets a fresh, new, legitimate, trusted identity to send out his crap. He uses it for two days to send out spams for a customer's marketing campaign, and then he throws it away.
(In fact, this is a problem in general with identities that are secured solely with secret digital information. They give the holder of the secret the power to sell out and let someone else pretend to be them. And that's the one reason I can think of that replacing social security numbers and credit cards with GPG keys might not be so hot.)
Anyway, the point of all this is that if you rely on digital signatures to protect us from spammers, you have to be able to revoke identities very quickly, preferably in a matter of minutes, but hours would be acceptable in some cases. (It is preferable to allow the mail server to reject the spam before it ever gets placed in a mailbox. But if that can't be done, then the notification that a message is spam can come while it sits unread in the mailbox. The mail client can periodically re-check the status of all unread e-mails. Or, the mail server can push the latest spam information updates to the mail clients. Either way, the point is that it only has to be caught as spam before the user reads it, not before it winds up in their inbox.)
I don't believe this is proprietary. Yahoo is releasing a patch for Sendmail. AFAI can tell, while they're funding the dev work (because the spam rate is killing them), they aren't trying to milk this for more money.
One major problem with standards groups is that people like Verisign are on most security standards groups. Verisign has extremely strong motivations to ensure that email uses a Web-like interface, where one purchases an (expiring) Verisign cert for each email server one runs. They have strong incentive to block competing solutions. If you want to come out with a good system that prevents existing folks from milking a market, both industry consortiums and standards groups are pretty much useless. You need to do what happened with PNG -- have a bunch of talented, aggravated engineers sit down, write up a technically good spec, and put out reference code. Later on, let standards committees follow what's in place.
I can't figure out why replay attacks are an issue. I, personally, would suggest, off the cuff, including any To: or CC: lines in the message body (just for signing purposes, not actually sending either header in the body). This way, a replay attack would only allow resending the same email to the same destination from the same source. It's also pretty easy to include a timestamp, if folks are *really* concerned about replays.
Yahoo is pretty much doing what ESR and RMS have been hoping for for years -- contributing to open source systems because there's an itch that needs scratching.
Paul Vixie (disclaimer -- I don't move in his circles, and what I know about him is entirely secondhand) seems to be involved a great deal in politics, rather than technology. He leaves a bit of the same bitter tang in the mouth that Verisign does. He is, apparently, the source of at least some of the IETF objections. Vixie has also made a number of antispam statements that I tend to disagree with, including advocating mass blocking of mail servers on home email connections by netblock.
May we never see th
First let me say I agree with your premise. I have never received an anonymous delivery, email or otherwise, that I desired.
But let me show the fallicy of yahoo's actions.
Yahoos step 1 is to reject forged headers. Forged headers was just made illegal by the Bush administration IIRC. I completely approve.
Yahoos step 2 is to force a signature on every email by the server. Interestingly, Step 2 removes the need for step 1 and makes you wonder if step 2 is their real desire. Note that a solid step 1 also removes the need for step 2, given that open relays are shut down.
This is where I disapprove.
This proposes the same problem as DRM. Who controls which signatures are accepted? Once again we are right back with Verisign, et al. So unless your server has a PURCHASED KEY from verisign, or the like, your server won't be sending email to yahoo or any of the ISPs that adopt this.
I promise they won't be suggesting PGP either And so the spiral begins. Yahoo sells the rights to the certificates it will accept on a yearly basis. Verisign subsells this right in the form of the infamous certificate chain.
So what if the code is free, the certificates are not!
The fact is that anyone can raise a new standard, it will have to do something useful or it will simply be ignored, but it is hardly difficult to get the process started, by raising an Internet Draft, and in a case like this it should only take a few months to become a standard. The IETF work much more efficiently than any commercial standards body that I know of. The process is documented at ftp://ftp.isi.edu/in-notes/rfc2026.txt amongst other places, and surely must be the correct procedure to use. Who cases about ANSI, or BSI, or CENELEC, or any of these bodies that sell you a few pages of copyrighted standard for silly money? The RFCs are published for everyone to use, which is why ithe net works as well as it does, despite the efforts and intentions of some, such as the Convicted Monopolist (had to get him in somewhere..), to "de-commoditise the protocols".
There is no reason why they can't raise an Internet Draft right now and start using the thing, people can then follow the Draft at their own risk of having to do more work if it changes.