Slashdot Mirror
Yahoo! Develops Anti-Spam Architecture
prostoalex writes "Yahoo!, the owner of one of the largest e-mail systems in the world, is said to be developing a cryptographic product that will be offered freely to mail servers. 'Domain Keys,' according to the Reuters article, would require the message sender to authenticate in order for message to come across a trusted e-mail network. The idea has been around for ages, however, it required someone from the big league like Yahoo! to step in." While Yahoo! isn't the first name that comes to mind when I think of trusted email, it's still a step in the right direction.
But ultimately one has to worry about the lock that Yahoo! might have on servers once they get it installed all over the place.
Could you imagine this becoming really popular and then Yahoo! getting bought by someone like oh say Microsoft? (or any other big commercial interest)
"Trusted email" and "Yahoo!" should not be mentioned in the same sentence, except perhaps to say that these two things should not be mentioned in the same sentence.
I can't see how they can "lock" anything since it is clearly stated that the initiative will be open sourced...
Of course, Microsoft will probably figure out a way to break it so that it only works with their products but that's a different story...
SpamCon's Barrett cautioned "It's a good approach for those that are willing to use it," he said. "Any kind of cryptographic solution is going to involve some computing overhead, and that's not cheap."
...
Whereas the latter completely true, I think the weakness of the argument is a testament to the idea being an excellent one. CPU horsepower is very very cheap. If Yahoo think they can do it, then who exactly will have a problem ?
Just as long as I can incorporate it into my server, I'll be a happy bunny - all the other proposals put forward so far seem to limit the mail providers to the big boys
Simon.
Physicists get Hadrons!
But how am I going to get my special penis enlargement information now? And what about that family matter I am resolving with Mr. Mobotu?
In all seriousness, I think this is a good idea. But, sadly, it's going to be cracked. Domain keys can be forged, and that will be the first thing that these spam servers will be focussing on right now. They'll set up a Yahoo acct and monitor traffic to see what the domain keys look like. They will then duplicate the acks and be back in business. It's only a matter of time.
This is a good step, no doubt. It is just that we should be looking at ways of putting spammers out of business, too. Hit their wallets, not their tech. Tech can always be worked around, especially by dubious people.
Instead of domain keys, I had a different idea that might work a lot better.
What if nobody sent email over the Internet?
Today we have the ability to use web forms to pass messages back and forth to other users on the same service. With that option, the server admin would be able to flag spammers and ban them. If you wanted to message another user of another server, you could type in their location as USERNAME@DOMAIN, and that would queue to be sent in batch to the other server after authentication.
No outside contact. No spam. One message per customer. If you send more than a certain number of messages in a day, they are held as possible spam.
Privacy goes out the window, but hey... it's not like there is any privacy in non-encrypted email anyway.
Does anyone know what software Yahoo's mailservers run?
Feel that power? That's mah MOUSING FINGER
...you'll only be spammed by Yahoo??
If Google really cared they would fix Android Chrome to reflow text, instead of discriminating
As long as it's an open standard that eventually becomes RFC3821, I'll be okay with it. But if it's one of those proprietary "pay us to participate" schemes, they can go jump. Oh, and there should be no scope for someone to say "pay us or we won't accept email from you.
Personally, I'd like to see two things.
1. The software Yahoo! is developing should be open-source, so nobody can monopolize it. At the very minimum, the protocols involved should be well documented so open-sourcers can make their own implementations if they have to.
2. Give this software a few months to propogate to a good chunk of the ISPs out there. Then, Yahoo! should announce that they will NOT accept any email that is not signed with this software. I'll guarantee that everyone will be using this new protocol in a matter of weeks, since no ISP wants customers screaming because they can't get mail through to Yahoo! accounts.
Meldroc, Waster of Electrons
I'm assuming that what is sent out is an encypted token for which the public key can be used to decrpyt, so:
So, the token to be encoded will change from mail to mail, thus making replay techniques pretty much impossible, I think. At least, that's the way I'd do it, and I'm pretty sure I've seen it presented before as well...
On the other hand, I ain't a security expert, so there's probably a gaping hole in the above
Simon
Physicists get Hadrons!
It can be open sourced, but that doesn't mean anything about preventing lock-in.
Presumably a 'domain key' is some cryptographic element that authenticates that your domain is who it claims to be. To me this sounds an awful lot like SSL where a third party issues the keys, or acts as a clearinghouse for self-issued keys.
Either way, Yahoo could be the man in the middle acting as either issuer or clearinghouse. Think of it this way, OpenSSL is open sourced, but that doesn't keep the SSL issuers from having a lock on that market.
If someone announced a cure for all cancers, this crowd would immediately dismiss it because it could possible be bought by Microsoft. You pimply-faced pessimists remind my of Eor from Winnie the Pooh.
One guy's take on why it won't work
If the spammer...or anyone for that matter is spoofing a header anyway, it shouldn't be difficult to find out the encrypted private key, since it is sent out with every message originating from the domain.
I could, presumably send an email from my secure email address to a non-existent email address of the domain whose encrypted private key I wish to find out: eg bounce@email.com. The bounced message should have it in the header.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
The first time that I heard about a teergrube to use as a way to block -- or at least make it damned difficult for -- spammers I was intrigued at its simplicity. And tho' I find references to it all over the 'net, I don't think that it has been mainstreamed yet, and frankly I don't know why. Have spammers developed a counter to a teergrube? Or do mail admins simply not know enough about them?
Mit der Dummheit kämpfen Götter selbst vergebens.
when you think about it, BUT this should come from IETF or some other body not from a company. A few important points:
...
1) Who will issue the keys?
2) Is anonymous mail possible if the receiver allows it?
Furthermore spamming is a social problem emerging from our commercial world and technical solutions can never be 100%. What if:
a) I send spam from a "secure" domain?
b) forge certificates?
c) the certificates are too expensive? (like SSL, I think it should be included with a domain)
I like the "Bayes" spam filters best. You get 99.5% spam protection and keep anonymous mail.
We all see the need for authenticated senders (biz communication, etc.), but we should be careful
Me personally, if spam makes it through my filter, I ban off the offending address working my way up towards the class c - b - a. All attempts at a port 25 connection is drop point blank, http, https, etal are kept open. I also have dontspam#somefreemailaccount.com's to use for form shit. Once in a while when registering for say an upper-crust website account, I'll use something like msndoesntspam@mydomain.com to see who exactly is sharing my addresses, then null the account if I see anything odd coming in to that account, and never trust the site again. Procmail works the most wonders though.
MoFscker
First let them implement some user account verification, so that a RCPT TO: results in a 550 reply when that user does not exist.
This enables SMTP callbacks to stop spam being spoofed "from yahoo", just like everyone else does.
I've used my Y! acct as my main (personal) e-mail acct since sometime in late 1998/early 1999, so I'm very glad to hear about this. Hopefully it will help combat the 100-200 SPAM msgs I get per day. The Bulk Mail folder was a step in the right direction, as it does catch the majority of the crap, and allows me to delete it with a single click.
Thanks! Again! Yahoo!</elRegStyle>
-bZj
.sig
Okay, so they're developing a system that they'll release to open-source developers.... why not DEVELOP it in the open in the first place?
Ed R.Zahurak
You know, oblivion keeps looking better every day.
My experience over-all has been excellent, with on minor exception: This was from a google search on "yahoo marketing preferences", and pretty well sums up what happened. They basically reset user marketing preferences, twice in about 4 years if I remember correctly.
It was a bad decision IMHO, but easy enough to reset your preferences, here is yahoo's page on privacy, with links to reset your marketing preferences:
http://privacy.yahoo.com/
Other than this one issue, I've been very happy with Yahoo. Being able to check all my e-mail on one Web site for free is great. Never have lost any e-mails, no problems at all.
Bala Krishnamurthy at AT&T Labs has given a number of talks recently, including to the IETF, on a spam disincentive program he calls SHRED. My understanding is that it uses offline cryptographic computation to amortize this overhead and distribute it to parties willing and able to devote the computational resources.
In any case, the tag line for this article had it right, standardizing this will be hard and heavy-hitters like Yahoo will need to take the lead. But a key problem is getting the new system to interoperate with the old.
Mencken had it right. So glad that's old news.
From the article: Yahoo said its "Domain Keys" software, which it hopes to launch in 2004, will be made available freely to the developers of the Web's major open-source e-mail software and systems.
But later: Garlinghouse also argued that Yahoo's proposal should be attractive to other e-mail providers because it is free and comes with no special restrictions. Is the GPL considered a "special restriction"? Will it not actually be GPL, just available to open systems?
I'm guessing that you'll need to be a GPL mail server to both require the private key for receipt, and to be able to use the system to give the email the private key for sending. So, what will this do to non-open mail systems?
Is Yahoo trying to break MicroSoft's mail service? Will this work? What's MSFT's option--reverse this and include it in their system anyways? Switch to an open system for a mail server, like, say, something based on a BSD license? Or ignore it, in an attempt to deprive it of critical mass?
Indeed, this might all be moot; Yahoo might make it free and available to everyone, either on a free system or a non-free system; the article isn't clear as it says both. It could also be that MSFT already uses an OSS mailserver in IIS for all I know about MSFT product. But I suspect this is a power-grab, like everything else these days. And, I have to say, if it is I wish Yahoo the best of luck--this would be another demonstration of the power of OSS; it allows the community to change together on a dime and play well together. Whereas makers of proprietary systems each have to modify their own systems with their own coders.
--
$tar -xvf
I don't see how lock in will be an issue. Imagine the following scenario:
I can't see how this would neccesitate a clearinghouse.
a thing to remember is that if someone can prevent a spammer from communicating based on identity (or lack thereof), you can be silenced as well.
This is why I have put my efforts into sender-pay systems and specifically the camram project. We invite you to please come and join us in the effort to build a decentralized, user-friendly, freedom-of-speech supporting antispam system and hit spammers in the pocketbook.
camram antique documentation (too busy writing code to write new documentation)
If they're offering it for free, BEWARE. IT'S A TRICK. There's some hidden patent they're going to decide to enforce once the entire world adopts the architecture.
*waves hands ominously*
Seriously. This solution needs the cooperation of most. It is the exact solution I have been longing for, and to be successful when it is released it needs every significant domain to follow suit. Your ISP won't use Domain Keys ? Rant to them till they do ! They still won't ? Set up your own MX and sign in to the certified network. Have your friends and relatives get aboard too.
As soon as the certified network is considered a valid alternative to the current spam-ridden, scam-infested open email exchange system people will switch boards in a blink... provided it is easy enough to get a certificate.
Maybe we deserve this world ?
Would you rather choose a Yahoo product over an open standard that is under development? I'm speaking of AMTP, of course. (See AMTP author's site).
Yahoo's size doesn't give that much weight to their proposal. Yahoo's email is not used in business to business communication (do not count hot dog stands as businesses), so businesses can just aswell block everything that originates from *@yahoo.com if it is not directed to their consumer service department.
Also, reverse mx records provide much of the same benefits with minimal alterations needed to current email infrastructure. One DNS record added and small change in MTA software.
If Yahoo would really like to do a service to the internet community, they should rather consider looking AMTP and reverse mx records.
I joined two users too late.
Under Yahoo's new architecture, a system sending an e-mail message would embed a secure, private key in a message header. The receiving system would check the Internet's Domain Name System for the public key registered to the sending domain.
If the public key is able to decrypt the private key embedded in the message, then the e-mail is considered authentic and can be delivered. If not, then the message is assumed not to be an authentic one from the sender and is blocked.
For every message, I have to check and unpack the header, go out to some PK server, and validate the keys, before I decide to accept/reject? That introduces a big latency into SMTP.
Also, this doesn't do anything to stop 'legitimate email marketers'. There's a death penalty (blacklist) for a site or particular sender's key, but nothing to stop a spammer from changing keys and starting over.
Or will everyone have to get their own key pair? Who's going to validate them, and at what cost per key pair?
This won't do a thing to stop spam, and imposes too big a burden on the infrastructure and on the 99% of us who don't spam.
sigs, as if you care.
Read: trusted network == commercial network
Why do you think this is in the "Money & Investing" department (see the linked article). No, this isn't for me. Businesses may well choose to use something like this for their communications, but they will not have the pleasure of communicating with me. While SMTP has its flaws, it still allows any IP host to send mail to any other IP host and that is a good thing.
To gain insight into what's going to happen with email and Internet communications in general over the next couple of years, you have to adopt a business mindset to see it from their eyes. There is a big problem (spam) hence a potential to make money. Various companies are going to try and cash in on this situation by offering a solution that might very well decrease spam -- some sort of commercially controlled communication network -- but this is definitely not in the best interest of the Internet. Of course, it's in the best interest of the company that's peddling the solution (duh!)
The Internet isn't Compuserve, or AOL. It's a network of IP hosts, and those are the entities which should have a facility for sending communications back and forth. There is no need for a central carrier for communications
The proposal is very simple and most of the posts are just plain wrong about what it means.
All mail servers will have a public/private key of some type. The public key will be stored in the DNS system as extra data.
When an SMTP server connects to another SMTP server, the sending server will encrypt something (likely a checksum) with the private key for the domain the mail is from (likley the envelope from, not the From: header) and place it in a header.
The receiving server will then grab the public key for the domain in the envelope and verify the message is being sent by a server that is authoritative for that domain name.
Very simple. Now spammers can't send spam and make it look like it came from my domain. I'm currently getting flooded with bounces from a spammer doing this, so I really want this proposal adopted.
The implementation can be phased in, too. The mail server could add a header saying if the domain was verified and spamassassin could then adjust the spam rating of the message appropriately. Eventually servers would be configured to refuse mail from unverified domains.
So if you own a domain name, you just have to generate a key pair, add the public key to DNS, and add the private key to any SMTP server you send through.
Once this is required, you theoretically will always be able to contact a person responsible for the sending of the spam (whoever is listed in the whois database for the domain). Contacting them (or abuse@) would solve the problem with any major email provider, and you can just ban email from any small provider that doesn't give an adequate response.
Aside from the possible computational requirements for all the crypto work, I don't see any downsides to this. If by some chance a spammer broke/acquired your private key, you'd just generate a new one and update your DNS entry.
The way the IETF and other standards bodies have worked is that some organization wouldtry out a new concept for a technology and once they feel the concept is working, they will create a Request For Comments (RFC) which allows others to implement and offer feedback. Over time the RFC gains support and ultimately becomes a recommendation.
This process was used to create the internet today, including all of the network protocols and services that run on top of it. Even SMTP was an RFC first.
Brennan Stehling - http://brennan.offwhite.net/blog/
The basic idea, as I understand it, is that the DNS for a domain holds a public key, and mail sent with a "from" address in that domain must be signed with that public key. That's an old idea, and not all that bad. You create your own public/private key pair; you don't have to buy a "certificate" from somebody. (I think.) If you control a domain's DNS info, you can send mail from anywhere with that domain listed as the sender, as long as you know the private key.
For the free-mail services, it's fine. All their mail is authored via web applications and sent from their own servers. Only the service has the private key. Only the outgoing SMTP servers need to know the private key. That's the Yahoo Mail case.
If you own a domain, you should have full control over your own public and private keys. But adding additional info to a DNS record is not well supported by most hosting services. If you're not running DNS yourself, you may have problems setting your public key. Hosting services have to support this.
Signing can occur either in the original user agent (the SMTP sender) or in a mail forwarder. It's easier to implement this in mail forwarders, but if you want to send using a return address other than the one of the mail forwarder you're using, your user agent has to know how to sign mail.
If you're downstream from an ISP and don't control a domain, the ISP owns the key for the domain and can control what they sign. That has implications. They might force you to use web mail, for example. Or run their client software on your machine.
Spammers can still register domains, run their own DNS, sign their mail, and spam. It doesn't really stop spam.
Your public key is now valuable, and a target for spyware and viruses. Expect to see viruses that steal public keys from (inevitably) Outlook and send them to spammers. Or just send spam from the attacked machine.
What this really does is provide a clear way to identify joe-jobs using addresses from major mail services like Yahoo Mail. That helps Yahoo more than anybody else.
I don't know if anyone's interested, but over the weekend I put together a white-list, white-phrase, auto-response human-sender verification system which has been 100% effective so far. Meaning that everything it identified as spam was (which in now bounces instead of holding for my perusal) and every email from both human strangers and machine generated email from companies I wanted to hear from got right through. I wanted to write a program that would do this automatically for my web host, but even though it wasn't an option (they used off the shelf Ipswitch software that they couldn't reprogram), I was still able to set it up using existing filters. Which means you probably can too.
If you want to know how it works, either to use it or to find a flaw, say so and I'll post the specs.
Here's an article that gives an overview of doing this with smart cards.
End of Line.