Yahoo! Develops Anti-Spam Architecture

prostoalex writes "Yahoo!, the owner of one of the largest e-mail systems in the world, is said to be developing a cryptographic product that will be offered freely to mail servers. 'Domain Keys,' according to the Reuters article, would require the message sender to authenticate in order for message to come across a trusted e-mail network. The idea has been around for ages, however, it required someone from the big league like Yahoo! to step in." While Yahoo! isn't the first name that comes to mind when I think of trusted email, it's still a step in the right direction.

Comments

Oh yeah it seems like a good idea right now....

[i_want_you_to_throw_]

But ultimately one has to worry about the lock that Yahoo! might have on servers once they get it installed all over the place.

Could you imagine this becoming really popular and then Yahoo! getting bought by someone like oh say Microsoft? (or any other big commercial interest)

Oh come on!

[Space cowboy]

SpamCon's Barrett cautioned "It's a good approach for those that are willing to use it," he said. "Any kind of cryptographic solution is going to involve some computing overhead, and that's not cheap."

Whereas the latter completely true, I think the weakness of the argument is a testament to the idea being an excellent one. CPU horsepower is very very cheap. If Yahoo think they can do it, then who exactly will have a problem ?

Just as long as I can incorporate it into my server, I'll be a happy bunny - all the other proposals put forward so far seem to limit the mail providers to the big boys ...
Simon.

Temporary

[dolo666]

But how am I going to get my special penis enlargement information now? And what about that family matter I am resolving with Mr. Mobotu?

In all seriousness, I think this is a good idea. But, sadly, it's going to be cracked. Domain keys can be forged, and that will be the first thing that these spam servers will be focussing on right now. They'll set up a Yahoo acct and monitor traffic to see what the domain keys look like. They will then duplicate the acks and be back in business. It's only a matter of time.

This is a good step, no doubt. It is just that we should be looking at ways of putting spammers out of business, too. Hit their wallets, not their tech. Tech can always be worked around, especially by dubious people.

Instead of domain keys, I had a different idea that might work a lot better.

What if nobody sent email over the Internet?

Today we have the ability to use web forms to pass messages back and forth to other users on the same service. With that option, the server admin would be able to flag spammers and ban them. If you wanted to message another user of another server, you could type in their location as USERNAME@DOMAIN, and that would queue to be sent in batch to the other server after authentication.

No outside contact. No spam. One message per customer. If you send more than a certain number of messages in a day, they are held as possible spam.

Privacy goes out the window, but hey... it's not like there is any privacy in non-encrypted email anyway.

OS?

[awx]

Does anyone know what software Yahoo's mailservers run?

Hmmm, why

[panxerox]

would they want to cut off 90% of their own customers? Mabee they will sell a new "technology" to circumvent the block system to the spammers. not like nobodys ever done that before.

Not necessarily

[meldroc]

If they use decent encryption, cracking this scheme will be nearly impossible. If they use a digital signature algorithm such as DSA or MD5, or public key algorithms such as RSA, the computational power required to crack these keys will be far beyond the means of the richest spammers.

Personally, I'd like to see two things.

1. The software Yahoo! is developing should be open-source, so nobody can monopolize it. At the very minimum, the protocols involved should be well documented so open-sourcers can make their own implementations if they have to.

2. Give this software a few months to propogate to a good chunk of the ISPs out there. Then, Yahoo! should announce that they will NOT accept any email that is not signed with this software. I'll guarantee that everyone will be using this new protocol in a matter of weeks, since no ISP wants customers screaming because they can't get mail through to Yahoo! accounts.

Broken already?

[CaptainSuperBoy]

One guy's take on why it won't work

Re:Broken already?

[uhoreg]

I thought Yahoo's new scheme was designed to authenticate the mail server that originated a transaction with a Yahoo mail server, not to authenticate the domain in the "From:" line.

That is correct. Yahoo's scheme is to provide authentication for the Received: headers, not the From: header. Currently, the Received: headers frequently get forged, so it is hard to tell where spam is coming from. A real person can usually tell fairly easily, but you can't reliably tell a computer how to do it. It would be much nicer to be able to feed your spam through a program that will send off complaints to the appropriate sysadmin, or that will blacklist the appropriate server, than having to analyze the headers by hand.

Not sure if I understand it right

[GillBates0]

How do they propose to keep the encrypted private key secure? I did RTFA but couldn't find any explanation of how the encrypted version of the private key could not be spoofed since it is part of the message header.

If the spammer...or anyone for that matter is spoofing a header anyway, it shouldn't be difficult to find out the encrypted private key, since it is sent out with every message originating from the domain.

I could, presumably send an email from my secure email address to a non-existent email address of the domain whose encrypted private key I wish to find out: eg bounce@email.com. The bounced message should have it in the header.

Re:Not sure if I understand it right

[RevMike]

How do they propose to keep the encrypted private key secure? I did RTFA but couldn't find any explanation of how the encrypted version of the private key could not be spoofed since it is part of the message header.

If the spammer...or anyone for that matter is spoofing a header anyway, it shouldn't be difficult to find out the encrypted private key, since it is sent out with every message originating from the domain.

I could, presumably send an email from my secure email address to a non-existent email address of the domain whose encrypted private key I wish to find out: eg bounce@email.com. The bounced message should have it in the header.

The authentication token would likely be some sort of hash of the message contents. In that way, a token is only valid for that particular message. The sender would generate a checksum of the message, encrypt it with a private key, then transmit the encrypted checksum as the token. The receiver would generate the same hash of the message contents, and decrypt the token with the public key. If the decrypted checksum equals the generated checksum, then one can be confident that the message came from the server it said it came from.

So what about a teergrube?

[rah1420]

The first time that I heard about a teergrube to use as a way to block -- or at least make it damned difficult for -- spammers I was intrigued at its simplicity. And tho' I find references to it all over the 'net, I don't think that it has been mainstreamed yet, and frankly I don't know why. Have spammers developed a counter to a teergrube? Or do mail admins simply not know enough about them?

Re:So what about a teergrube?

[Saint Aardvark]

Like you said, the problem is that once the mail is delivered, the connection is closed, and the spammer is off the hook. There's two ways you can get around this.

One is to set up a Teergrube/Tarpit (it's easy using the Linux ipchains TARPIT target) on a machine that shouldn't receive any mail by SMTP. You can tarpit everything, and nothing will get lost. (I think this is something everyone should do; it'd be neat if this sort of functionality was built into those little Linksys/Dlink firewall boxes...)

The other possibility is to set up your mail server so that, as soon as the client connects to your SMTP server spam filtering begins, and as soon as a message is determined to be spam -- ie, when the client is still connected -- you start tarpitting. By contrast, a lot of spam filtering happens after the message has been accepted and the connection closed.

TarProxy is meant to do just that. Here's an excellent article on how it works. The project page says it's in the middle of a big redesign, so I'm waiting for that; once something comes out, though, I'll definitely be trying it out.

romancing the stone

[segment]

AOL has recently started banning SMTP servers who don't have reverse addresses, as seen on the NANOG lists. Personally there are so many methods to eliminate spam that an administrator can take I don't see what the issue is.

Me personally, if spam makes it through my filter, I ban off the offending address working my way up towards the class c - b - a. All attempts at a port 25 connection is drop point blank, http, https, etal are kept open. I also have dontspam#somefreemailaccount.com's to use for form shit. Once in a while when registering for say an upper-crust website account, I'll use something like msndoesntspam@mydomain.com to see who exactly is sharing my addresses, then null the account if I see anything odd coming in to that account, and never trust the site again. Procmail works the most wonders though.

good to hear

[Down8]

I've used my Y! acct as my main (personal) e-mail acct since sometime in late 1998/early 1999, so I'm very glad to hear about this. Hopefully it will help combat the 100-200 SPAM msgs I get per day. The Bulk Mail folder was a step in the right direction, as it does catch the majority of the crap, and allows me to delete it with a single click.

Thanks! Again! Yahoo!</elRegStyle>

-bZj

So where's the info?

[TrebleJunkie]

Okay, so they're developing a system that they'll release to open-source developers.... why not DEVELOP it in the open in the first place?

Are cycles that cheap?

[Frisky070802]

As I understand it, the proposal requires public-key encryption for every email sent, done by the sender at the time of sending. (If the "private key" -- something encrypted with the private key -- could be computed once and reused in every message, it could be copied and replayed by a forger.) This can dramatically raise the overhead associated with sending mail. Perhaps that overhead is reasonable, perhaps not.

Bala Krishnamurthy at AT&T Labs has given a number of talks recently, including to the IETF, on a spam disincentive program he calls SHRED. My understanding is that it uses offline cryptographic computation to amortize this overhead and distribute it to parties willing and able to devote the computational resources.

In any case, the tag line for this article had it right, standardizing this will be hard and heavy-hitters like Yahoo will need to take the lead. But a key problem is getting the new system to interoperate with the old.

identity based antispam is censorship tool

[esj at harvee]

a thing to remember is that if someone can prevent a spammer from communicating based on identity (or lack thereof), you can be silenced as well.

This is why I have put my efforts into sender-pay systems and specifically the camram project. We invite you to please come and join us in the effort to build a decentralized, user-friendly, freedom-of-speech supporting antispam system and hit spammers in the pocketbook.

camram antique documentation (too busy writing code to write new documentation)

Re:Open standards?

[Afty0r]

there should be no scope for someone to say "pay us or we won't accept email from you.

Why's that? If Yahoo doesn't accept email from anyone except the biggest 50 companies in the world who could afford to take part, you can place a bet that there won't be many people using their email service anymore.

This needs HYPE

[Jesrad]

Seriously. This solution needs the cooperation of most. It is the exact solution I have been longing for, and to be successful when it is released it needs every significant domain to follow suit. Your ISP won't use Domain Keys ? Rant to them till they do ! They still won't ? Set up your own MX and sign in to the certified network. Have your friends and relatives get aboard too.

As soon as the certified network is considered a valid alternative to the current spam-ridden, scam-infested open email exchange system people will switch boards in a blink... provided it is easy enough to get a certificate.

Yahoo beats eariler proposals? I hope not.

[kerubi]

Would you rather choose a Yahoo product over an open standard that is under development? I'm speaking of AMTP, of course. (See AMTP author's site).

Yahoo's size doesn't give that much weight to their proposal. Yahoo's email is not used in business to business communication (do not count hot dog stands as businesses), so businesses can just aswell block everything that originates from *@yahoo.com if it is not directed to their consumer service department.

Also, reverse mx records provide much of the same benefits with minimal alterations needed to current email infrastructure. One DNS record added and small change in MTA software.

If Yahoo would really like to do a service to the internet community, they should rather consider looking AMTP and reverse mx records.

I ended spam

[RexDevious]

I don't know if anyone's interested, but over the weekend I put together a white-list, white-phrase, auto-response human-sender verification system which has been 100% effective so far. Meaning that everything it identified as spam was (which in now bounces instead of holding for my perusal) and every email from both human strangers and machine generated email from companies I wanted to hear from got right through. I wanted to write a program that would do this automatically for my web host, but even though it wasn't an option (they used off the shelf Ipswitch software that they couldn't reprogram), I was still able to set it up using existing filters. Which means you probably can too.

If you want to know how it works, either to use it or to find a flaw, say so and I'll post the specs.

Re:User account verification

[Anonymous Coward]

You must be one of these exim or milter-sender users who think that callbacks for sender verifications are going to help. Well, here's a news flash: you're just encouraging them to forge domains which have delayed bounces. Now let's say Yahoo and everyone else starts bouncing mail with 5xx during the transaction, even on their secondary MXs.

The spammers will switch to using REAL source addresses. You know, the same ones they're sending the spam to? It's easy - instead of making up a u@h as the sender, they just pull one from their list of "probably valid" destinations.

Now your server calls back, does the check, and hey! It's a valid account! OK, let the mail through! *THUD* You just got spammed.

You may think I'm making this up, but it's already happening. Some of them even try to correlate the recipient with a similar sender. That is, bobbyjones@example.edu might get mail from bobbysmith@example.com. At least one spammer looks for another address with the same first few letters when making up a sender. These are all valid accounts.

I'm fed up with all of these callbacks that are implemented so stupidly. imail@verizon.net shows up here every time a mail goes out from one of my mailing lists. They've obviously never heard of a cache. Sourceforge's MTAs are no better. It's damned annoying, and it's no better than those idiots who do dictionary attacks, since it looks identical in the logs.

Re:Public key spam control - technical implication

[hawaiian717]

Definately a problem. One possibility would be to store the private key on a smart card, not on the machine itself, and make it so that the key cannot be removed from the card. The card itself does the digital signing. Problem here is that we'd suddenly need everybody to get smart card readers on their computers. But it would mean we could still sign our email from anywhere.

Here's an article that gives an overview of doing this with smart cards.