SPF To Be Integrated With MS 'Caller ID' System

An anonymous reader submits "CNET's news.com is reporting 'An ongoing effort to consolidate antispam authentication schemes took a big step forward with the merging of Sender Policy Framework (SPF) and Microsoft's Caller ID for E-mail.' This is potentially good news." For more background, here are three previous mentions of Microsoft's proposed Caller ID-style system.

Comments

Good they've merged. Why XML ?

[Space cowboy]

The combined SPF and Caller ID, which has yet to be named, will use XML (Extensible Markup Language) to let Net service providers post IP addresses in the Domain Name System, the giant database that translates alphanumeric domain names like "news.com" into numerical IP addresses for Web servers.

I have yet to see a good reason why XML is the choice for the payload. I'm not really buying the argument that it's easier to shoehorn XML into TXT fields rather than have another tag. Either way, in order to implement the proposal the MTA authors will have to do some work, and I don't think there's much to choose between the two...

I still can't really rid myself of the nagging suspicion that the extensibility of an XML-driven anti-spam system plays into the hands of 'embrace and extend' that MS has used successfully since time began...

On the other hand, getting some authentication that it really came from where it says it came from will be very useful. The corollory is that 'owning' a mail server will become a higher priority for the hacker/spammer coalitions. Look for more attacks on MX machines if this becomes widespread...

Next on the agenda - get everyone to use digitally-signed certificates :-)

Simon

Re:Good they've merged. Why XML ?

[Allen Zadr]

That's a good point, but I see the eXtensability of XML as the power here. It would be relatively simple to extend the Email-Caller-ID XML specification to include an <spf:details/> tag. Which, would naturally allow for other extensions as well.

Remember, too, that XML is not a Microsoft technology. It's a W3C technology that Microsoft also uses. That's a big difference. If this proposal included a .NET extension to my Mail server, then I'd be suspicious.

My question is: How will SPF or Email-Caller-ID take into account mailing lists? Will this block Emails from my address sent through sourceforge.net's many fine list servers?

Re:Good they've merged. Why XML ?

[Allen Zadr]

Actually - nevermind. I found a reason why I can't impliment this technology, ever.

Use of this technology requires submitting to a Microsoft license. This license allows distribution (but not re-distribution), and is not compatible with the GPL. That is to say, no GPL mail server will ever be able to directly impliment checks for this.

From the license (forgive typos, I typed this from the PDF):

2.2. Source Code Distribution You also have a nontransferable, non-sublicenseable, personal, license to distribute or otherwise disclose source code copies of such Licensed Implementation licensed in Section 2.1 only if You (i) prominently display the following notice in all copies of such source code, and (ii) distribute or disclose the source code only under a license agreement that includes the following notice as a term of such license agreement and does not include any other terms that are inconsistent with, or would prohibit, the following notice:

"This source code may incorporate intellectual property owned by Microsoft Corporation. Our porvision of this source code does not include any licenses or any other rights to you under any Microsoft intellectual property. If you would like a license from Microsoft (e.h. rebrand, redistribute), you need to contact Microsoft directly."

That, my friend, is embrace, extend and assimilate. Nothing under strict GPL can impliment this natively. IIRC, SPF (Sender Permitted From) did not have source restrictive terms.

SPF 30

[FireBug]

Why on earth are they integrating SPF into technology? I mean, it's not like Slashdotians ever go out into the sun or anything...

Laugh! It was a joke!

Re:Why not XML?

[DrPizza]

Because, since XML is not a format (but rather a standardized way of creating one's own formats) the issue of "creating a format" is not solved by the decision to use XML.

What XML "wins" is off-the-shelf parsers; one still needs to write some amount of code to convert dumb XML (elements and attributes and all that crud) into something with semantic meaning to your application.

For a simple application like this it's not clear that the overheads of XML (both in terms of size, computational complexity, and programmer overhead to make the aforementioned conversion) are at all worthwhile.

Re:Sounds like a truly awful idea

[Albanach]

This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

If spammers have to buy new domains for every couple of thousand spams they face a big problem.

• Firstly it all adds to the cost - with tiny response rates you'd have to imagine the margins are tight.
• Secondly if they have to buy domains they need to pay for them - that leaves a physical paper trail to spammers, now legislation can help.
• Thirdly we have plenty of existing technology such as black hole lists that will be a lot more effective if lots of spam comes from one newly registered domain.
• Fourthly we don't need the entire web to be using SPF for it to become effective. If you receive spam from an AOL account it's now possible to easily check if it in fact came from an AOL mailserver. That other people haven't yet implimented SPF is irrelevant - we can use the technology to our advantage today. Once it's widely implimented we can even start to apply a small spamassassin score to not SPF confirmed mail. As adoption grows we can increase that score still further.

Spam solution already exists

[Rashkae]

Spam would very quickly cease being a problem if mail clients were configured to start using PGP (GPG) keys and signatures by default. There is no need to re-invent or even change the e-mail RFC's.

Very simply, people can choose whether they want to receive unsigned e-mail, or accept sinatures from unkown keys. We'll eventually start building a web of turst (mistrust), such as, being able to automatically accept a key signed by some people or orgs, and similarly, blacklisting keys.

I could very easily, for example, instruct unknown senders (people who aren't in my contact list yet) to download my public key from a specified location to encryp a message that would bypass my filtes. Only a person who followed the instructions would be able to send me an unsolicited message.

Re:Spam solution already exists

[gclef]

You know, people have been saying that for almost a decade now. Face it: digitally signed email isn't working. Key management is a pain in the ass, the bootstrapping necessary to check user's keys is a mess, and it doesn't really gain you that much in the end. We've had 10 years to get signed email working, and it didn't happen. Time to find another way (whether it's this SPF or something else is a point for argument).

breaks forwarding

[close_wait]

I dislike SPF because it breaks forwarding. There is a "workaround" but that's required on every MTA in the world that allows forwarding, and is intensely ugly - it requires adding a bunch of garbage to the sender address, and also requires the MTA to main a cache of forwarded addresses so that bounces can be passed back down the chain.

The problem is this. Suppose AOL start adding SPF records to their DNS, saying effectively 'only the following IP addresses are authorized to send @aol.com emails. Suppose also that Hotmail start rejecting emails from SPF domains where the IP addresses don't match. Now suppose that joe@small.biz is going to be away from the office for a couple of weeks, so he gets the small.biz mail server to forward his emails to his hotmail account. At this point anyone from AOL who emails him will find the emails bouncing (although if they're from AOL, this may not be such a bad thing...)

Re:Sounds like a truly awful idea

[Chang]

> SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted

This is false. There is no requirement for every domain on the internet to adopt SPF before it becomes useful.

Instead each domain owner decides when to flip the switch on for SPF enforcement for their individual domains. Since 14,000 domain already have valid SPF records and many of them have enabled enforcement, SPF is useful for not accepting worthless spoofed emails TODAY. Not in some far off future.

Re:Sounds like a truly awful idea

[FattMattP]

But wait -- SPF doesn't block spam!

Correct. It's not meant to. SPF's goal is to prevent domain name forgery. Blocking spam, if it does that, is a side effect. Authenticating the sender is the primary goal.

It just blocks spam where the From: is not right.

No, that's what MS "Caller-ID" does. SPF checks the MAIL FROM in the SMTP transaction. Think of it this way, SPF does its checks on the envelope and caller-id does its checks on the header.

The only potential real benefit, I suspect, would be to make phishing harder.

That's the point.

Re:dynamic dns users

[ahodgson]

In theory, SPF should make it easier for these people to send E-mail. They can publish a valid SPF record for their domain, which should make mail from their system more trustworthy than mail from dynamic IP space is generally. Ie, the reason people block mail from dynamic IP space is because of the incredible amount of crud coming from trojanned Windows machines in that space.

If a real sender can somehow distinguish themselves via a valid SPF record, they might actually have better luck sending mail than they do now.

AOL and MS say: publish SPF records

[wayne]

AOL just added a webpage saying that you should publish SPF records if you want to be whitelisted with AOL.

The MicroSoft Caller-ID/SPF merger proposals say that SPF records will be honored, so you can publish them without fear of losing support.

So, go ahead and publish SPF records.

MicroSoft supporting SPF records is a really smart move. Last week, I posted results of a survey of 1.3 million email domain names to the IETF MARID mailing list. Now that I'm back from the MARID meeting, I just finished a survey of Caller-ID records. There appears to be about a factor of 500-1000 more domains that have published SPF exclusively than Caller-ID exclusively and only a tiny fraction of the 1.3 million domains have published Caller-ID records. In short, MicroSoft isn't changing to support SPF records because they are better (I think they are), but because it is an acknowledgement that MicroSoft's Caller-ID hasn't caught on.

Meng Weng Wong (the SPF author) and MicroSoft are still discussing how exactly this merger will work on. I personally don't see any reason to support XML right away. MicroSoft has not come out with a single concrete extention that can't be done with SPF already.

I also think that there are alternatives to the complex Caller-ID algorithm and that doesn't require every Ezmlm and other mailing lists to upgrade their software. From the research that I've done (and yes, this is something I have really researched), there appears to be far more mailing lists broken by MS's Caller-ID system than email forwarders broken by SPF.

(I'm the author of libspf-alt and the maintainer of the trusted-forwarder global whitelist. So, now you know why I have researched this stuff so much.)