SPF To Be Integrated With MS 'Caller ID' System

An anonymous reader submits "CNET's news.com is reporting 'An ongoing effort to consolidate antispam authentication schemes took a big step forward with the merging of Sender Policy Framework (SPF) and Microsoft's Caller ID for E-mail.' This is potentially good news." For more background, here are three previous mentions of Microsoft's proposed Caller ID-style system.

Comments

The only good anti-spam solution

[FictionPimp]

Stop using email. Require all communication to come on 11X17 inch plastic sheets sent via fedex. Thats what I do.

Re:The only good anti-spam solution

[Da Fokka]

Hail, brother!

We don't use cars at my business. They're too cumbersome. I still prefer the good ol' horse-and-carriage. They can make a mess but doing so in front of the offices of the guys who keep sending us directmail solves that problem as well.

But what is this 'fax' you are talking about?!

Re:The only good anti-spam solution

[BlackHawk-666]

I've also simplified my communications system by using two tins and a piece of string. If vendors want to call they have to buy their own tin and some string and tie it into the string-ter-net. They need to replace the string whenever local cats play with it or after six months, because it rots quickly but I sure as hell don't get spam anymore :-) Time to move in with the Amish communities.

Good they've merged. Why XML ?

[Space cowboy]

The combined SPF and Caller ID, which has yet to be named, will use XML (Extensible Markup Language) to let Net service providers post IP addresses in the Domain Name System, the giant database that translates alphanumeric domain names like "news.com" into numerical IP addresses for Web servers.

I have yet to see a good reason why XML is the choice for the payload. I'm not really buying the argument that it's easier to shoehorn XML into TXT fields rather than have another tag. Either way, in order to implement the proposal the MTA authors will have to do some work, and I don't think there's much to choose between the two...

I still can't really rid myself of the nagging suspicion that the extensibility of an XML-driven anti-spam system plays into the hands of 'embrace and extend' that MS has used successfully since time began...

On the other hand, getting some authentication that it really came from where it says it came from will be very useful. The corollory is that 'owning' a mail server will become a higher priority for the hacker/spammer coalitions. Look for more attacks on MX machines if this becomes widespread...

Next on the agenda - get everyone to use digitally-signed certificates :-)

Simon

Re:Good they've merged. Why XML ?

[Anonymous Coward]

not only that but xml will slow down the dns lookup since the answer will exceed the udp limit and will have to switch over to tcp.

Re:Good they've merged. Why XML ?

[Allen Zadr]

That's a good point, but I see the eXtensability of XML as the power here. It would be relatively simple to extend the Email-Caller-ID XML specification to include an <spf:details/> tag. Which, would naturally allow for other extensions as well.

Remember, too, that XML is not a Microsoft technology. It's a W3C technology that Microsoft also uses. That's a big difference. If this proposal included a .NET extension to my Mail server, then I'd be suspicious.

My question is: How will SPF or Email-Caller-ID take into account mailing lists? Will this block Emails from my address sent through sourceforge.net's many fine list servers?

Re:Good they've merged. Why XML ?

[thogard]

Cracker will love the xml format. It turns out that the record size will exceed the UDP packet size for DNS records so they get upsized to use TCP packets.

The thing is how many people allow TCP packets on port 53 on their firewall? There is no reason execpt to talk to your second-dns records. All other cases should be turned off but this requires that it be turned on.

Re:Good they've merged. Why XML ?

[Allen Zadr]

Actually - nevermind. I found a reason why I can't impliment this technology, ever.

Use of this technology requires submitting to a Microsoft license. This license allows distribution (but not re-distribution), and is not compatible with the GPL. That is to say, no GPL mail server will ever be able to directly impliment checks for this.

From the license (forgive typos, I typed this from the PDF):

2.2. Source Code Distribution You also have a nontransferable, non-sublicenseable, personal, license to distribute or otherwise disclose source code copies of such Licensed Implementation licensed in Section 2.1 only if You (i) prominently display the following notice in all copies of such source code, and (ii) distribute or disclose the source code only under a license agreement that includes the following notice as a term of such license agreement and does not include any other terms that are inconsistent with, or would prohibit, the following notice:

"This source code may incorporate intellectual property owned by Microsoft Corporation. Our porvision of this source code does not include any licenses or any other rights to you under any Microsoft intellectual property. If you would like a license from Microsoft (e.h. rebrand, redistribute), you need to contact Microsoft directly."

That, my friend, is embrace, extend and assimilate. Nothing under strict GPL can impliment this natively. IIRC, SPF (Sender Permitted From) did not have source restrictive terms.

Re:Good they've merged. Why XML ?

[Feyr]

imagine the pain for all those losers using tinydns in a default setup (eg, no tcp listening deamon by default)

Re:Good they've merged. Why XML ?

[pjrc]

The plan appears to be to support both XML and v=spf1 formats (according to the summary of the meeting between the SPF and Microsoft folkw). The expectation is that the vast majority of sites will publish the simple and compact v=spf1 format for the forseeable future. XML allows extensibility for the unforseen future that follows afterwards.

Re:would be nice

[YetAnotherDave]

??

the user can set up their system to reject anything they damn well feel like rejecting.

the hard part is the testing, not the actions taken after testing...

In the year 2000....

[ArmenTanzarian]

You will be truly disturbed to find out that your Grandmother is apparently the one who's been pushing for you to use Cialis.

SPF 30

[FireBug]

Why on earth are they integrating SPF into technology? I mean, it's not like Slashdotians ever go out into the sun or anything...

Laugh! It was a joke!

Re:SPF 30

[Pxtl]

Cute. Still, I do wonder why they even advertise this as "caller ID" - after all, everyone knows caller id can be spoofed. Really, the existing e-mail header is the same as caller ID. In theory it says who's calling, but anyone dedicated can get around it.

Why don't they just call it like it is? A secure substitute for the "source" field of the e-mail header.

Why not XML?

[Mz6]

Why build a new format when there is one already available that would suit their needs?

Re:Why not XML?

[lpontiac]

XML is not a format. It's a metaformat.

Re:Why not XML?

[DrPizza]

Because, since XML is not a format (but rather a standardized way of creating one's own formats) the issue of "creating a format" is not solved by the decision to use XML.

What XML "wins" is off-the-shelf parsers; one still needs to write some amount of code to convert dumb XML (elements and attributes and all that crud) into something with semantic meaning to your application.

For a simple application like this it's not clear that the overheads of XML (both in terms of size, computational complexity, and programmer overhead to make the aforementioned conversion) are at all worthwhile.

Re:Why not XML?

[silas_moeckel]

How about because XML is pretty bloated and an embrase and extend sort of language thats designed to let people insert new things without breaking old things. Meaning MS or anytbody else can introduce new tags that arent compatable with somebody elses tags, they shouldent collide but the bloat gets pretty bad when you have to support 2 3 4 20 differnet version of the same tags to be compabible with everybody. XML is good for programs that should be embraced and extended. Internet related thigns should come from RFC and be fixed one set of rules, at least I think so. One set of rules is what has gotten us email that works around the world in less than 20 years.

Re:Why not XML?

[BlackHawk-666]

Plain old text files might be better I reckon. e.g.

MX:MyFatServer:212.169.24.12,212.169.24.13
MX:MyOtherServer:212.169.24.16

Not taxing to parse, simple, grepable, fast, lightweight, human readable. Even if you work for MS you should be able to write the code to parse this ;->

Re:Why not XML?

[Russ Nelson]

Uhhhhhhhh, because a DNS packet is limited to 512 octets??
-russ

Sounds like a truly awful idea

[isdnip]

Now it sounds like a bad idea for both semantic (what it does) and syntactic (how it is coded) reasons!

The syntactic bit is easy -- XML is hardly appropriate for a DNS function. Mickeysoft is running around patenting XML schemas, and it adds a new layer of complexity to DNS. But then bad syntax is usually dealt with by code.

The semantic bit is worse -- SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted. But wait -- SPF doesn't block spam! It just blocks spam where the From: is not right. Spammers can still create new domains on a hit-and-run basis, and they'll pass SPF. So it's another blast-proof vault door stuck onto a grass hut, a silly waste of time. The only potential real benefit, I suspect, would be to make phishing harder. The address will have to be slightly different from the spoofed domain. But that leaves plenty of opportunity to create deceptively-close hit-and-run domains (like, say, pay-pa1-approva1.com).

Worse, of course, is the collateral damage. How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server? My "from" address is an alias, not a real sender, and I use it to send via more than one ISP, depending on where I am. SPF seems to make this a lot harder, thereby forcing more people to put their ISPs' name in the From: field, rather than their own. Since email is not portable, a user's address is lost when they change ISPs, or when their ISP changes names (mediaone->attbi->comcast). Personal domains (forwarded via a service like mydomain) solve this. Will SPF kill mydomain?

I repeat what I've said before. The only way to kill spam is to stop having all email be totally, absolutely, "free" of charge in any quantity. This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

Re:Sounds like a truly awful idea

[jafomatic]

It just blocks spam where the From: is not right.

Correct, but what this means is that there should now be some level of accountability to the originator. One of the biggest complaints I would have, looking into email-spam as a problem, would be that there's no way to hold a sender accountable when the true origination of the message is unknown. If I understand the proposal correctly, that accountability will at least be marginally present.

The ability to spoof this system is another issue entirely. The system's intention would appear to add value.

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

A valid concern; as I read it, this wouldn't be possible. Either that or we're overlooking exactly what makes up a valid original sender.

Re:Sounds like a truly awful idea

[Albanach]

This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

If spammers have to buy new domains for every couple of thousand spams they face a big problem.

• Firstly it all adds to the cost - with tiny response rates you'd have to imagine the margins are tight.
• Secondly if they have to buy domains they need to pay for them - that leaves a physical paper trail to spammers, now legislation can help.
• Thirdly we have plenty of existing technology such as black hole lists that will be a lot more effective if lots of spam comes from one newly registered domain.
• Fourthly we don't need the entire web to be using SPF for it to become effective. If you receive spam from an AOL account it's now possible to easily check if it in fact came from an AOL mailserver. That other people haven't yet implimented SPF is irrelevant - we can use the technology to our advantage today. Once it's widely implimented we can even start to apply a small spamassassin score to not SPF confirmed mail. As adoption grows we can increase that score still further.

Re:Sounds like a truly awful idea

[AlienFactor]

As for email not being portable, I'd expect more personal domains (or permanent email addresses) as a result.

<Conspiracy>Funny, that's exactly the business that SPF's author (pobox.com) is in.</Consipiracy>

Re:Sounds like a truly awful idea

[Chang]

> SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted

This is false. There is no requirement for every domain on the internet to adopt SPF before it becomes useful.

Instead each domain owner decides when to flip the switch on for SPF enforcement for their individual domains. Since 14,000 domain already have valid SPF records and many of them have enabled enforcement, SPF is useful for not accepting worthless spoofed emails TODAY. Not in some far off future.

Re:Sounds like a truly awful idea

[djmurdoch]

The real problem is compliance, until 99% of mail servers provide this data, I can't reject mail from non SPF listed domains.

There aren't many tests which are perfect spam identifiers with no false positives. You should use the SPF compliance as part of a scoring scheme. Messages that fail SPF are more likely to be spam than messages that pass, so they get a higher spam score. If the score exceeds a threshold, mark it as possible spam. If it exceeds a higher one, delete it unread.

This is the strategy Spamassassin allows, and it works really well, especially if you let the scoring be adaptive (i.e. use "Bayesian tests").

Re:Sounds like a truly awful idea

[FattMattP]

But wait -- SPF doesn't block spam!

Correct. It's not meant to. SPF's goal is to prevent domain name forgery. Blocking spam, if it does that, is a side effect. Authenticating the sender is the primary goal.

It just blocks spam where the From: is not right.

No, that's what MS "Caller-ID" does. SPF checks the MAIL FROM in the SMTP transaction. Think of it this way, SPF does its checks on the envelope and caller-id does its checks on the header.

The only potential real benefit, I suspect, would be to make phishing harder.

That's the point.

Re:Sounds like a truly awful idea

[leviramsey]

An SPF record is a record stating what hosts are registered senders of mail from a given domain. There's nothing stopping you from adding, say, smtp.verizon.net or whatnot to the record.

Of course, there's also SMTP AUTH...

Re:Sounds like a truly awful idea

[Nugget]

SPF enforces the envelope sender (from the smtp "mail from:" comman", not the header "From:" line. I believe if you look, you'll see that your mailing list mails have the envelope sender set to the mailing list's sender address.

It works just fine.

Re:Sounds like a truly awful idea

[Smallpond]

You don't need to have 99%. You can just delete mail from spoofed SPF-compliant domains, from non-resolving domains, and from domains in blocklists. This will force the spammers to send "from" real, unblocked, non-SPF domains. These will quickly get lots of bounce messages from spam they didn't send and get added to block lists. This will encourage the holdout domain owners to use SPF. Critical mass ensues. All domains will end up using SPF if they want to send mail. If not, they end up in block lists.

Re:Sounds like a truly awful idea

[CustomDesigned]

Worse, of course, is the collateral damage. How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server? My "from" address is an alias, not a real sender, and I use it to send via more than one ISP, depending on where I am. SPF seems to make this a lot harder, thereby forcing more people to put their ISPs' name in the From: field, rather than their own. Since email is not portable, a user's address is lost when they change ISPs, or when their ISP changes names (mediaone->attbi->comcast). Personal domains (forwarded via a service like mydomain) solve this. Will SPF kill mydomain?

Your objection mentions a multitude of configuration scenarios. I will address a few:

• A home user with a personal domain simply lists their ISP as a valid sender in SPF.
• A travelling salesman with a laptop should use a VPN or SMTP AUTH or webmail.
• A travelling salesman using anonymous PCs wherever he finds them is crazy. But if he insists, he should use webmail. That is probably less of a risk than entering an SMTP AUTH user and password.
• SPF does not prevent you from putting whatever you want into the From: header. (Yahoo domain keys addresses that.) SPF only authenticates the envelope sender. The envelope sender is used for bounces, return receipts, and the like.
• If you can't authenticate from the field in any way, you could always publish SPF that lists your home system and leaves everything else neutral. Since anyone, including me or a spammer can currently send email claiming to be from you, that accurately reflects the situation.

Re:Sounds like a truly awful idea

[jaseuk]

The envelope sender would be mailinglist-owner@mail.sourceforge.net (or similar).

Re:Sounds like a truly awful idea

[mr_mischief]

So the people who break into Windows boxes with weak security and send through the proper server for that box but advertise biggerpenis.com in the email still get their mail delivered, and have it "verifiably" traced to their victim.

This is worse, not better.

Re:Sounds like a truly awful idea

[pjrc]

This statement is simply wrong:

SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted.

I'll type out an example, to show you exactly how it is already working and rejecting forgeries TODAY, without being mandatory, and long before widespread implementation.

Suppose your MTA receives a message from a spammer who is impersonating me. The message claims to be from "paul@pjrc.com", but it isn't. Your SPF-aware MTA (or spam filter) does a TXT query to my domain, and because my domain is one of the 14500-some returning SPF, you'll get this:

paul@preston ~ > host -t txt pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

pjrc.com text "v=spf1 a:outgoing.pjrc.com mx -all"

That means there's only two sources that are authorized to send messages for pjrc.com. So your software will first do a lookup on "outgoing.pjrc.com", sorta like this:

paul@preston ~ > host outgoing.pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

outgoing.pjrc.com has address 168.143.160.91

If the IP number of the MTA sending the message is 168.143.160.91, then you know the message is coming from my server. If not, then the next authorized source is checked. Your software will do a MX lookup, like this:

paul@preston ~ > host -t mx pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

pjrc.com mail is handled by 10 mail.pjrc.com.

Then, the name "main.pjrc.com" is looked up, such as:

paul@preston ~ > host mail.pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

mail.pjrc.com has address 168.143.173.65

If the MTA sending the message has the IP number 168.143.173.65, then you also know the message came from an authorized server for my domain, and thus the message is not a forgery by a spammer.

However, the last part of the SPF record was "-all", which means NO OTHER sources are authorized to send pjrc.com's email. So if the MTA sending the message is any IP number other than 168.143.160.91 or 168.143.173.65, then you know it's being sent from an authorized source, and thus it's a forgery (or I've messed up my SPF record).

Please take notice that for this to work, only you and I need to have SPF implemented. All the communication was between your receiving MTA (or SPF-aware spam filter), and my the DNS server for my domain. No other servers or other hosts on the internet needed to implement SPF in order for you to discover that a forged message isn't really from me.

Of course, only about 14500-some domains are currently publishing SPF records... so you can only detect forgeries if the spammer is foolish enough to forge one of our domain names. However, "aol.com", a very commonly forged name, is on the list. Likewise, "hotmal.com" is publishing caller ID (which works more or less the same as SPF, but with different syntax and with the minor distinction of sender vs from).

The bottom line is that your statement is incorrect. SPF does not have to be mandatory to function. It already works for those cases where the receiver bothers to check and the (claimed) sender publishes a SPF record. You can already reject all forgeries from 14500-some domains, including little ones like mine and big ones like AOL and Hotmail... and this is fully functional TODAY, long before 100% compliance is reached.

Stupid /., ruining my joke

[numbski]

[erwin: ~] root# *67 && mail -s "enlarge your elbows" mpost4@mikeoconnor.net << cat enlarge.txt ;)

Re:i still don't trust it...

[WormholeFiend]

I think their main motivation is to stop the spread of virus attachments... anytime there's a MS-targetting worm going around, using similar distribution processes as spam, it creates an additional workload, not to mention that it tars Microsoft's image.

From my point of view, the spam cleanup would just be collateral.

Lets hope they ditch the patent then.

[patthoyts]

having looked into doing an implementation of CallerID to add to an SPF parser -- I have come away with crossed eyes trying to work out if I'm allowed to release a BSD licensed implementation of this. I think maybe I can -- I'm pretty sure I can't release a GPL implementation though.

Damn, now where did I put that lawyer....

Re:i still don't trust it...

[jellomizer]

Yes more people who will still use their email clients vs. switching to ones that block spam mail an other way. Plus they can say. We Stopped spam. Plus sience they have made the protocal they will be the first to implement it in their email servers and their clients giving a month lead over the rest, and obtaining market share and lockin.

Re:IP addresses?

[Allen Zadr]

No that won't work.

Because the SMTP protocol requires two-way communications, the packet has to have a valid IP address, so that the TCP/IP traffic can go between the two mail servers (sender, reciever). Because of this, you are guaranteed that the IP address is correct to within a given sub-net. Within that sub-net, yes, spoofing is possible (convince the router that you are the real 131.107.3.124). This is definately "close enough" to usually be accurate.

You could've read all about it last week...

[jgardn]

...in a comment I made here.

Basically, this is a simply classic way to "embrace and extend" Microsoft's Caller ID. Before the flag day, SPF will work the way it is now. After the flag day, which will probably occur later rather than sooner, SPF will have all the functionality of Caller ID. The idea of allowing both XML and text descriptors is simply brilliant. Microsoft wanted to force everyone to use XML, but now you have a choice. I believe most (like 99.9%) will use the text descriptors, both because it is easier and because it is sufficient for 99.9% of the cases.

The net result is Microsoft can't claim ownership anymore. Caller ID will be a footnote in the history of email authentication.

Re:You could've read all about it last week...

[Russ Nelson]

Uhhhhhhhh, so now everybody has to have both an SPF and an XML parser in their MTA? And that's an improvement ... how?
-russ

Re:Boycott of Microsoft's Caller ID for E-mail

[Rayban]

I heartily agree! It's good to see them cooperating, but I hope that the final license has a royalty-free patent grant with no attribution clauses.

If the two camps agree, this will speed up adoption of SPF records enormously.

SPF is harmful. Adopt it.

[Anonymous Coward]

Check this article for an interesting commentary on SPF.

Spam solution already exists

[Rashkae]

Spam would very quickly cease being a problem if mail clients were configured to start using PGP (GPG) keys and signatures by default. There is no need to re-invent or even change the e-mail RFC's.

Very simply, people can choose whether they want to receive unsigned e-mail, or accept sinatures from unkown keys. We'll eventually start building a web of turst (mistrust), such as, being able to automatically accept a key signed by some people or orgs, and similarly, blacklisting keys.

I could very easily, for example, instruct unknown senders (people who aren't in my contact list yet) to download my public key from a specified location to encryp a message that would bypass my filtes. Only a person who followed the instructions would be able to send me an unsolicited message.

Re:Spam solution already exists

[gclef]

You know, people have been saying that for almost a decade now. Face it: digitally signed email isn't working. Key management is a pain in the ass, the bootstrapping necessary to check user's keys is a mess, and it doesn't really gain you that much in the end. We've had 10 years to get signed email working, and it didn't happen. Time to find another way (whether it's this SPF or something else is a point for argument).

Re:Spam solution already exists

[Rashkae]

Besides, how can you say it's failed for 10 years, when it was never *even* tried. Thunderbird is the first mainstream e-mail client I've installed that supports Encryption out of the box. But calling it mainstream is a stretch, and it doens't really include key management. If MS is serious about fighting spam and improving security, I think they should start with *Implementing* technology and standards that have been around for 10 years.

Re:Spam solution already exists

[Rashkae]

Well, I have to disagree with your proposal. I'm one of those stubborn bastards who sends mail through my own e-mail server.

1) I really hate webmail.

2) As the administrator for our small network, I prefer to have direct access to the e-mail server logs for security and verification purposes.

3)Your suggestion would still not solve the spam problem.

As annoying as the constant Viagra and porn ads are, not to mention offensive to some people, 3 years from now, those will be pleasant memories compared to what's going to happen once MS and friends convice the powers that be (FCC et al) that the real problem is porn and scams, and therefore the solution is to create a mechanism for legit businesses to send legit messages. I forget the statistics, but I remember a post somewhere about how many spams we'll be getting per day if every business in the US sent 1 unsolicted message per year.... it was frigthening.

((Besides, the Viagra ads are starting to get artful and entertaining.))

Re:Spam solution already exists

[Ageless]

This is the fatal flaw with any kind of mail encryption or signing system. It's not easy to use, and it's not easy to make it easy to use. Someone has to vouch for the fact that you are who you say you are. Current systems do this by having you apply for and receive an X.509 certificate from the likes of Verisign. Everyone trusts Verisign so if Verisign says that I am Jason von Nieda, you can be guaranteed that I am Jason von Nieda. The problem with this, of course, is that before I can use the system I have to go through that process.

If there were a real demand for this, support could be added to mail clients to quickly and easily request a certificate from a certifying authority, but you still have the problems of losing your certificate, forgetting your password, needing to send mail from another computer and so on.

This is why client/personal based authentication fails. Users just generally can't be bothered. Making it the sysadmin's burden, or the ISP's makes it possible to quickly secure and authenticate a large group of people instead of just one person here and there that's willing to spend some time on it.

Re:Spam solution already exists

[OrenWolf]

It would stop even faster if all email servers simply had a rule that ignored any transfer requests from any machine that DID NOT have a Fully Qualified Domain entry and the top lever is not mail. or mail2. or mail3.

By extension, then, you figure if only ISP email servers could send mail, spam would be greatly reduced.

Congratulations! You just explained why SPF is a good idea! The whole point of SPF is to point out which servers for a given domain are allowed to send email.

if you are a roaming road warrior then you are forced to use webmail or VPN into the network.

Not heard of SMTP AUTH? Have your ISP setup AUTH on both ports 25 and 587 (MSA, to get around port 25 filters) and you can now relay your mail, via your ISP server, from anywhere.

Re:Spam solution already exists

[pjrc]

if mail clients were configured to start using PGP (GPG) keys and signatures by default.

If you reject messages without a PGP signature, spammers will simply sign their messages.

If you reject based on the signing author being a known spammer, spammers will simply generate a new key for each message. This isn't a computational burden (as it is in PGP) if the keys aren't generated in a secure manner.

If you reject all unknown senders, people unknown in your "web of trust" will be rejected.

instruct unknown senders (people who aren't in my contact list yet) to download my public key from a specified location to encryp a message that would bypass my filtes.

These challenge-response systems are already in use today, by people willing to be rude to previously unknown senders. They suffer from all sorts of problems (out-of-office autoresponders, automatically generated shipping notices for online purchases, and other legitimate automated senders). How is adding the complexity of PGP an improvement?

The original statement is missing one important word "if [all] mail clients were configured".

This is an idea that just won't work if any significant number of senders still send out plain, old email as they currently do.

Contract with SPF and MS's Caller-ID, which are effective at detecting forgey when only the (claimed) sender publishes a SPF and/or Caller-ID DNS record and the receipient checks it.

To be successful, an change needs to provide value for those who have implemented it, even before widespread adoption occurs. SPF and MS Caller ID meet this goal, as people checking SPF and Caller ID can already reject forgeries claiming to be among 14000-some domain names (including AOL and Hotmail), and domain name admins who publish their SPF and/or Caller ID records are already receiving some protection from having their names forged.

The scheme "let's all switch to PGP" doesn't allow receivers to filter out messages until almost all legit senders sign their messages, and senders who sign their messages don't get much value from the signature until almost all receivers reject unsigned messages. Even then, PGP still suffers from the problem of unknown signatures, requiring a massive worldwide trust database or all the problems of callenge-response negotiation for new contacts.

PGP is an excellent answer to protecting privacy and proving strong authentication. Saddly, crypto only answers the question "is this message certainly authentic", but the question that needs answering is instead "is this message almost certainly a forgery".

Re:Boycott of Microsoft's Caller ID for E-mail

[Rayban]

Since the parent of my last comment was rated as a troll (I don't agree with that) here is the text:

---
Make sure you let them know that patents on email technology are unacceptable. Merging is okay, let's just keep the SPF license, not the Microsoft one.
---

It's just saying that patents on email tech are unreasonable. That's pretty reasonable to me.

breaks forwarding

[close_wait]

I dislike SPF because it breaks forwarding. There is a "workaround" but that's required on every MTA in the world that allows forwarding, and is intensely ugly - it requires adding a bunch of garbage to the sender address, and also requires the MTA to main a cache of forwarded addresses so that bounces can be passed back down the chain.

The problem is this. Suppose AOL start adding SPF records to their DNS, saying effectively 'only the following IP addresses are authorized to send @aol.com emails. Suppose also that Hotmail start rejecting emails from SPF domains where the IP addresses don't match. Now suppose that joe@small.biz is going to be away from the office for a couple of weeks, so he gets the small.biz mail server to forward his emails to his hotmail account. At this point anyone from AOL who emails him will find the emails bouncing (although if they're from AOL, this may not be such a bad thing...)

Re:breaks forwarding

[Anonymous Coward]

No it doesn't - SPF works on the _envelope_ headers and the fact that the forwarding machine is declared OK to send. Rewrite the envelope header (the one you never get to see in most mail clients) and make sure forwarding happens on a machine SPF-permitted for the rewritten address (or is routed via a mail gateway permitted by SPF).

Caller ID

[supe]

WTF, I thought I owned the patent on that?

Pobox.com antispam working like gangbusters for me

[DeadVulcan]

Well, I'm a pobox.com customer, and my own experience of their new antispam measures is absolutely nothing but fantastic. They recently overhauled their spam filters, and the result (again, this is just my experience) has been stunning.

Of course, this says little about SPF itself, but at the very least, for what it's worth, the company that invented it comes with my recommendation.

...until 100% compliance is reached, non-SPF mail will still have to be accepted.

Well, the way pobox.com has done it, you can choose to have your E-mail "flagged." SPF is one of those possible flags. If an E-mail gets X (a user-definable number) or more flags, it can be rejected as spam. This makes SPF useful even when there isn't 100% compliance.

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

I would think that if your ISP is interested in doing honest business, they would make the effort to list their own mail server.

If you're running your own mail server, then, yes, this is a valid concern.

The only way to kill spam is to stop having all email be totally, absolutely, "free" of charge in any quantity.

I don't deny that that would be a very effective way, but I don't agree that it is the only way.

Legislation hasn't helped yet

[swb]

Secondly if they have to buy domains they need to pay for them - that leaves a physical paper trail to spammers, now legislation can help.

Sorry, but I have to hit the bullshit button. Legislation hasn't helped yet, and I'm not talking about CAN-SPAM or any of the other anti-bulk mail bills, but the existing laws dealing with all manner of fraud, FDA regulations and any of the other various and sundry state and federal laws regulating the almost-universally fraudulent commercial content of spam.

You're suffering from the same delusion that many people, myself included, often suffer from -- "Can't we pass a *law*"? -- when there are many good laws already on the books that better deal with the problem in general.

I'd suggest a RICO investigation into some of the top-level spammers, their clients, and the people involved in the payments, the network access, and find out how dirty they really are. I don't know, but I suspect, that most of these people know they're involved in deeply fraudulent activity. A few racketeering convictions involving major ISPs, banks, spammers, and their business clients with some noisy investigation of other spammers could have a *real* impact -- squeezing the spammers out of ISP suppliers and banking services they literally can't do business without.

dynamic dns users

[tacocat]

How will this effect dynamic DNS users who send email? I'm not talking about some rogue spammer, but the people who have legitimate servers running on real IP addresses with domain names that are managed by the likes of dyndns.org

In the past, these DHCP hosted addresses have been under a lot of grief with people erroneously RBLing them simply because they are DHCP (like it ever really expires!!!) managed IP addresses.

Much of the workaround for this has been to RELAY all the email up to the ISP for delivery from a non DHCP hosted IP address. But some people block these because they show evidence of being relayed by anyone and hence must be evil.

So what will have to do in order to get my mail server considered acceptable for sending email under this SPF/CallerID scheme?

I'm also really curious to see how this can be a good thing at the same time that it involved Microsoft, but I'm trying to keep an open mind on this one...

Re:dynamic dns users

[ahodgson]

In theory, SPF should make it easier for these people to send E-mail. They can publish a valid SPF record for their domain, which should make mail from their system more trustworthy than mail from dynamic IP space is generally. Ie, the reason people block mail from dynamic IP space is because of the incredible amount of crud coming from trojanned Windows machines in that space.

If a real sender can somehow distinguish themselves via a valid SPF record, they might actually have better luck sending mail than they do now.

Re:Too late !

[ahodgson]

Yes they can. The viruses for the most part aren't sending from the real address of the person whose computer they're on. They're forging other addresses found on the machine as the sender address.

XML ?

[defsdoor]

Wasn't XML last century's buzz word ? The old saying that "when the only tool you have is a hammer - everything looks like a nail" seems totally appropriate.

Except for Microsoft's embraced and extended email

[kyz]

where all you get is (if you're lucky) a text version of the email message, then a WINMAIL.DAT uuencoded or MIMEd attachment, which contains all the useful data in a proprietary binary format.

Rather than simply create compliant MIME mails, Microsoft uses this secret format to say "yeah, we'll try and send email, but if you really want to communicate with companies that use Exchange Mail Server, you need to buy a copy of Exchange Mail Server".

What if.

[man_ls]

What if, say, businesses started showing up promising "unrestricted email" to get around SPF.

They set their SPF to everyone/everyone...or something.

Then it's an open relay with an SPF signature that matches.

and we're back to square 1.

Re:What if.

[taustin]

More likely is that spammers will take the doznes, or hundreds of domain names they register at a time now, and that they run their own DNS for now, and simply automate adding SPF records for, based on their current IP address.

No "email for this domain is allowed to come from these machines" program that is under control of the domain owner has any hope of reducing spam, because spammers will just use it, and keep cycling through disposable domain names at $5-10 each. Only .mail solves that, and that's at the expense of giving someone else control over some of your email settings (and paying them $2000 for the privilige, per domain).

In the short term, if whitelisting based on SPF becomes popular, it will increase the amount of spam coming through. In the long run, it will have zero effect.

Oh, and how to turn it off.

[kyz]

How to actually send compliant, fully functional email instead of encumbered, lock-in Microsoft crap.

Why don't Microsoft set this by default? Email is email. People have got to learn that Microsoft are responsible for this abomination, and the hassle required is Microsoft's fault for not complying to the standard.

I knew it...

[Daneurysm]

Knowing microsoft, they're gunna toss this thing in there as an afterthought...much like real caller-id.

In fact, how surprised would you be if it was just a 1200-baud half duplex signal leading every email?

AOL and MS say: publish SPF records

[wayne]

AOL just added a webpage saying that you should publish SPF records if you want to be whitelisted with AOL.

The MicroSoft Caller-ID/SPF merger proposals say that SPF records will be honored, so you can publish them without fear of losing support.

So, go ahead and publish SPF records.

MicroSoft supporting SPF records is a really smart move. Last week, I posted results of a survey of 1.3 million email domain names to the IETF MARID mailing list. Now that I'm back from the MARID meeting, I just finished a survey of Caller-ID records. There appears to be about a factor of 500-1000 more domains that have published SPF exclusively than Caller-ID exclusively and only a tiny fraction of the 1.3 million domains have published Caller-ID records. In short, MicroSoft isn't changing to support SPF records because they are better (I think they are), but because it is an acknowledgement that MicroSoft's Caller-ID hasn't caught on.

Meng Weng Wong (the SPF author) and MicroSoft are still discussing how exactly this merger will work on. I personally don't see any reason to support XML right away. MicroSoft has not come out with a single concrete extention that can't be done with SPF already.

I also think that there are alternatives to the complex Caller-ID algorithm and that doesn't require every Ezmlm and other mailing lists to upgrade their software. From the research that I've done (and yes, this is something I have really researched), there appears to be far more mailing lists broken by MS's Caller-ID system than email forwarders broken by SPF.

(I'm the author of libspf-alt and the maintainer of the trusted-forwarder global whitelist. So, now you know why I have researched this stuff so much.)

Re:The way I see it...

[randomencounter]

What you are describing is SMTP AUTH. It works nicely for sending e-mail.

SPF is used so that the receiver can verify that the host it is receiving the e-mail from is authorized to do so for the domain, thusly:
SMTP server gets connection from zombie.bigISP.com
zombie claims to be sending mail FROM example.com
example.com's SPF record says that zombie.bigISP.com is not authorized to send mail for example.com.
You get to refuse to accept the mail, mark it as spam, or whatever you please with it.

Simple, eh?
Most importantly, SMTP AUTH makes SPF easier because it lets you have your remote users use your authorized mail servers without making them open relays.

SPF To Be Integrated With MS 'Caller ID' System?

[fox8118]

SPF To Be Integrated With MS 'Caller ID' System

For some reason when I first read the title I thought they were integrating sun block with the email tracking system. I must be tired or just stupid this morning.

Why XML is bad

[tacocat]

It's simple really. DNS is one of the highest areas of traffic and hits out there. Every web page generates multiple DNS hits and so does email and P2P and everything else.

XML, is a bunch of text that wraps around a bunch of data and is called meta data. It's not the data you need, but data about the data you need. In DNS, you already know what you need, so the "meta" is silly.

Point being, you add a lot of extra characters to the data transmissions. UDP won't support it anymore so we have to to with TCP, which has even more overhead being added to the process.

Compound this with MSFT's tendency to send shitloads of data across every network they touch just because they can, and you've DDOSed the Internet.

XML may have a place, but DNS sure as hell isn't it.

Microsoft employing open source ideals?

[PHP Wolf]

From the Caller ID for Spam page:
"Send your comments. We are circulating this initial technical specification for comment because we believe that your feedback can help make it stronger."

Is it just me, or does that process sound dangerously close to open source software development? ;-)

How will the spammers fight back?

[WoodstockJeff]

I take it you either don't operate a mail server, or you don't monitor it. At the present time, over 90% of the spam attempts against our servers come from open proxies around the world, most likely home machines that have been infected by SoBig, Netsky, or whatever worm-of-the-day is installing proxy software at the moment. The spam is coming with envelope senders not associated with spam, in general, because it's forged - heck, lately, nearly 5% of it is coming in with the envelope sender forged to match the mail to address!

The percentage of spam that comes through a server that is associated with the domain cited in the envelope sender field is very low, even those "throwaway" domains you mention. Granted, if SPF catches on, some of the more savvy spammers will use throwaways to associate some of those proxies with, but that increases their cost in time quite a bit, and you can block proxies by other means (dial-up and open proxy lists are available from several sources).