Slashdot Mirror
SPF To Be Integrated With MS 'Caller ID' System
An anonymous reader submits "CNET's news.com is reporting 'An ongoing effort to consolidate antispam authentication schemes took a big step forward with the merging of Sender Policy Framework (SPF) and Microsoft's Caller ID for E-mail.' This is potentially good news." For more background, here are three previous mentions of Microsoft's proposed Caller ID-style system.
XML is not a format. It's a metaformat.
Correct, but what this means is that there should now be some level of accountability to the originator. One of the biggest complaints I would have, looking into email-spam as a problem, would be that there's no way to hold a sender accountable when the true origination of the message is unknown. If I understand the proposal correctly, that accountability will at least be marginally present.
The ability to spoof this system is another issue entirely. The system's intention would appear to add value.
A valid concern; as I read it, this wouldn't be possible. Either that or we're overlooking exactly what makes up a valid original sender.
::jafomatic
Check this article for an interesting commentary on SPF.
Cracker will love the xml format. It turns out that the record size will exceed the UDP packet size for DNS records so they get upsized to use TCP packets.
The thing is how many people allow TCP packets on port 53 on their firewall? There is no reason execpt to talk to your second-dns records. All other cases should be turned off but this requires that it be turned on.
The problem is this. Suppose AOL start adding SPF records to their DNS, saying effectively 'only the following IP addresses are authorized to send @aol.com emails. Suppose also that Hotmail start rejecting emails from SPF domains where the IP addresses don't match. Now suppose that joe@small.biz is going to be away from the office for a couple of weeks, so he gets the small.biz mail server to forward his emails to his hotmail account. At this point anyone from AOL who emails him will find the emails bouncing (although if they're from AOL, this may not be such a bad thing...)
<Conspiracy>Funny, that's exactly the business that SPF's author (pobox.com) is in.</Consipiracy>
The real problem is compliance, until 99% of mail servers provide this data, I can't reject mail from non SPF listed domains.
There aren't many tests which are perfect spam identifiers with no false positives. You should use the SPF compliance as part of a scoring scheme. Messages that fail SPF are more likely to be spam than messages that pass, so they get a higher spam score. If the score exceeds a threshold, mark it as possible spam. If it exceeds a higher one, delete it unread.
This is the strategy Spamassassin allows, and it works really well, especially if you let the scoring be adaptive (i.e. use "Bayesian tests").
Prevent email address forgery. Publish SPF records for y
Use of this technology requires submitting to a Microsoft license. This license allows distribution (but not re-distribution), and is not compatible with the GPL. That is to say, no GPL mail server will ever be able to directly impliment checks for this.
From the license (forgive typos, I typed this from the PDF):
That, my friend, is embrace, extend and assimilate. Nothing under strict GPL can impliment this natively. IIRC, SPF (Sender Permitted From) did not have source restrictive terms.Kinetic stupidity has a new brand leader: Allen Zadr.
An SPF record is a record stating what hosts are registered senders of mail from a given domain. There's nothing stopping you from adding, say, smtp.verizon.net or whatnot to the record.
Of course, there's also SMTP AUTH...
SPF enforces the envelope sender (from the smtp "mail from:" comman", not the header "From:" line. I believe if you look, you'll see that your mailing list mails have the envelope sender set to the mailing list's sender address.
It works just fine.
Uhhhhhhhh, so now everybody has to have both an SPF and an XML parser in their MTA? And that's an improvement ... how?
-russ
Don't piss off The Angry Economist
You don't need to have 99%. You can just delete mail from spoofed SPF-compliant domains, from non-resolving domains, and from domains in blocklists. This will force the spammers to send "from" real, unblocked, non-SPF domains. These will quickly get lots of bounce messages from spam they didn't send and get added to block lists. This will encourage the holdout domain owners to use SPF. Critical mass ensues. All domains will end up using SPF if they want to send mail. If not, they end up in block lists.
Yes they can. The viruses for the most part aren't sending from the real address of the person whose computer they're on. They're forging other addresses found on the machine as the sender address.
How to actually send compliant, fully functional email instead of encumbered, lock-in Microsoft crap.
Why don't Microsoft set this by default? Email is email. People have got to learn that Microsoft are responsible for this abomination, and the hassle required is Microsoft's fault for not complying to the standard.
Does my bum look big in this?
Your objection mentions a multitude of configuration scenarios. I will address a few:
The envelope sender would be mailinglist-owner@mail.sourceforge.net (or similar).
SPF is used so that the receiver can verify that the host it is receiving the e-mail from is authorized to do so for the domain, thusly:
SMTP server gets connection from zombie.bigISP.com
zombie claims to be sending mail FROM example.com
example.com's SPF record says that zombie.bigISP.com is not authorized to send mail for example.com.
You get to refuse to accept the mail, mark it as spam, or whatever you please with it.
Simple, eh?
Most importantly, SMTP AUTH makes SPF easier because it lets you have your remote users use your authorized mail servers without making them open relays.
Forget diamonds, copyright is forever.
The plan appears to be to support both XML and v=spf1 formats (according to the summary of the meeting between the SPF and Microsoft folkw). The expectation is that the vast majority of sites will publish the simple and compact v=spf1 format for the forseeable future. XML allows extensibility for the unforseen future that follows afterwards.
PJRC: Electronic Projects, 8051 Microcontroller Tools
This is the fatal flaw with any kind of mail encryption or signing system. It's not easy to use, and it's not easy to make it easy to use. Someone has to vouch for the fact that you are who you say you are. Current systems do this by having you apply for and receive an X.509 certificate from the likes of Verisign. Everyone trusts Verisign so if Verisign says that I am Jason von Nieda, you can be guaranteed that I am Jason von Nieda. The problem with this, of course, is that before I can use the system I have to go through that process.
If there were a real demand for this, support could be added to mail clients to quickly and easily request a certificate from a certifying authority, but you still have the problems of losing your certificate, forgetting your password, needing to send mail from another computer and so on.
This is why client/personal based authentication fails. Users just generally can't be bothered. Making it the sysadmin's burden, or the ISP's makes it possible to quickly secure and authenticate a large group of people instead of just one person here and there that's willing to spend some time on it.
SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted.
I'll type out an example, to show you exactly how it is already working and rejecting forgeries TODAY, without being mandatory, and long before widespread implementation.
Suppose your MTA receives a message from a spammer who is impersonating me. The message claims to be from "paul@pjrc.com", but it isn't. Your SPF-aware MTA (or spam filter) does a TXT query to my domain, and because my domain is one of the 14500-some returning SPF, you'll get this:
That means there's only two sources that are authorized to send messages for pjrc.com. So your software will first do a lookup on "outgoing.pjrc.com", sorta like this:If the IP number of the MTA sending the message is 168.143.160.91, then you know the message is coming from my server. If not, then the next authorized source is checked. Your software will do a MX lookup, like this:
Then, the name "main.pjrc.com" is looked up, such as:
If the MTA sending the message has the IP number 168.143.173.65, then you also know the message came from an authorized server for my domain, and thus the message is not a forgery by a spammer.
However, the last part of the SPF record was "-all", which means NO OTHER sources are authorized to send pjrc.com's email. So if the MTA sending the message is any IP number other than 168.143.160.91 or 168.143.173.65, then you know it's being sent from an authorized source, and thus it's a forgery (or I've messed up my SPF record).
Please take notice that for this to work, only you and I need to have SPF implemented. All the communication was between your receiving MTA (or SPF-aware spam filter), and my the DNS server for my domain. No other servers or other hosts on the internet needed to implement SPF in order for you to discover that a forged message isn't really from me.
Of course, only about 14500-some domains are currently publishing SPF records... so you can only detect forgeries if the spammer is foolish enough to forge one of our domain names. However, "aol.com", a very commonly forged name, is on the list. Likewise, "hotmal.com" is publishing caller ID (which works more or less the same as SPF, but with different syntax and with the minor distinction of sender vs from).
The bottom line is that your statement is incorrect. SPF does not have to be mandatory to function. It already works for those cases where the receiver bothers to check and the (claimed) sender publishes a SPF record. You can already reject all forgeries from 14500-some domains, including little ones like mine and big ones like AOL and Hotmail... and this is fully functional TODAY, long before 100% compliance is reached.
PJRC: Electronic Projects, 8051 Microcontroller Tools
By extension, then, you figure if only ISP email servers could send mail, spam would be greatly reduced.
Congratulations! You just explained why SPF is a good idea! The whole point of SPF is to point out which servers for a given domain are allowed to send email.
Not heard of SMTP AUTH? Have your ISP setup AUTH on both ports 25 and 587 (MSA, to get around port 25 filters) and you can now relay your mail, via your ISP server, from anywhere.
The percentage of spam that comes through a server that is associated with the domain cited in the envelope sender field is very low, even those "throwaway" domains you mention. Granted, if SPF catches on, some of the more savvy spammers will use throwaways to associate some of those proxies with, but that increases their cost in time quite a bit, and you can block proxies by other means (dial-up and open proxy lists are available from several sources).
If you reject messages without a PGP signature, spammers will simply sign their messages.
If you reject based on the signing author being a known spammer, spammers will simply generate a new key for each message. This isn't a computational burden (as it is in PGP) if the keys aren't generated in a secure manner.
If you reject all unknown senders, people unknown in your "web of trust" will be rejected.
instruct unknown senders (people who aren't in my contact list yet) to download my public key from a specified location to encryp a message that would bypass my filtes.
These challenge-response systems are already in use today, by people willing to be rude to previously unknown senders. They suffer from all sorts of problems (out-of-office autoresponders, automatically generated shipping notices for online purchases, and other legitimate automated senders). How is adding the complexity of PGP an improvement?
The original statement is missing one important word "if [all] mail clients were configured".
This is an idea that just won't work if any significant number of senders still send out plain, old email as they currently do.
Contract with SPF and MS's Caller-ID, which are effective at detecting forgey when only the (claimed) sender publishes a SPF and/or Caller-ID DNS record and the receipient checks it.
To be successful, an change needs to provide value for those who have implemented it, even before widespread adoption occurs. SPF and MS Caller ID meet this goal, as people checking SPF and Caller ID can already reject forgeries claiming to be among 14000-some domain names (including AOL and Hotmail), and domain name admins who publish their SPF and/or Caller ID records are already receiving some protection from having their names forged.
The scheme "let's all switch to PGP" doesn't allow receivers to filter out messages until almost all legit senders sign their messages, and senders who sign their messages don't get much value from the signature until almost all receivers reject unsigned messages. Even then, PGP still suffers from the problem of unknown signatures, requiring a massive worldwide trust database or all the problems of callenge-response negotiation for new contacts.
PGP is an excellent answer to protecting privacy and proving strong authentication. Saddly, crypto only answers the question "is this message certainly authentic", but the question that needs answering is instead "is this message almost certainly a forgery".
PJRC: Electronic Projects, 8051 Microcontroller Tools