SPF is easy for sending domains to implement, which is one of the reasons it's
becoming popular. During the last six months we've seen a major increase
in the number of domains that use SPF (including many of the big ones) as well
as an increase in the percentage of messages we receive from an SPF-protected domain.
As far as its effectiveness goes, in one analysis where we sampled a set of messages in which the purported sender's
domain was that of a major ISP, we found that if the SPF authentication check returned
'softfail', the probability of the message being junk was near 100%. When we checked
our MTAs "Received" headers, they indicated that the messages were being sent from
IP addresses in different countries and domains (as one would expect). Of those
messages that passed, only about 30% of the messages were junk. Clearly there is
'signal' in the SPF score.
Interestingly, of those messages that passed and yet were junk (those that composed the 30%), all appeared
to be sent by a legitimately registered user at the ISP. This is the double-edged sword
of authenticating your messages if you are an ISP: if your own user base is sending
junk, other ISPs and recipients will be able to figure it out. And you might be
perceived poorly.
Yet this is exactly what should happen; it's the point of authentication. There should be
motivation for ISPs, either financial or brand-related (which ends up being financial),
to establish and operate procedures that screen members or deter them from sending unwanted messages.
Reputation (or concern of damage to it) is a great motivation.
The real promise in sender-authentication though is DKIM. While SPF is easier to implement for senders than DKIM, SPF
is rather fragile; it doesn't survive forwarding without re-writing the envelope-from. Too few systems
are set up to do this (list management software is the exception), and although changing the
behavior of MTAs is just software, doing so will effect the efficiency of status (bounced mail)
reporting. Messages that would be delivered 'point to point' in the past end up being
'source routed' with many unnecessary hops, increasing the odds of failure. DKIM is a little
more involved to set up, but doesn't have these fragility issues (setting up checks when receiving is about the same level of difficulty for SPF and DKIM).
At Boxbe, we check both DKIM and SPF. The reason is that strong sender identity
gives a pre-approval policy its teeth. We quarantine messages which fail EITHER form
of authentication, but because DKIM is "forward-friendly" and SPF is not, if a message
passes a DKIM check but fails an SPF check, we let the message pass (according to our
member's preferences).
Using both has merit as each type is a little different. Gmail has been
signing/authenticating with both DKIM and SPF for quite some time. We also use
both forms of authentication when we send out messages or forward messages to our members.
As other organizations adopt sender authentication (Comcast
has announced it is implementing DKIM by year end)
it will become a very effective tool.
The FUSSP assumes that your attention is so important that strangers will pay money to send you mail.
needs to be put aside for good.
The post office reports that in 2002, $46 billion was spent on direct mail campaigns. Each item of real junk mail that you receive in your US Mail box costs money to send (typically greater than $0.35 each). There are companies out there that will happily pay for your attention - it's just that right now, you never see any of the payment since it all goes to the post office or the printers.
See section 7.4 of the Q and A for the full details.
While strong authentication would eliminate alot of spam, it has some other undesirable properties. The first is a loss of anonymity. Also, with just a whitelist, how will unlisted senders get through to you (like a friend who had to change their email address)? If you use a greymail box, you still end up having to look at email from unknown senders - so spammers can continue to reach you. See section 5.1 in the Q and A for a summary.
Strong identities are an important part of any realistic spam solution, but not necessarily tying the digital id to the person's real-world id (aka authentication). However, strong authentication alone is insufficient to solve spam because of the problem of first-contact.
The protocol intro and the short FAQ are now posted.
We're going to sort through everything posted here too and incorporate it. Thanks/. for some excellent comments!
Hi, I'm one of the authors of the paper mentioned in this post. We have a short summary of reasoning behind the design posted here
It is a little less dense than the SSRN paper.
Also, I'll get a protocol diagram up shortly, and a short FAQ, linked from the one pager.
Microsoft seems to be pursing the use of sender-warranties (for first time contact) as a means of ending spam.
We have researched this concept from the perpective of the economics of
information, and the results might be of interest to the readers of this thread. Our work was recently described Jan 15th in the WSJ,
and was presented at the MIT spam conference on the 16th of Jan.
Results are publically available on SSRN (Social Science Research Network).
We really think that this is
the right way to do it; it restores control to the individual recipient,
halts spam, and creates a market for information exchange. (We'd advocate using an open standard and multiple competitive bond underwriters).
Slashdot Mirror
As far as its effectiveness goes, in one analysis where we sampled a set of messages in which the purported sender's domain was that of a major ISP, we found that if the SPF authentication check returned 'softfail', the probability of the message being junk was near 100%. When we checked our MTAs "Received" headers, they indicated that the messages were being sent from IP addresses in different countries and domains (as one would expect). Of those messages that passed, only about 30% of the messages were junk. Clearly there is 'signal' in the SPF score.
Interestingly, of those messages that passed and yet were junk (those that composed the 30%), all appeared to be sent by a legitimately registered user at the ISP. This is the double-edged sword of authenticating your messages if you are an ISP: if your own user base is sending junk, other ISPs and recipients will be able to figure it out. And you might be perceived poorly.
Yet this is exactly what should happen; it's the point of authentication. There should be motivation for ISPs, either financial or brand-related (which ends up being financial), to establish and operate procedures that screen members or deter them from sending unwanted messages. Reputation (or concern of damage to it) is a great motivation.
The real promise in sender-authentication though is DKIM. While SPF is easier to implement for senders than DKIM, SPF is rather fragile; it doesn't survive forwarding without re-writing the envelope-from. Too few systems are set up to do this (list management software is the exception), and although changing the behavior of MTAs is just software, doing so will effect the efficiency of status (bounced mail) reporting. Messages that would be delivered 'point to point' in the past end up being 'source routed' with many unnecessary hops, increasing the odds of failure. DKIM is a little more involved to set up, but doesn't have these fragility issues (setting up checks when receiving is about the same level of difficulty for SPF and DKIM).
At Boxbe, we check both DKIM and SPF. The reason is that strong sender identity gives a pre-approval policy its teeth. We quarantine messages which fail EITHER form of authentication, but because DKIM is "forward-friendly" and SPF is not, if a message passes a DKIM check but fails an SPF check, we let the message pass (according to our member's preferences). Using both has merit as each type is a little different. Gmail has been signing/authenticating with both DKIM and SPF for quite some time. We also use both forms of authentication when we send out messages or forward messages to our members.
As other organizations adopt sender authentication (Comcast has announced it is implementing DKIM by year end) it will become a very effective tool.
--
Thede Loder
E: thede@boxbe.com
The FUSSP assumes that your attention is so important that strangers will pay money to send you mail.
needs to be put aside for good.
The post office reports that in 2002, $46 billion was spent on direct mail campaigns. Each item of real junk mail that you receive in your US Mail box costs money to send (typically greater than $0.35 each). There are companies out there that will happily pay for your attention - it's just that right now, you never see any of the payment since it all goes to the post office or the printers. See section 7.4 of the Q and A for the full details.
Strong identities are an important part of any realistic spam solution, but not necessarily tying the digital id to the person's real-world id (aka authentication). However, strong authentication alone is insufficient to solve spam because of the problem of first-contact.
The protocol intro and the short FAQ are now posted. We're going to sort through everything posted here too and incorporate it. Thanks /. for some excellent comments!
Thede Loder
University of Michigan.
We have researched this concept from the perpective of the economics of information, and the results might be of interest to the readers of this thread. Our work was recently described Jan 15th in the WSJ, and was presented at the MIT spam conference on the 16th of Jan. Results are publically available on SSRN (Social Science Research Network).
We really think that this is the right way to do it; it restores control to the individual recipient, halts spam, and creates a market for information exchange. (We'd advocate using an open standard and multiple competitive bond underwriters).