Slashdot Mirror
Lead Developer of SPF Anti-Spam Scheme Interviewed
penciling_in writes "CircleID has a great two-part interview with Meng Wong, lead developer of the anti-spam authentication scheme Sender Policy Framework (SPF). He has responded to various questions (which also touches on issues previously raised by Slashdot folks), including the merger with Microsoft's Caller ID, incompatibility of SPF with email forwarding services, and what he thinks about Yahoo's DomainKeys, as well as where he believes the fight against spam is headed. (He has also confirmed that the name SPF and references to sunblock are intentional!) In response to the first question in the interview on how SPF got started, Meng says: 'In 2002 Paul Vixie, the brains behind BIND, wrote a short paper titled 'Repudiating Mail-From'. That inspired two other proposals, 'Reverse MX' by Hadmut Danisch and 'Designated Mailer Protocol' by Gordon Fecyk. In late 2003 I combined the best of both proposals and called the result SPF.' Vixie replies to this reference in comments following the first article."
No these schemes are designed to work with SMTP.
It would be very difficult to replace something as as popular as SMTP. It would at least have to be backwards compatible with SMTP which means it would expose itself to the same problems as SMTP.
IMO SMTP does a good job apart from its obivous faults.
for every article about new anti-spam schemes, I would now be richer than I would have been had I gotten $0.01 for every spam mail urging me to buy penis-enhancing pills.
My understanding of SPF is this:
the recipient checks that the sender has authoritiy to send out email for the domain, i.e. if you send an email from whatever.com via SMTP server 123.123.123.123, the recipient checks that 123.123.123.123 has the authority to send email for whatever.com by checking it's SPF record (which similar to an ordinary DNS record).
So, we all have to set up SPF records for our domains or our emails will get rejected by some ISPs. Is my understanding right?
IANAL, but the text of this agreement seems to indicate that this implementation license applies to any products that "implement and are compliant with" Sender ID (section 1.2), and that Microsoft may subsequently terminate the license (section 3).
Anybody familiar with this? Is there a RFC for Sender ID?
Paul Vixie's comment:
I'd have to disagree with Paul Vixie here. Most of the spam today comes through compromised home machines on a broadband line. Of course, spammers could include the zombie they're using in their SPF record and use their own domain in the "Purported Return Address", but that would make them so traceable that they might as well spam from their own systems.
So I'd definitely disagree that SPF/SenderID will not discourage spammers. It will certainly discourage the worst of them: the guys who don't want to be found out.
OK, I'm worried; unless I've completely missed something here, it seems as though the 'little guy' could get hit quite badly by SPF.
Let me explain; my domain is handled by a hosting provider here in the UK. Because I don't have a static IP address (and also because I don't want the hassle of handling a publicly visible SMTP server), I've set up a single mailbox with the hosting provider that acts as a catch-all account.
Locally, behind my firewall, I use fetchmail to retrieve the contents of this account, and I use qmail to distribute the mail into various IMAP folders; naturally I'm also using ClamAV and SpamAssassin as well.
All well and good, but the problem is that my domain hosting provider does not allow SMTP relay *at all*. Therefore, I use the SMTP relay service provided by my ADSL provider.
Obviously, neither my local qmail system nor my ADSL providers' SMTP relay will be listed in any SPF records; how will I be able to carry on locally managing my mail without automatically being rejected by SPF-aware mail servers?
Life is like a sewer; what you get out of it depends on what you put into it...
explain this to me
if you are going to HAVE to use ESMTP why not add the ability to look up public key for domain ?
if you are doing the domain why not query for user ?
finger server or in DNS record ?
is this in the spec ?
in the future then everyone can use weak crypto for emails and not send everything plain text
(speak to the person in internet cafe or bussiness and they dont understand that their msg is transmited plaintext and maybe through other peoples servers who may or may not read the email )
it would be nice to say we thought of providing keys but people dont have to use them...
regards
John Jones
my webmail rightly lets me send with whatever From: field I choose
So I can emailwise be both at work and at play from the same webmail
for spamming from a zombie (with an IP of 111.222.123.124)
From: zombie@111.222.123.124
Subject: Stop spam now!
or it wouldn't be too much trouble to look up the MX record of 111.222.123.124 and set an appropriate From: header accordingly
This scheme is as temporary as any other and it also prevents me from sending mail with my own computer, I will have to route my mail through my ISP's mail server in order to tag on to their SPF
oh well, something's got to give
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
For a moment, neglect the high costs, complexity, worldwide legality and PKI problems of crypto, which are all solvable at some cost. They aren't the fundamental problem.
Strong authetication answers the question:
That's a very nice to know if you need to place a high level of trust in the message, such as correspondance from your bank. Today email is virually worthless (in the absence of PGP) for trusted messages.
Saddly, the answer to this question is not valuable for filtering out junk. The question that needs to be answered instead is:
PGP does not answer this question. Neither does Yahoo's DomainKeys. There are many circumstances where the signature can not be verified. You can not use the failure to verify a PGP signature as a reliable means to filter out junk.
SPF and MS Caller ID are designed to answer this second question reliably enough to discard forged messages. They answer this question for all domains that publish SPF records (aol.com, gmail.com, etc) regardless of whether SPF is widely adopted by the rest of the world.
When the claimed domain published a list of designated senders, and the sending MTA's IP number isn't among them, you can be certain the message is a forgey and discard it (or close the connection before the data phase, saving bandwidth).
PGP and other crypto signature schemes simply do not deliver this ability to detect forgey. They only detect authenticity.
PJRC: Electronic Projects, 8051 Microcontroller Tools
Your envelope FROM is not the same as your From: header. Get that through your skull and then come back.
(x) It is defenseless against brute force attacks
Explain: how would you brute force it? One way would be to search until you find a domain without SPF information and fake addresses from that. That will reduce the pool of domains you can fake, and be an incentive for them to start using it. In a way it's shifting the damage over to those that doesn't try to help solving the problem, they decide to be easy targets they take the consequenses.
(x) It will stop spam for two weeks and then we'll be stuck with it
It will stop spam from beeing sent with faked addresses from a domain, if the owners of that domain wishes it. That means I will never se a bounce for spam using my address if the recieving end uses SPF, and the reciever willl not see spam that fakes my address as its sender
(x) Users of email will not put up with it
Why not, all I have to do is configure my mail program to use the correct mailserver for outgoing mail
(x) The police will not put up with it
The police has never cared about anything to do with spam, why should they care about this?
(x) Requires immediate total cooperation from everybody at once
Bull. Mail (allegedly) from domains that doesn't publish SPF information will get through, and mail to recievers that doesn't check SPF will come through. Domain owners will be encouraged to implement this because they will se fever (misdirected) abuse reports. Users will be encouraged to implement this because they will see less spam
(x) Lack of centrally controlling authority for email
The owner of a domain is the central controlling authority for email from that domain, that's all you need
(x) Ease of searching tiny alphanumeric address space of all email addresses
Eventually all spam will use a sender address from domains without SPF informations (or nonexistent domains), people will start dumping mail from domains without SPF (or give it a high spamassassin score) and those domains will effectively be forced to implement SPF or see their users unable to send email
(x) Asshats
Which is why you have to use additional measures, such as scoring mail without SPF low, blacklisting domains and ISPS that allow spammers and other kinds of filters. There is no tool that will block ALL spam, but the right combination will reduce it drastically
(x) Unpopularity of weird new taxes
No tax involved
(x) Huge existing software investment in SMTP
It's compatible (actually uses) SMTP, no software has to be replaced. (Unless it already sucks)
(x) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
And all the mail sent out to SPF using clients using from addresses with SPF domains will be dropped, eventually this number will rise as more people adopt SPF.
(x) Extreme profitability of spam
Making them work harder and reach less people will decrease the profitability, that will make the situation improve
(x) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
Eventually the spammers will have fewer domains to use for sender addresses, they will have to buy domains (increasing cost) or use domains without SPF. Both can be blacklisted
(x) Outlook
This can be implemented serverside, outlook is not an issue
(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
None has ever been standardised and tried large scale
(x) We should be able to talk about Viagra without being censored
You can as long as you don't forge the sender domain. But if you try to sell it to someone a complaint may well make you lose access to that domain
(x) Countermeasures must work if phased in grad
- We are the slashdot. Resistance is futile. Prepare to be moderated -
SPF is not an anti-spam system. SPF is an anti-joe-job system. It happens that the most frequent joe jobs tend to come from spammers, however:
1. Not all joe jobs are spam.
2. Most spam (looking at my bulk mail folder on Yahoo) doesn't involve joe jobs.
A number of people are equating this with anti-spam systems, and that's just wrong. I thought one of the most revealing answers in the interview above was:
What's signficant about this answer? It's that it doesn't answer the question. More specifically, it answers the first part of the question, but (rightfully) ignores the second part, because the second part of the question is a "Why are you still beating your wife?" question (a question based upon a false assumption.)Proper anti-spam systems are based upon dealing with unwanted email, not dealing with non-relevent characteristics. If you create a system that makes it easier to control who gets your email address (such as my solution, Yahoo!'s AddressGuard(tm), and TMDA), then you're implementing a relevent solution, because dealing with spam is about controlling who sends you email. Likewise, if you operate Bayesian, CRM114, Mail.app, etc, AI filters based upon spam, then you're coming up with a relevent (if imperfect) solution, and the solution can work if combined with a whitelist, especially if you can automate the generation of the whitelist through systems like Camram.
SPF isn't such a system, it's designed to deal with a totally different issue. Arguably, given that it breaks mailing lists and forwarding and has many other documentable faults, it really ought to be being used as a measure of last resort, though it's a good idea for mail receipients to implement the logic so that domains that are having these issues can make the choice when they find themselves being targetted.
It'd be nice if every solution to every problem on the net wasn't always promoted as an anti-spam solution...
You are not alone. This is not normal. None of this is normal.
SRS/SPF still have a large number of problems to solve. SPF alone is very good idea, but the special-case of mail-forwarding is not compatible with the current design. As SPF breaks forwarding, SRS is an ugly hack which tries to repair the damage done.
I am not convinced that SRS does not introduce ugly bugs, which enables spammers to circumvent SPF alltogether.
Specifically I think SRS can as easily be forged as mails can be forged today, as noone hinders you to fake a forward which hasnt taken place in the first time. If this forward looks ok, you are in.
Now: Forging Spammer -> Recipient
SRS/SPF: Forging Spammer -> Recipient
^
|
Imaginary Origin
Why should this work at all? Please enlighten me.
Meme of the day: I browse "Disable Sigs: Checked". So should you.
# Its parsing is too complex
7 101"
Complex or not, it's working just fine with quite an array of software.
# No sane firewall is going to let TXT records through
A firewall would need to examine the contents of packets to differentiate a TXT record from a, say, A record or cname. Comparable wizardry is already being performed by mail servers world wide, on a vast scale:
[smegma@cartman smegma]$ host -t txt 84.137.116.38.sbl.spamhaus.org
84.137.116.38.sbl.spamhaus.org text "http://www.spamhaus.org/SBL/sbl.lasso?query=SBL1
# No sane firewall is going to let TCP DNS packets through
SPF does not rely on anything that every other app using DNS doesn't. Also see above.
# The parsing can loop forever
In a horribly written parser perhaps. The same could be said about IRC clients, Web Browsers and just about any application out there.
# It will increase DNS scaning as spamers hunt for broken SPF records
DNS is quite efficient. Unlike RBLs, SPF will work just fine with traditional SOA settings, so cache hits will be plentiful.
# Its too complex to be implimented inside the MTA where it needs to be done
Just where do you think it's already being implemented?
# It can't be properly parsed in sendmail
It is already being used with all popular MTAs, including sendmail, postfix, qmail, exim, courier and ms exchange.
# ISO 8839 8859 59-15 utf-8 issues for domain names may kill some dns servers
Huh?
Parsing complexity might become a bit of a concern with the advent of XML, but as of now, it's dead-simple.
3, Interesting? And I feel like I'm feeding a troll here!
ever heard of key signing ?
so you end up with a web of trust...
SPF and Caller ID are a solution until you end up sending emails from your outlook automatically...
oh wait thats been done before
OpenPGP and email clients that support it, go a long way to solving the problem i.e. set your boundrys
I trust bob
bob trusts jane
jane trusts bart
bart trusts lisa
so I think that I want only 3 degrees and everyone else has to tell me via phone fax and friends they want to email then I trust them and their friends...
works a bit like friendster and gmail invites only way of doing things I can see that will work
regards
John Jones
As for why you'd need a license for this, it may the case that MS has a number of pending patents on the concept (orginally termed Caller ID) and the license mentioned prior is meant to assure people that if this makes it out there as a standard, they will have a license to practice with having to pay royalties.
... they certainly should not be given government monopoly entitlements to the concept, nor if our corrupt government does grant them such entitlements, should anyone respect it. Locate the infrastructure off shore instead.
Rather than give tacit support of software licenses to Microsoft (or anyone else), I'd rather just locate my DNS and mail server overseas, in a sane regime that doesn't recognize software patents, and use SPF irrespective of Microsofts intellectual property grab.
Microsoft obviously didn't invent anything here (the SPF folks did)
The Future of Human Evolution: Autonomy
If you don't flag anything that doesn't match as a fail (-all at the end), then you won't see much difference. It also depends on the server receiving such spoofed emails actually doing checks against SPF records.
Most domains out there are defaulting to "neutral" or "softfail" as the default in their SPF records for now until sites that do mail forwarding or web mail (preserving the original envelope FROM address) implement SRS. If you have fairly solid control over where mail from your domain comes from, then you can go ahead and set the default to "fail".
All I want is a kind word, a warm bed and unlimited power.
All your questions are answered in the papers on SRS which were written by a professional cryptographer and security researcher at the University of Bath.
A lot of very smart people just like you have spent a lot of time thinking through the problems. Much of their thinking is captured in essays and FAQs available online.
http://www.libsrs2.org/docs/
If you run your own mailserver, you can setup port 587/tcp, which is the smtp-auth port. Configure your MTA to only accept authorized connections. SASL is very common for this. When your MTA connects it issues a STARTTLS, which encrypts the session, then it authenticates as you, and then your MTA can allow it to relay. The 'traveling executive' problem is now solved.
RFC 2554 for more information.
Of course, he told me it would never work and that it was unreasonable to use DNS to provide such a service.
He's also dead-on right. There are many ways to simply bypass SPF; SPF-like schemes impose a number of limitations on regular email use; the use of DNS is a major part of the problem in SPF, as it's entirely inappropriate for a trusted transport.
The only reason anyone's deploying SPF, broken as it is, is because people will do *anything* at this point if someone promises them *any* reduction in spam -- look at the RIAA, and how they keep trying copy-protection scheme after copy-protection scheme. They've had PhDs doing studies come forward and say "there isn't any feasible way to copy-protect simple stereo audio as you're currently selling it", but they so badly want to believe that they keep buying into the latest scheme. ISPs get a huge amount of flak from customers over spam, and are more than willing to take even idiotic and nonfunctional measures if it (a) lets them believe that they're doing something that will be long-term effective and (b) lets their marketing people have something to trumpet about ("Earthlink's advanced spam-blocking technology")
May we never see th
So what's the story on the patent claims? Boycott Email caller id still seems wary, and I see no evidence that Eben Moglen's concerns (such as incompatibility with the GPL) have been addressed.
Any such patent application is likely to be granted, since Microsoft has lots of $$$ to press their case and the patent office has neither the knowledge or time to determine if they're obvious or in any other way counter them.
I remember the joys of dealing with GIF patents. We're better off without this combined approach if the patent applications will make it unworkable.
So, what's the situation?
- David A. Wheeler (see my Secure Programming HOWTO)