Slashdot Mirror
Lead Developer of SPF Anti-Spam Scheme Interviewed
penciling_in writes "CircleID has a great two-part interview with Meng Wong, lead developer of the anti-spam authentication scheme Sender Policy Framework (SPF). He has responded to various questions (which also touches on issues previously raised by Slashdot folks), including the merger with Microsoft's Caller ID, incompatibility of SPF with email forwarding services, and what he thinks about Yahoo's DomainKeys, as well as where he believes the fight against spam is headed. (He has also confirmed that the name SPF and references to sunblock are intentional!) In response to the first question in the interview on how SPF got started, Meng says: 'In 2002 Paul Vixie, the brains behind BIND, wrote a short paper titled 'Repudiating Mail-From'. That inspired two other proposals, 'Reverse MX' by Hadmut Danisch and 'Designated Mailer Protocol' by Gordon Fecyk. In late 2003 I combined the best of both proposals and called the result SPF.' Vixie replies to this reference in comments following the first article."
yeah cool wheeey :)
I use SPF 30 suntan lotion as masturbation lube.
Would we need a new universal mail protocol? Or could we use the method merely by altereing the current SMTP?
(\_/)
(O.o) This is Bunny. (> <)
I think we can all agree that Mercatur is a hottie...
Her current webcam image
Write her a message fellas: Her message board
How long until he is charged for crimes using the anti-patriotic act, and seen as a general threat to free enterprise? This might sound quite negative, but over the last few years the legislation and general tendency of court decisions, as well as in parts of the population anything that restricts the ability to screw the people over, is seen as such.
FRosty Pee Pee
for every article about new anti-spam schemes, I would now be richer than I would have been had I gotten $0.01 for every spam mail urging me to buy penis-enhancing pills.
Your job has been outsourced to a GNOME!
This article advocates a
(x) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
(x) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
(x) Users of email will not put up with it
( ) Microsoft will not put up with it
(x) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
(x) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
(x) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
( ) Jurisdictional problems
(x) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
(x) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
(x) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
(x) Joe jobs and/or identity theft
( ) Technically illiterate politicians
(x) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
(x) Outlook
and the following philosophical objections may also apply:
(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
(x) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(x) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, l0ser! I'm going to find out where you live and burn your house down!
It's better to be the foot on the boot than the face on the pavement. ~~ tkx Kadin2048
My understanding of SPF is this:
the recipient checks that the sender has authoritiy to send out email for the domain, i.e. if you send an email from whatever.com via SMTP server 123.123.123.123, the recipient checks that 123.123.123.123 has the authority to send email for whatever.com by checking it's SPF record (which similar to an ordinary DNS record).
So, we all have to set up SPF records for our domains or our emails will get rejected by some ISPs. Is my understanding right?
IANAL, but the text of this agreement seems to indicate that this implementation license applies to any products that "implement and are compliant with" Sender ID (section 1.2), and that Microsoft may subsequently terminate the license (section 3).
Anybody familiar with this? Is there a RFC for Sender ID?
May do, may not Same worthless Task. Research
Paul Vixie's comment:
I'd have to disagree with Paul Vixie here. Most of the spam today comes through compromised home machines on a broadband line. Of course, spammers could include the zombie they're using in their SPF record and use their own domain in the "Purported Return Address", but that would make them so traceable that they might as well spam from their own systems.
So I'd definitely disagree that SPF/SenderID will not discourage spammers. It will certainly discourage the worst of them: the guys who don't want to be found out.
Great, SPF now not only protects from the sun, but from spam as well. I bet we could get XML involved somehow, too.
OK, I'm worried; unless I've completely missed something here, it seems as though the 'little guy' could get hit quite badly by SPF.
Let me explain; my domain is handled by a hosting provider here in the UK. Because I don't have a static IP address (and also because I don't want the hassle of handling a publicly visible SMTP server), I've set up a single mailbox with the hosting provider that acts as a catch-all account.
Locally, behind my firewall, I use fetchmail to retrieve the contents of this account, and I use qmail to distribute the mail into various IMAP folders; naturally I'm also using ClamAV and SpamAssassin as well.
All well and good, but the problem is that my domain hosting provider does not allow SMTP relay *at all*. Therefore, I use the SMTP relay service provided by my ADSL provider.
Obviously, neither my local qmail system nor my ADSL providers' SMTP relay will be listed in any SPF records; how will I be able to carry on locally managing my mail without automatically being rejected by SPF-aware mail servers?
Life is like a sewer; what you get out of it depends on what you put into it...
explain this to me
if you are going to HAVE to use ESMTP why not add the ability to look up public key for domain ?
if you are doing the domain why not query for user ?
finger server or in DNS record ?
is this in the spec ?
in the future then everyone can use weak crypto for emails and not send everything plain text
(speak to the person in internet cafe or bussiness and they dont understand that their msg is transmited plaintext and maybe through other peoples servers who may or may not read the email )
it would be nice to say we thought of providing keys but people dont have to use them...
regards
John Jones
my webmail rightly lets me send with whatever From: field I choose
So I can emailwise be both at work and at play from the same webmail
for spamming from a zombie (with an IP of 111.222.123.124)
From: zombie@111.222.123.124
Subject: Stop spam now!
or it wouldn't be too much trouble to look up the MX record of 111.222.123.124 and set an appropriate From: header accordingly
This scheme is as temporary as any other and it also prevents me from sending mail with my own computer, I will have to route my mail through my ISP's mail server in order to tag on to their SPF
oh well, something's got to give
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
The machine was taken over from Joe Sixpack in the first place.
He won't know why his box is just a bit slower than usual. Soo he thinks DSL is all bull because is hardly faster than his ol' dial-up.
Get a thousand zombies sending a hundred spam a minute and you see that its not so tough to send a hundred ACK signals as well...
Potential end result: 1,440,000,000 spams/day from ONE infected net.
Go after the spammer's clients instead. Spam and you get jail time and a whopping big fine, paid locally, regardless of the jurisdiction you're in.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
That's interesting! I'd like to plug two bug reports of mine (I wish I had time to hack, but I haven't). Friend-of-a-friend makes great start for a reputation system, at least for whitelisting people you know well.
So, I there's a Spamassassin bug on this, and it has generated some interest.
Now, the problem is to generate FOAF-records easily and reliably, and for that, I suggest for example enabling KAddressbook to export them.
Employee of Inrupt, Project Release Manager and Community Manager for Solid
For a moment, neglect the high costs, complexity, worldwide legality and PKI problems of crypto, which are all solvable at some cost. They aren't the fundamental problem.
Strong authetication answers the question:
That's a very nice to know if you need to place a high level of trust in the message, such as correspondance from your bank. Today email is virually worthless (in the absence of PGP) for trusted messages.
Saddly, the answer to this question is not valuable for filtering out junk. The question that needs to be answered instead is:
PGP does not answer this question. Neither does Yahoo's DomainKeys. There are many circumstances where the signature can not be verified. You can not use the failure to verify a PGP signature as a reliable means to filter out junk.
SPF and MS Caller ID are designed to answer this second question reliably enough to discard forged messages. They answer this question for all domains that publish SPF records (aol.com, gmail.com, etc) regardless of whether SPF is widely adopted by the rest of the world.
When the claimed domain published a list of designated senders, and the sending MTA's IP number isn't among them, you can be certain the message is a forgey and discard it (or close the connection before the data phase, saving bandwidth).
PGP and other crypto signature schemes simply do not deliver this ability to detect forgey. They only detect authenticity.
PJRC: Electronic Projects, 8051 Microcontroller Tools
shemes, the whoreabully infactdead/pateNTdead PostBlock censorship devise, gnu online dating, & a host of other quick buck scams, interview, still forthcoming?
remember, keep it simple. no questions about the monIE, or whatever happened to the won-eyed girl?
the scriptdead 'answers' have already been prepared buy a cubicle full of phonIE ?pr firm? hypenosys talknicians, & will be delivered onto you, when your questions match them acceptabully.
remember, lookout bullow.
consult with/trust in yOUR creators.... self-promoting by natural attraction since/until forever. see you there?
Your envelope FROM is not the same as your From: header. Get that through your skull and then come back.
There's something I don't get: What if the spamming computer sends a mail, but pretends to be relaying it? He could act as if he actually got the mail from someplace else which makes the fake from-address valid in the view of the receiver.
How does that work?
lucas
...i own a domain name. i use it for email forwarding to whatever account i happen to be using, so that i can keep the same email address. this is going to break that, isn't it? i'm just a little guy who doesn't run his own smtp server (i'd have a job: my ISP blocks that anyway...)
SRS/SPF still have a large number of problems to solve. SPF alone is very good idea, but the special-case of mail-forwarding is not compatible with the current design. As SPF breaks forwarding, SRS is an ugly hack which tries to repair the damage done.
I am not convinced that SRS does not introduce ugly bugs, which enables spammers to circumvent SPF alltogether.
Specifically I think SRS can as easily be forged as mails can be forged today, as noone hinders you to fake a forward which hasnt taken place in the first time. If this forward looks ok, you are in.
Now: Forging Spammer -> Recipient
SRS/SPF: Forging Spammer -> Recipient
^
|
Imaginary Origin
Why should this work at all? Please enlighten me.
Meme of the day: I browse "Disable Sigs: Checked". So should you.
Too bad its still broken:
# Its parsing is too complex
# No sane firewall is going to let TXT records through
# No sane firewall is going to let TCP DNS packets through
# The parsing can loop forever
# It will increase DNS scaning as spamers hunt for broken SPF records
# Its too complex to be implimented inside the MTA where it needs to be done
# It can't be properly parsed in sendmail
# ISO 8839 8859 59-15 utf-8 issues for domain names may kill some dns servers
Its a step in the right direction but its the wrong step.
I have people claiming to be from my domain sending spam, so if this will stop or slow them down I'm all for it. I published my records a while ago. It hasn't seemed to make any difference, but maybe it is making a difference and I just don't know it.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
I've seen a lot of heavily virus infected and spyware infected PCs but I have never seen a PC that I knew was zombified as a spam sending monster.
Maybe I just didn't know it.
Has anyone seen one? What applications are running on them? It seems like maybe we could mess with the spammers if we knew what they were using.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
Boycott Caller-ID for E-mail is still wary of the new, combined scheme. Neither Meng nor Microsoft has clarified whether the draconian patent schemes from Microsoft's earlier attempt still exist!
We need to pressure the people coming up with the standards to avoid patented schemes at all costs, or assign the patents to a neutral third-party with the mandate of using them defensively.
I haven't seen any clarification of these patent issues yet, personally. What about the anti-GPL license that CID has? Is that still there?
æeee!
ever heard of key signing ?
so you end up with a web of trust...
SPF and Caller ID are a solution until you end up sending emails from your outlook automatically...
oh wait thats been done before
OpenPGP and email clients that support it, go a long way to solving the problem i.e. set your boundrys
I trust bob
bob trusts jane
jane trusts bart
bart trusts lisa
so I think that I want only 3 degrees and everyone else has to tell me via phone fax and friends they want to email then I trust them and their friends...
works a bit like friendster and gmail invites only way of doing things I can see that will work
regards
John Jones
As for why you'd need a license for this, it may the case that MS has a number of pending patents on the concept (orginally termed Caller ID) and the license mentioned prior is meant to assure people that if this makes it out there as a standard, they will have a license to practice with having to pay royalties.
... they certainly should not be given government monopoly entitlements to the concept, nor if our corrupt government does grant them such entitlements, should anyone respect it. Locate the infrastructure off shore instead.
Rather than give tacit support of software licenses to Microsoft (or anyone else), I'd rather just locate my DNS and mail server overseas, in a sane regime that doesn't recognize software patents, and use SPF irrespective of Microsofts intellectual property grab.
Microsoft obviously didn't invent anything here (the SPF folks did)
The Future of Human Evolution: Autonomy
SPF bascially assumes that if you have chosen the solutions that the grandparent has chosen that you are wrong. You now have to do something else. What that something else is is of no concern to SPF. Deal with it.
As a spammer, this is just a cost of doing business.
I'll apply to become a registrar. I'll stock up on cheap $3 domains, and use one for each few million emails.
My bulletproof DNS host will include SPF records that are completely valid - my peer2peer zombies will let me update the SPF records dynamically.
Shame about not leaving any 2-word domain names for you kind folks. You can have 3-dictionary-word domain names until I get there.
If the zombie is actually using the user's proper (sender) domain and sending through the ISPs mail server, then SPF doesn't help directly, it gives the ISP the power to simply monitor what is sent and shut the zombie down if spam is being sent.
So, SPF contributes nothing. ISPs already have the ability to monitor their clients for spam and shut them down. It also shows that Vixie is not correct about the only benefit he sees for SPF, prevention of domain spoofing. The owner of a zombie machine will send out spam with the targeted domain. Spammers will simply slow the number of messages from each zombie to whatever does not trigger the ISP's monitoring program. The spam will go on and it will come from every big dumb ISP until people quit using crap from M$.
Friends don't help friends install M$ junk.
All your questions are answered in the papers on SRS which were written by a professional cryptographer and security researcher at the University of Bath.
A lot of very smart people just like you have spent a lot of time thinking through the problems. Much of their thinking is captured in essays and FAQs available online.
http://www.libsrs2.org/docs/
If you run your own mailserver, you can setup port 587/tcp, which is the smtp-auth port. Configure your MTA to only accept authorized connections. SASL is very common for this. When your MTA connects it issues a STARTTLS, which encrypts the session, then it authenticates as you, and then your MTA can allow it to relay. The 'traveling executive' problem is now solved.
RFC 2554 for more information.
I'm getting the impression that a lot of the people who post objections haven't bothered to read the FAQ.
I suppose they also write lengthy criticisms of movies based on the teaser trailer and on reading reviews, without actually seeing the movie.
How is SPF going to deal with the Malicious forwarder?
Resent-From: pretendsitslegitforwarder@example.com
From: security@ebay.com
Subject: $I_wanna_steal_your_identity
$Body
Assume that example.com publishes SPF records and that this mail passes that check. The spammer has just passed its authentication check and successfully forged an email.
Lets make this a little more practical. Spammer buys CD of million names.
>grep '@pobox.com'
$id@pobox.com
He then sends mail to $id@pobox.com through his domain (even with valid SPF records). To hide his tracks, he continues forging -- saying that he received the mail from yet another forwarding service (ie, puts in a fake resent-from).
Mail From:
RCPT TO:
Resent-From: $id@pobox.com
Resent-From: spammer
Resent-From: another_forged_legit_domain
From: security@ebay.com
Subject: $I_wanna_steal_your_identity
$Body
Pobox.com receives the mail, validates that the mail is from spammer and forwards it along to the end user. The end user's system verifies pobox's SPF -- bingo, we have gotten a forged email through. SPF delegates the identity check to domains that you do not control. In today's world that is not a good idea.
Today, 60-80% of the mail coming from pobox.com is spam. If the receiver applies pobox's reputation to its mail, then it should reject all its mail. I assume that would not make Meng happy.
More and more people are realizing that authentication is not an anti-spam solution, but that authentication allows reputation and other antispam components to be built on top of it. Unfortunately this is exactly why SPF will fail.
I know that I personally had an exchange with Scott Hazen Mueller (of CAUCE) via email back in 1999 or 2000, proposing this very thing.
Where we use MX records for sending, I proposed an MS record for authorized Mail Senders. That's all it did, but at least you could look to DNS to see if this remote system was actually a designated speaker for that domain.
Of course, he told me it would never work and that it was unreasonable to use DNS to provide such a service.
Oh well...
The general approach taken by SPF and friends does
,
nothing directly to stop SPAM. As numerous other posters
have pointed out, the throw-away domain is still available
to spammers and it is easy for them to generate valid
SPF records for those throw-away domains, even using
zombie networks (with dynamic DNS, you can even limit
the number of IP addresses in the records to mask
the size of the network).
It is useful, though, both immediately and in the medium
term. The immediate usefulness is in bounce processing.
An MTA whose role is to generate or forward bounces
can check the SPF record before action and not send
those for which there is no connection between the
message and the target of the bounce. That's very
useful, since it removes the clutter from the mailboxes
of those who were the targets of joe jobs. (Note that
an MTA forwarding a bounce can do this, as well as
one originating a bounce--that means an enterprise
can do this at the MTAs listed in the MX records).
In the medium term, the SPF record can be used to
limit certain kinds of forgery, making it harder for
social engineering attacks to work. It's not perfect,
by any means, since registration of domains like
"companyname-support.com" will still fool some
users. It also relies to some extent on the first
MTA in a series to perform the checks, as the scheme
gets weaker and weaker with the need to grovel
through more headers to make an association.
In other words, it works well when it is
ORIGINMTA--DESTMTA, but less well when it
is ORGINMTA--IntermediateMTA-IntermediateMTA-DESTMTA
since the RFC 2822 header (From:, Resent-From:, etc.)
processing gets more complicated. Since the latter is a core
part of the design of email (store and forward), there
are limits to SPFs role in this arena--especially when
spammers decide to pass there messages through multiple
MTAs under their own control to generate the appearance
of this complexity.
In the long term, the hope is that it starts folks toward
a more generic mechanism that allows the sending domain
to describe the characteristics of its email (e.g. all mail
is signed as S/MIME and all signatures derive from this CA)
and the receiving domain to check the characteristics
of the messages it's received against those.
Again, though, this is all about forgery, not SPAM.
How is this supposed to stop spam, then? The spammers will simply make sure they use a domain that doesn't have SPF records.
Simply put, it won't stop spam. (There, I've said it.)
It's not supposed to. The goal of SPF (and the other reverse-MX proposals) is to allow a domain owner to put a dent in joe-jobbing by whitelisting all of the mail servers that are authorized to send e-mail on behalf of the domain. As a mail admin, it lets me say: "Don't bother accepting any e-mail purporting to be from my domain unless it comes from machines X, Y, and Z".
It's the natural mirror of the MX record (which specify what machines are authorized to accept incoming mail for my domain).
Wolde you bothe eate your cake, and have your cake?
That won't help the Joe much, when the MTA's block a message due to the SPF, and then the email bounces back to the Joe just the same. Unless the MTA's are changed to NOT send non-delivery-bounces, it will increase the amount of bounces the poor guy receives.
AFAICT, the only thing it can decrease is the amount of delivered forced email, at the expense of increasing bounces and making forwarding and vanity domains behind an uncooperative (read: corporate) ISP impossible.
Unless, of course, everybody starts using web mail, through companies like Microsoft and Pobox.com, of course... Hmmm...
--
*Art
Hey, you can't be new here.
You figure that out WAAAY to fast!
-oo-OO-oo-
The "web of trust" idea is not quite right. Unless you're going to require all incoming email to be signed, it doesn't help you to reject forged messages. Such a bold move would result in most people losing all their incoming email (since it wouldn't be signed).
What you'd need would be a way to look up the sender and see if they always sign their messages. If they do and this message is unsigned, it would be known to be a forgery. This method is not as effective as SPF because each user would have to change their email habits individually and publish their intent to always sign their messages individually (somehow). With SPF, each domain can effect the change for all their users, and (as long as the user's email software is setup right), the users will not have to learn a new habit for sending mail -- things just improve as more and more sites publish SPF records (and check it on delivery).
You'll note that all recent viruses and spam-bots spoof the From header, which means that they would NOT be valid email with SPF enabled for your domain. If virus writers and spammers have to go back to sending their emails with the real domain name of the host they're running on that would be very useful in helping to identify who is responsible for the virus or spam.
Remember, SPF doesn't claim to be a 100% solution to all email problems, nor does it need to be. It slams shut one huge open door in the current protocol, and it does so very effectively. I think it is a very good, targetted solution that will be of great benefit to the net.
..wayne..
So we have the Addressing Spam Channel of the leading Intelligence Hub for The Internet's Core Infrastructure & Policies interviewing the lead developer of the lead antispam solution, and half the comments under the interview are from a spamming all-caps shill who repeats his identical message over and over.
This is...
O
U
T
R
A
G
E
O
U
S
!
Forcing people to use a providers mail server stinks big time. Why should I do that? It compromises my privacy and slows things down. The only people suggesting that as a solution are people that do not run their own mail server at the end of their broad band connection. It's a classic "I'm not using it therefore it's ok to ban it for others".
The very principle of clustering mail from a provider is a little bit stupid as the number of potential mail clients grows. In an idea world, all mail would go directly peer to peer.
- Kim
So what's the story on the patent claims? Boycott Email caller id still seems wary, and I see no evidence that Eben Moglen's concerns (such as incompatibility with the GPL) have been addressed.
Any such patent application is likely to be granted, since Microsoft has lots of $$$ to press their case and the patent office has neither the knowledge or time to determine if they're obvious or in any other way counter them.
I remember the joys of dealing with GIF patents. We're better off without this combined approach if the patent applications will make it unworkable.
So, what's the situation?
- David A. Wheeler (see my Secure Programming HOWTO)
MUDs use many ports, some say to connect with port 23, others say port 25, while more say 1997, 2040, etc. Sometimes people can log into some muds while they can't others. This is because they're using different ports. I think it is a fair assumption to say I am using port 25, 1997, 2040, whatever the mud is telling me to.
And why would I want something more secure then telnet just to play a game? Sheeesh.
Why not set up trust networks a la gpg?
Distribute the standard mail apps with gpg compiled in by default and let each user either import or create a key upon account information setup.
Then, when reading mails, there should be two buttons: trust or untrust sender if the message has been signed. When clicking "trust", a message dialog pops up and informs the user how he can do that, eg. not to trust keys just because he had an email conversion with the sender for some time, but rather make a phone call or exchange keys via secure ways.
I wonder if there is any downside -- except that the OSS community should persuade the user somehow to create and trust keys.